Domain-based Routing
Time estimate: 30 minutes
This exercise is split into three tasks. Initially, you will investigate the current configuration of the EDGE appliance of Stabank (task 1). Then, you will configure the EDGE appliance of Stabank (task 2) to enable communication with remote workers. Finally, you will verify that a user in Webspeed’s network can access the service exposed by Stabank (task 3).
Overview
Multiple ISD setup
For the purpose of this exercise, we have modified the topology to include another ISD.
When using the scion tool suite, you need to specify the ISD-AS you want to work with by using the
--isd-as flag.
For example, to work with ISD-AS 2-ff00:1:1, use:
scion showpaths 2-ff00:1:3 --isd-as 2-ff00:1:1
Refer to the diagram below, which visualizes the network topology we work on in this hands-on session. The depicted infrastructure consists of two ISDs.
TODO misd-topology.drawio.svg diagram
ISD 1 is the Finance ISD and has three ASes:
- Webspeed (ISD-AS 1-ff00:1:1)
- Corpbank Switzerland (ISD-AS 1-ff00:1:2)
- Stabank Private Banking (ISD-AS 1-ff00:1:3)
ISD 2 is the Switzerland ISD and has two ASes:
- Webspeed (ISD-AS 2-ff00:1:1)
- Stabank Private Banking (ISD-AS 2-ff00:1:3)
We can see that two of the organizations belong to two different ISDs at the same time. This is a common setup in SCION networks. Each ISD has a different governing entity, defines a trust root and can have separate membership requirements.
From the perspective of SCION and remote entities, each organization will be treated as two distinct entities in the network depending on the ISD. It is, in fact, possible to define different policies towards the same organization for the different ISDs to which it belongs.
The Webspeed AS consists of three sites in Zurich, Geneva, and Lugano. Each of these sites has one
CORE instance. The CORE hosts are called core.zurich.webspeed, core.geneva.webspeed, and
core.lugano.webspeed, respectively. The Zurich site also has a GATE instance called
gate.zurich.webspeed. The GATE appliance is connected to the BGP network of Webspeed with a
private BGP peering. The purpose of this connection is for the GATE to learn the IP prefixes which
are reachable in Webspeed's BGP network and for the BGP network to learn the prefixes which are
reachable in the SCION network. The GATE is also connected to the EDGE appliance of Stabank, using
the IP-in-SCION tunneling functionality. The Webspeed AS is a core AS in both ISDs.
TODO fig/gate.drawio.svg diagram
The Corpbank Switzerland AS has two sites, one in Geneva and one in Zurich. Each of them has a host,
respectively called edge.zurich.corpbank and edge.geneva.corpbank. Furthermore, the Stabank
Private Banking AS includes only one site, in Lugano, with one host, called edge.lugano.stabank.
They are both leaf ASes and each of them is connected to the Webspeed AS via two links, as depicted
in the diagram. In this topology, the same physical links are reused for connection within both
ISDs. However, the SCION interfaces are treated as separate logical links depending on the ISD to
which they belong.
Over the course of this lab, you will be working in a cloud-hosted playground of the SCION infrastructure. All the ASes run in a virtualized environment on a cloud machine.
Routing Domains
As we saw in the IP-in-SCION Tunneling: Basics exercise, domains are an essential part of the IP-in-SCION tunneling configuration. They allow an IP-in-SCION tunneling endpoint, such as an Anapaya EDGE or an Anapaya GATE, to exchange prefix information with remote entities, enabling end-to-end communication. An IP-in-SCION tunneling endpoint can announce different prefixes to different remote entities. Additionally, they can accept different received prefixes depending on the remote entity. This enables the IP-in-SCION tunneling endpoint to have full control over which prefixes/services are reachable to which remote entities.
Another benefit of domains is that they facilitate the definition of desired communication partners without having to specify how the different entities are interconnected. With domains, you can specify the necessary information for end-to-end communication, e.g. collection of remote user accesses. The details of how the remote users are connected to the organization are tackled separately according to the SCION links defined on the IP-in-SCION tunneling endpoint and the SCION backbone.
For further information on domains and on how the protocol works in detail, you can refer to our documentation.
Preparation
This step is required to follow the hands-on session.
From the cloud machine, run the setup script:
operator@training:~/workspace$ ./appliance_domain_based_routing_exercise setup
After the tasks in this exercise are completed, the topology should be in a working state. However, in case something is not working properly and you need to revert the changes, you can run the following command:
operator@training:~/workspace$ ./appliance_domain_based_routing_exercise restore
Unless stated otherwise, all commands are assumed to be run from the
workspace directory on your training VM. The built-in terminal in
the editor will put you automatically in the right
directory. To open the built-in terminal use the Ctrl+` shortcut.
Alternatively, you can click the Menu button in the top left, then
select View -> Show Terminal . This will bring up the terminal and puts
you in the correct working directory (~/workspace) for all the tasks in
this training.
Task 1. Investigate the current configuration
In the first task, you will familiarize yourself with the current configuration of the Stabank EDGE appliance.
First, download the configuration file for the host edge.lugano.stabank and open it by running the
commands:
operator@training:~/workspace$ appliance-cli context select edge.lugano.stabank
operator@training:~/workspace$ appliance-cli get config > edge.lugano.stabank.appliance.json
operator@training:~/workspace$ code-server edge.lugano.stabank.appliance.json
Inspect the scion-tunneling section of the appliance configuration.
You can also use the appliance CLI and the debug/scion-tunneling/domains/config to get the domain
configuration of the EDGE:
operator@training:~/workspace$ appliance-cli get debug/scion-tunneling/domains/config
How many domains are part of the domain configuration and what are their names?
Solution
There is one domain configured and it is named corpbank.
What is the local ISD-AS in the configured domains? With what remote entity do they enable communication?
Solution
The local ISD-AS is 1-ff00:1:3. The corpbank domain configures communication with the Corpbank AS
in ISD 1 (1-ff00:1:2).
What prefixes are exchanged with the remote communication partner?
Solution
Stabank announces prefix 10.8.0.0/24 to Corpbank and accepts prefix 10.2.0.0/24 from Corpbank.
With what prefixes is Stabank allowed to communicate in ISD 2?
Solution
Stabank has no domain configured where the local or remote ISD-AS belongs in ISD 2. Therefore, there are no IP-in-SCION tunneling sessions in ISD 2.
Task 2. Configure the EDGE
As part of this task, Stabank wants to connect its remote workers that are Webspeed customers to Stabank's VPN gateway. You will configure the Stabank EDGE to allow remote workers to connect to Stabank's office through the Swiss ISD.
Stabank's VPN gateway is reachable at the IP address 198.51.100.10/28, as observed from the topology diagram above. Webspeed users are part of Webspeed's BGP network, therefore part of the IP prefix 192.0.2.0/24. Additionally, the connection between Stabank and Webspeed users is not related to the Finance network, but instead to the Swiss network - the Webspeed GATE is only available in the Swiss ISD.
Based on the above information, you will need to add a new domain through which the EDGE announces the prefix 198.51.100.0/28 to Webspeed's AS in ISD 2. Stabank should also accept all announced prefixes from Webspeed in ISD 2.
Update the scion_tunneling section of the appliance configuration and configure a new domain using
the above information.
First, add the Webspeed ISD-AS number of ISD 2 to the remotes list:
Solution
{
"isd_as": "2-ff00:1:1"
}
Then, add a domain called user-remote-access, where the remote ISD-AS is Webspeed's ISD-AS number
in ISD 2 that accepts all prefixes. You also need to add a traffic policy to the domain; you can
reference the existing traffic matchers and path filters. In a real deployment setup, you may need
to define custom traffic policies.
Solution
Upload the updated configuration to the EDGE appliance:
operator@training:~/workspace$ appliance-cli put config <edge.lugano.stabank.appliance.json
Task 3. Test Connectivity
In this task, you will test the connectivity between the Stabank EDGE appliance and the Webspeed GATE appliance. First, you will make sure the running configurations on the EDGE and GATE appliances are as expected. Then, you will perform an end-to-end connectivity test.
Investigate the running configurations
You can also use Grafana to see if your appliance is correctly configured. For this, open the IP-in-SCION tunneling dashboard.
On the Stabank EDGE, check whether the GATE of Webspeed has been discovered by the SCION Gateway Routing Protocol:
operator@training:~/workspace$ appliance-cli get debug/scion-tunneling/discovery
{
"sessions": [
{
"last-success": "2023-06-26T11:25:57Z",
"local-isd-as": "1-ff00:1:3",
"path": "1-ff00:1:3 1>2 1-ff00:1:1 1>1 1-ff00:1:2",
"peers": [
{
"control": "10.2.0.1:30256",
"data": "10.2.0.1:30056",
"probe": "10.2.0.1:30856"
},
{
"control": "10.2.0.2:30256",
"data": "10.2.0.2:30056",
"probe": "10.2.0.2:30856"
}
],
"remote-isd-as": "1-ff00:1:2"
},
{
"last-success": "2023-06-26T11:25:58Z",
"local-isd-as": "2-ff00:1:3",
"path": "2-ff00:1:3 2>4 2-ff00:1:1",
"peers": [
{
"control": "10.1.0.5:30256",
"data": "10.1.0.5:30056",
"probe": "10.1.0.5:30856"
}
],
"remote-isd-as": "2-ff00:1:1"
}
]
}
Check that the Stabank EDGE is announcing the correct prefix to the Webspeed GATE and that it is also receiving prefixes from Webspeed. You can check this by running:
operator@training:~/workspace$ appliance-cli inspect tunneling summary --domain user-remote-access
DOMAIN: user-remote-access
PREFIXES: 192.0.2.0/24
TRAFFIC MATCHER: match-all-traffic-matcher
PATH FILTER: allow-all-path-filter
REMOTE: 2-ff00:1:1,10.1.0.5:30856
STATE LATENCY JITTER DROPS EXPIRY PATH
--> alive 5.55ms 1.98ms 0.00% 5h57m40s 2-ff00:1:3 1>2 2-ff00:1:1
In real deployment setups, there are usually more prefixes received in IP-in-SCION tunneling
sessions. If you want to specify the number of prefixes that should be included in the output, you
can use the --prefixes flag, such as:
operator@training:~/workspace$ appliance-cli inspect tunneling summary --prefixes 1000 --domain user-remote-access
Additionally, list the available paths for the user-remote-access domain:
operator@training:~/workspace$ appliance-cli inspect tunneling summary --all-paths --domain user-remote-access
DOMAIN: user-remote-access
PREFIXES: 192.0.2.0/24
TRAFFIC MATCHER: match-all-traffic-matcher
PATH FILTER: allow-all-path-filter
REMOTE: 2-ff00:1:1,10.1.0.5:30856
STATE LATENCY JITTER DROPS EXPIRY PATH
--> alive 5.73ms 2.00ms 0.00% 5h57m23s 2-ff00:1:3 1>2 2-ff00:1:1
alive 5.55ms 1.99ms 0.00% 5h57m26s 2-ff00:1:3 2>4 2-ff00:1:1
Now, let's check that the Webspeed side of the connection is working as expected.
First, check if the Stabank EDGE has been discovered by the Webspeed GATE:
You will need to update the selected context for the appliance-cli. You can do so by running the command:
operator@training:~/workspace$ appliance-cli context select gate.zurich.webspeed
operator@training:~/workspace$ appliance-cli get debug/scion-tunneling/discovery
{
"sessions": [
{
"last-success": "2023-06-26T12:31:44Z",
"local-isd-as": "1-ff00:1:1",
"path": "1-ff00:1:1 3>2 1-ff00:1:2",
"peers": [
{
"control": "10.2.0.2:30256",
"data": "10.2.0.2:30056",
"probe": "10.2.0.2:30856"
},
{
"control": "10.2.0.1:30256",
"data": "10.2.0.1:30056",
"probe": "10.2.0.1:30856"
}
],
"remote-isd-as": "1-ff00:1:2"
},
{
"last-success": "2023-06-26T12:31:44Z",
"local-isd-as": "2-ff00:1:1",
"path": "2-ff00:1:1 2>1 2-ff00:1:3",
"peers": [
{
"control": "10.8.0.1:30256",
"data": "10.8.0.1:30056",
"probe": "10.8.0.1:30856"
}
],
"remote-isd-as": "2-ff00:1:3"
}
]
}
Check if the GATE is announcing the prefixes and receiving the correct prefixes from the Stabank EDGE:
operator@training:~/workspace$ appliance-cli inspect tunneling summary --domain stabank
DOMAIN: stabank
PREFIXES: 198.51.100.0/28
TRAFFIC MATCHER: match-all-traffic-matcher
PATH FILTER: allow-all-path-filter
REMOTE: 2-ff00:1:3,10.8.0.1:30856
STATE LATENCY JITTER DROPS EXPIRY PATH
--> alive 11.29ms 3.11ms 0.00% 5h55m16s 2-ff00:1:1 2>1 2-ff00:1:3
In real deployment setups, there are usually more prefixes received in IP-in-SCION tunneling
sessions. If you want to specify the number of prefixes that should be included in the output, you
can use the --prefixes flag, such as:
operator@training:~/workspace$ appliance-cli inspect tunneling summary --prefixes 1000 --domain user-remote-access
Perform the end-to-end connectivity test
Now, you can perform the end-to-end connectivity test. For this, you will use the ping command for
simply testing the reachability.
Log into the endhost which is part of the Webspeed BGP network. It simulates a residential customer of Webspeed. From this endhost, the services exposed by Stabank in ISD 2 should be reachable via the GATE:
operator@training:~/workspace$ lxc shell endhost-zurich-bgp-webspeed
root@endhost-zurich-bgp-webspeed:~# ping 198.51.100.10 -c 10
PING 198.51.100.10 (198.51.100.10) 56(84) bytes of data.
64 bytes from 198.51.100.10: icmp_seq=2 ttl=60 time=61.4 ms
64 bytes from 198.51.100.10: icmp_seq=3 ttl=60 time=40.2 ms
64 bytes from 198.51.100.10: icmp_seq=4 ttl=60 time=41.8 ms
However, services exposed by Stabank in ISD 1 towards Corpbank should not be reachable:
root@endhost-zurich-bgp-webspeed:~# ping 10.8.0.2 -c 10
PING 10.8.0.2 (10.8.0.2) 56(84) bytes of data.
^C
--- 10.8.0.2 ping statistics ---
8 packets transmitted, 0 received, 100% packet loss, time 7153ms
You can also view the traffic flow on Grafana. For this, open this IP-in-SCION tunneling dashboard.
Similarly, an employee at Corpbank can reach the services exposed by Stabank in ISD 1.
Log into the endhost of Corpbank and use the ping command to test the connectivity to Stabank's workload in ISD 1:
operator@training:~/workspace$ lxc shell endhost-zurich-corpbank
root@endhost-zurich-corpbank:~# ping 10.8.0.2 -c 10
PING 10.8.0.2 (10.8.0.2) 56(84) bytes of data.
64 bytes from 10.8.0.2: icmp_seq=1 ttl=61 time=47.9 ms
64 bytes from 10.8.0.2: icmp_seq=2 ttl=61 time=38.3 ms
64 bytes from 10.8.0.2: icmp_seq=3 ttl=61 time=37.0 ms
However, Stabank's services that are exposed in ISD 2 should not be reachable:
root@endhost-zurich-corpbank:~# ping 198.51.100.10 -c 10
PING 198.51.100.10 (198.51.100.10) 56(84) bytes of data.
--- 198.51.100.10 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5117ms