IP-in-SCION tunneling (v0.41 and newer)
The Configuration reference section provides the full configuration reference. Overview provides a summary of the IP-in-SCION tunneling fundamentals. Configuration examples provides examples for configuring each individual subsection of the IP-in-SCION tunneling configuration. Finally, in Use cases, we provide real-world scenarios where the IP-in-SCION tunneling is used and provide full configurations for those scenarios.
Configuration reference
Anapaya appliance configuration (scion_tunneling only)
scion_tunneling object
Top-level configuration and state for IP-in-SCION tunneling.
domains object[]
List of domains that define the rules by which IP packets are routed. A domain is a subset of the IP space that shares the same policies.
Whether this domain is the default domain. The default domain is assumed to accept the whole IP space that is not covered by other domains. Because of this it may not specify an accept-filter.
Optional description, or comment, for the domain.
The domain. It matches all packets and allows any
path to be used.Whether the domain should be disabled.
falseThe payload encryption mode for the domain.
Possible values: [DISABLED, ENABLED, OPTIONAL]
List of local ISD-AS identifiers that belong to this domain. Traffic towards remote ISD-ASes is guaranteed to only use paths that start at one of these local ISD-ASes.
The name of the domain.
Default Domainprefixes object
List of IP prefix matchers to filter the announced and received prefixes.
accept_filter object[]
List of IP prefix matchers to define which prefixes announced by remotes ISD ASes are accepted. Only the matching subset of a prefix announced by a remote ISD-AS is is accepted for routing.
Specify matchers action.
Possible values: [ACCEPT, REJECT]
Optional description for the prefix matcher.
The list of IP prefixes used for matching. The matcher matches all IP prefixes that are contained in the union of the specified IP prefixes, i.e. it matches all listed prefixes as well as their contained more specific prefixes.
["192.168.1.0/24"]The sequence ID determines the order in which sequence the prefix matchers are applied. The sequence ID must be unique for each entry. Target devices apply the prefix matchers in order of ascending sequence ID (low to high) accepting all IPs that are in accepted matchers and rejecting the ones that are in rejected matchers.
1announce_filter object[]
List of IP prefix matchers to filter prefixes announced to remotes. The prefixes to be announced are configured in the static announcements or BGP. Only the subset of the routes that matches the announce filter is advertised to the remotes.
Specify matchers action.
Possible values: [ACCEPT, REJECT]
Optional description for the prefix matcher.
The list of IP prefixes used for matching. The matcher matches all IP prefixes that are contained in the union of the specified IP prefixes, i.e. it matches all listed prefixes as well as their contained more specific prefixes.
["192.168.1.0/24"]The sequence ID determines the order in which sequence the prefix matchers are applied. The sequence ID must be unique for each entry. Target devices apply the prefix matchers in order of ascending sequence ID (low to high) accepting all IPs that are in accepted matchers and rejecting the ones that are in rejected matchers.
1remote_isd_ases object[]
List of remote ISD-AS identifiers that belong to this domain. Prefix announcements will be accepted from these remote ISD-ASes. All IP traffic will be tunneled over paths that end in one of these remote ISD-ASes.
Specify the matchers action.
Possible values: [ACCEPT, REJECT]
Description for the remote matcher.
The ISD-AS identifier. The matcher matches the ISD-AS identifier of a SCION AS. 0 indicates a wildcard (both for ISD and AS).
0-ff00:0:310The sequence ID determines the order in which sequence the remote matchers are applied. The sequence ID must be unique for each entry. Target devices apply the remote matchers in order of ascending sequence ID (low to high).
1traffic_policies object[]
List of traffic policies that configure the types of traffic that are tunneled via this domain and the tunnel properties. A traffic policy defines a matcher on the IP traffic (the traffic matcher). If the IP traffic matches, it is tunneled to the remote SCION AS. Acceptable paths for the tunnel are defined via the path policy
The optional description of the traffic policy.
Default traffic policyfailover_sequence object[]
A list of failover sequence entries, each of them associated with a path filter. If there's no live path left after applying the first filter the second one is tried and so on.
Name of the path filter associated with the failover sequence entry.
Sequence number of the failover sequence entry. Sequence numbers define the ordering of the items which turn detemines how the failover between different path filters happens.
1The sequence ID determines the order in which sequence the traffic policies are applied. The sequence ID must be unique for each entry. Target devices try to find the first entry with a matching traffic matcher in ascending order determined by the sequence ID (low to high).
1Reference of the traffic matcher that is utilized by this policy. The traffic matcher is a selector for the IP packets covered by this traffic policy.
endpoint object
Local IP-in-SCION tunnel endpoint configuration
allowed_interfaces object[]
The SCION interfaces for each local SCION AS that are allowed to be used by this IP-in-SCION tunneling endpoint. This can be used to control incoming traffic, e.g., if a tunnel endpoint should only be reachable via SCION interfaces 1 and 2, allowed-interfaces should list them explicitly. Remote tunnel endpoints will then only choose paths entering the respective local AS via SCION interface 1 or 2. If the IP-in-SCION tunneling endpoint on this appliance should be reachable via a SCION interface of a peer appliance, the allowed-interfaces list must be configured with the respective SCION interface of the peer appliance. By default the list is empty, in this case the appliance will automatically configure the locally configured SCION interfaces as allowed-interfaces. Automatic configuration is disabled if topology synchronization is enabled or if disable_auto_allowed_interfaces is set.
List of allowed interfaces for this SCION AS
[2,3]The SCION AS where the list of allowed interfaces applies. Packets to this IP-in-SCION tunnel endpoint in this SCION AS will only arrive on the listed interfaces.
Port number for control traffic. The control address is constructed from the ip address and this control port. The control address is used to exchange IP routing information as part of SGRP. If not set, or zero, the control port will be dynamically allocated.
40201Port number for data traffic. The data address is constructed from the ip address and this control port. The data address is used for the IP-in-SCION encapsulated traffic stream. If not set, or zero, the data port will be dynamically allocated.
40200Optional description of the IP-in-SCION tunnel endpoint.
Whether the automatic configuration of allowed interfaces should be disabled. When disabled, the IP-in-SCION tunneling endpoint will be reached by remote endpoints on all SCION interfaces of the locally configured AS. When enabled (default), the local IP-in-SCION tunneling endpoint will only be reached by remote endpoints on the SCION interfaces that are configured on the local appliance.
Flag to disable uRPF. When enabled (default), the gateway performs strict URPF for all the received IP-in-SCION-tunneled traffic, checking that incoming IP packets have a source address that is within the announced prefixes by a remote gateway, and that the SCION packets are sent from a valid remote ISD-AS and are encrypted as configured in the associated domain.
Flag to activate SCION RSS. If activated, the gateway utilizes UDP source port entropy on the underlay such that EDGE and CORE routers can leverage RSS for SCION traffic. This can greatly improve throughput performance.
trueWhether this endpoint is enabled.
encryption object
Payload encryption configuration for the IP-in-SCION tunnel endpoint.
Whether the payload encryption module is enabled. With payload encryption enabled, the IP packets are encrypted and authenticated before being sent to a remote tunnel endpoint for domains that have the payload encryption enabled. Note that this flag only enables the payload encryption system. Each domain for which payload encryption should be used must still explicitly enable it.
The maximum number of Security Associations (SAs) that can be established with a single remote AS. If the limit is reached, new SAs from all endpoints in that AS will be rejected.
10001000Port number for the secure data traffic. The address is constructed from the endpoint IP address and this port. If not set, or zero, the secure data port will be dynamically allocated.
40203The maximum number of Security Associations (SAs) that can be established with remote tunnel endpoints. If the limit is reached, new SAs will be rejected.
100000100000IP address of the local IP-in-SCION endpoint.
192.168.1.100Port number for probing traffic. The probe address is constructed from the ip address and this probe port. The probe address is used by remote tunnel endpoints in their health probing. If not set, or zero, the probe port will be dynamically allocated.
40202path_filters object[]
List of path filters that can be referenced by name from a path policies. A path filter defines a set of paths by applying the filter to all available paths.
The ACL that is applied on the path. An ACL consists of a list of
ACL entries. Each ACL entry has the form action hop-predicate,
where the action can either be accept (+) or deny (-). The hop
predicate is optional and has the form isd-as#interface,
where isd is the ISD identifier, as is the AS identifier,
and interface is the interface identifier of a SCION path hop.
The hop predicate can be fully or partially qualified, i.e., all
entries of the hop predicate are optional or can include
wildcards (0). If no hop predicate is specified the action
matches every hop, i.e., a single '+' is the default accept action
and a single '-' is the default deny action. The ACL is applied
by sequentially applying all ACL entries to paths. If the ACL is
empty, it defaults to accepting all paths.
["+ 64-0"]Description, or comment, for the path filter.
Match only paths in the Swiss Isolation Domain (ID 64).The sequence of hop predicates that a path has to match to be accepted. Each hop predicate can optionally be extended with a modifier '' or '+'. The '' modifier means 0 or more occurrences. The '+' means one or more occurrences.
0* 64+ 0+Name of the path filter. This is value is used by the path policy to reference the path filter.
CH ISD onlyremotes object[]
List of remote ISD-ASes that are connected with the gateway. The remote ISD-ASes can be referenced in the remote matchers of the domains.
Description or Comment on the remote.
The ISD-AS of the remote.
1-ff00:0:310static_announcements object[]
List of static routes that are advertised. The routes are only advertised to the domains with matching announce filters.
Description, or comment, for the target.
next_hop_tracking object
container for next hop tracking
Whether or not this next-hop is tracked.
The routes are only distributed if the address responds to pings. This can be used to implement dynamically retractable routes without having to resort to a dynamic routing protocol.
192.168.0.1The IP prefixes that are statically configured and advertised via SGRP
Possible values: >= 1
["192.168.1.0/24","172.30.100.0/28"]The sequence ID defines the order of the static route entries. The sequence ID must be unique for each entry.
1traffic_matchers object[]
List of traffic matchers that can be referenced by name from a traffic policy. A matcher is used to classify traffic for tunneling. Each packet is classified based on configured traffic matchers and put in a traffic class. A traffic class is used in a traffic policy to map a path policy to a traffic class.
The condition for traffic to match this traffic matcher.
BOOL=trueDescription, or comment, for the traffic matcher
'all packets' matches all packets.Name that identifies the traffic matcher. This is value is used by the traffic policy to reference the traffic matcher.
all packetsOverview
IP-in-SCION tunneling is a mechanism that allows to tunnel IP packets between two IP-in-SCION tunneling endpoints, i.e., Anapaya EDGE appliances. This mechanism enables any type of application and communication to take advantage of SCION-based networking without the need to update any application, client, or server.
On a high-level, IP-in-SCION tunneling involves the following steps:
- A sender sends an IP packet towards an IP destination.
- The IP packet reaches an IP-in-SCION tunneling endpoint, such as an Anapaya EDGE appliance, in the sender's network via standard local IP routing.
- Based on the destination IP address, the Anapaya EDGE appliance determines the destination SCION AS and possible IP-in-SCION tunneling endpoints within the destination SCION AS. To achieve this, the Anapaya EDGE appliance uses the SCION gateway routing protocol (SGRP).
- Next, the Anapaya EDGE appliance determines the optimal SCION path to reach the destination SCION AS. The choice of the path depends on the Routing Domain Configuration and the real-time network performance characteristics, such as latency, jitter, drop rate, etc.
- Then, the original IP packet is encapsulated within a SCION packet by the Anapaya EDGE appliance and sent to the remote IP-in-SCION tunneling endpoint in the destination SCION AS. The SCION path used is the one chosen in the previous step.
- The remote IP-in-SCION tunneling endpoint receives the SCION packet and decapsulates the original IP packet. It then forwards the packet to the final IP destination using standard local IP routing.
Routing domains
Routing Domains are the primary mechanism to configure SGRP endpoint. Each routing domain can be thought of as a rule specifying the acceptable local and remote ISD-ASes, the IP prefixes that are announced and accepted, and the traffic policies that are applied to outgoing traffic.
An example of a routing domain may be: "Announce route 192.168.0.0/24 to remote ASes 1-ff00:0:1
and 1-ff00:0:2. Accept route 10.0.0.0/8 or any more specific route from those ASes and route
traffic to them in such a way that it does not leave the borders of the country. All of this applies
only to the local SCION ISD-AS 1-ff00:0:0."
Example
How are the IP prefixes assigned to the routing domains
A routing domain can be thought of as rule to apply on incoming/outgoing IP prefixes. To understand how it works, order the domains by their priority (priority is a property of a domain) in ascending order and match each prefix with the first domain, then with the second and so on. The first matching domain wins and the matching ends at that point.
A domain matches if local AS identity matches, remote AS matches and the prefix itself matches.
The winning domain can either reject the prefix or accept it. In the latter case the traffic policy in the domain is used to select SCION paths for associated outgoing traffic.
To ensure that lower priority domains can not hijack traffic from the higher priority domains, there's an additional safeguard: If the received IP prefix is not matched by the domain, but it is fully within the IP space allowed by the domain, the matching ends and the prefix is automatically rejected.
Additionally, if the entire IP space is not covered by the announcements from the remote ASes, then prohibit routes are installed to the IP routing table, covering the IP space defined by the domain. That way, if there are no available routes within the domain, the traffic does not fail over to lower priority domains with less specific routes.
Example
In the example below, prefix 10.0.1.0/24 would match domain1 and prefix
10.0.2.0/24 would match domain2. However, because of the safeguard check,
prefix 10.0.1.1/32 will be rejected. If it was not, it would allow domain2
to accept it and, because of the longest prefix match routing, to hijack the traffic
that logically belongs to domain1.
Also, prohibit routes 10.0.1.0/24 and 10.0.0.0/8 are installed.
[
{
"name": "domain1",
"priority": 15,
"prefixes": {
"accept_filter": [
{
"prefixes": ["10.0.1.0/24"],
"action": "ACCEPT",
"sequence_id": 0
}
]
}
},
{
"name": "domain2",
"priority": 16,
"prefixes": {
"accept_filter": [
{
"prefixes": ["10.0.0.0/8 ge 8"],
"action": "ACCEPT",
"sequence_id": 0
}
]
}
}
]
Fallback domains
Setting fallback flag on a domain means that the aforementioned security check is omitted, as well as the prohibit routes. What that means is that the traffic can fall over from a higher priority domain to a lower priority domain.
Example
In the example below, if route 10.0.1.0/24 is not received within domain1
the traffic falls over to domain2. If domain2 has a appropriate route,
it will be sent using traffic policies of domain2.
[
{
"name": "domain1",
"priority": 15,
"fallback": true,
"prefixes": {
"accept_filter": [
{
"prefixes": ["10.0.1.0/24"],
"action": "ACCEPT",
"sequence_id": 0
}
]
}
},
{
"name": "domain2",
"priority": 16,
"prefixes": {
"accept_filter": [
{
"prefixes": ["10.0.0.0/8 ge 8"],
"action": "ACCEPT",
"sequence_id": 0
}
]
}
}
]
Traffic policies
Traffic policies allow users to influence and control the path selection process of the Anapaya EDGE appliance. For example, an operator can define policies to keep traffic within Swiss jurisdiction or avoid certain network providers. A traffic policy uses path filters to specify the set of SCION paths that should be considered for the path selection. Each path filter defines a subset of all available SCION paths.
Path filters are combined into a failover sequence to define the order in which the path filters are considered. The path filters in a failover sequence are applied separately to the set of possible paths until one of them returns a non-empty path set. To select a path for the tunnel, the following steps are taken:
- The path filters are traversed in order of their sequence ID within the failover sequence.
- Each path filter is applied to the candidate set of paths.
- The best path out of the acceptable set is chosen according to path health and performance metrics. If no such path exists, i.e., no candidate path is allowed by the filter or no path is healthy, selection continues to the next path filter in the failover sequence.
- If all filters have been considered and no healthy path was selected, traffic will be dropped.
Finally, the traffic policy includes a traffic matchers that defines the set of IP packets for which the failover sequence should be applied.
SCION gateway routing protocol (SGRP)
The SCION gateway routing protocol (SGRP) is a routing protocol that enables IP-in-SCION tunneling endpoints, i.e., Anapaya EDGE appliances, to map IP prefixes to SCION ASes.
At a high level, a tunneling endpoint participating in SGRP between two SCION ASes does the following steps:
- It discovers the tunneling endpoints in the remote SCION AS. To that end, it periodically sends a discovery message to the control plane of the remote AS which replies with a list of local IP-in-SCION tunneling endpoints.
- It periodically queries each discovered endpoint in the remote AS to learn the IP prefixes it announces. From that, the local endpoint builds a mapping from IP prefix to remote tunneling endpoints.
- When queried by a remote tunneling endpoint about its prefixes, the local endpoint replies with the set of IP prefixes it wants to announce to the remote.
For the Anapaya EDGE appliance, the set of announced IP prefixes can either be statically configured or is dynamically learned via BGP. Please refer to BGP for information on how to configure a BGP session to a peering BGP router.
SGRP rules
This section covers the practical application of routing domains. Understanding this process is useful for debugging complex domain configurations.
Routing domains are converted into a list of SGRP rules, which are similar to BGP rules, and a list of prohibit routes for non-fallback domains. SGRP rules filter the IP prefixes announced by remote SCION ISD-ASes, determining which are accepted or rejected. For accepted prefixes, the rules also specify which SCION paths will transport the traffic. A second set of SGRP rules exists to filter IP prefixes announced to remote SCION ISD-ASes; however, they work similarly and are not covered in this section.
Consider the following example configuration:
[
{
"name": "domain1",
"priority": 1,
"fallback": false,
"remote_isd_ases": [
{
"action": "ACCEPT",
"isd_as": "1-ff00:0:112",
"sequence_id": 0
}
],
"prefixes": {
"accept_filter": [
{
"action": "ACCEPT",
"prefixes": ["10.0.0.0/8", "172.20.5.0/24 ge 24 le 32"],
"sequence_id": 0
}
]
},
"traffic_policies": [
{
"failover_sequence": [
{
"path_filter": "filter1",
"sequence_id": 0
}
],
"sequence_id": 0,
"traffic_matcher": "default"
}
]
},
{
"name": "domain2",
"priority": 2,
"fallback": true,
"remote_isd_ases": [
{
"action": "ACCEPT",
"isd_as": "0-0",
"sequence_id": 0
}
],
"prefixes": {
"accept_filter": [
{
"action": "ACCEPT",
"prefixes": ["192.168.0.0/16 ge 16 le 32"],
"sequence_id": 0
}
]
},
"traffic_policies": [
{
"failover_sequence": [
{
"path_filter": "filter2",
"sequence_id": 0
}
],
"sequence_id": 0,
"traffic_matcher": "default"
}
]
}
]
The generated accept SGRP rules are as follows (inspect them using the appliance-cli inspect tunneling sgrp command):
10.0.0.0/8 remote 1-ff00:0:112 accept as domain1
172.20.5.0/24 ge 24 le 32 remote 1-ff00:0:112 accept as domain1
10.0.0.0/8 ge 8 le 32 reject
172.20.5.0/24 ge 24 le 32 reject
192.168.0.0/16 ge 16 le 32 accept as domain2
The domain domain1 is translated first because it has the highest priority (lowest numerical
value).
The first two rules are a direct translation of the conditions defined for domain1:
10.0.0.0/8 remote 1-ff00:0:112 accept as domain1
172.20.5.0/24 ge 24 le 32 remote 1-ff00:0:112 accept as domain1
The notation 172.20.5.0/24 ge 24 le 32 specifies any prefix within the 172.20.5.0/24 range with
a length of at least 24 bits and at most 32 bits.
The subsequent two reject rules are generated because domain1 is a non-fallback domain. This
means any prefix matching the ranges 10.0.0.0/8 or 172.20.5.0/24 but not originating from ISD-AS
1-ff00:0:112 will be rejected. This configuration prevents domains with lower priorities from
hijacking the traffic:
10.0.0.0/8 ge 8 le 32 reject
172.20.5.0/24 ge 24 le 32 reject
Next, domain2, which has a lower priority, is translated. Its conditions are converted into a
single SGRP rule:
192.168.0.0/16 ge 16 le 32 accept as domain2
Since domain2 is a fallback domain, no reject rules are generated for it.
Creation of Routes
With the rule generation process explained, the next step is to understand how routing decisions are made. Assume that remote ISD-ASes announce the following IP prefixes:
1-ff00:0:111announces10.0.0.0/8and192.168.0.1/32.1-ff00:0:112announces172.20.5.128/25and100.35.0.0/24.
The resulting routing table contains the following entries:
$ ip route
default via 172.30.200.1 dev enp5s0 proto dhcp metric 100
prohibit 10.0.0.0/8 proto sgrp metric 1
prohibit 172.20.5.0/24 proto sgrp metric 1
172.20.5.128/25 dev scion-gateway proto sgrp metric 1
192.168.0.1/32 dev scion-gateway proto sgrp metric 2
The derivation of these routes from the announced IP prefixes is explained below.
Case 1: 10.0.0.0/8 is announced by ISD-AS 1-ff00:0:111
The generated SGRP rules are:
10.0.0.0/8 remote 1-ff00:0:112 accept as domain1
172.20.5.0/24 ge 24 le 32 remote 1-ff00:0:112 accept as domain1
10.0.0.0/8 ge 8 le 32 reject
172.20.5.0/24 ge 24 le 32 reject
192.168.0.0/16 ge 16 le 32 accept as domain2
Incoming prefixes are evaluated against these rules sequentially.
- Rule 1: The prefix
10.0.0.0/8matches, but the remote ISD-AS1-ff00:0:111does not match the required1-ff00:0:112. Evaluation proceeds to the next rule. - Rule 2: Neither the prefix nor the remote ISD-AS matches.
- Rule 3: The prefix matches, and there is no restriction on the remote ISD-AS. The action is reject, so the prefix is dropped immediately, and no further rules are evaluated.
The routing table also contains a prohibit route for 10.0.0.0/8. This is generated automatically
because domain1 is a non-fallback domain, which creates prohibit routes for all its specified
prefixes:
prohibit 10.0.0.0/8 proto sgrp metric 1
prohibit 172.20.5.0/24 proto sgrp metric 1
This ensures that traffic to IP addresses covered by domain1 does not fall through to a more
general route, such as the host's default route:
default via 172.30.200.1 dev enp5s0 proto dhcp metric 100
Attempting to ping an address in this range demonstrates this behavior:
$ ping 10.0.0.1
ping: Do you want to ping broadcast? Then -b. If not, check your local firewall rules
Although the ping command may produce a confusing error message, the underlying network response is an ICMP "Communication Administratively Prohibited" error.
Case 2: 192.168.0.1/32 is announced by ISD-AS 1-ff00:0:111
Considering the SGRP rules again:
10.0.0.0/8 remote 1-ff00:0:112 accept as domain1
172.20.5.0/24 ge 24 le 32 remote 1-ff00:0:112 accept as domain1
10.0.0.0/8 ge 8 le 32 reject
172.20.5.0/24 ge 24 le 32 reject
192.168.0.0/16 ge 16 le 32 accept as domain2
The prefix 192.168.0.1/32 does not match the first four rules. It matches the final rule because
its length of 32 bits falls within the specified ge 16 le 32 (greater than or equal to 16, less
than or equal to 32) range.
Therefore, traffic to this IP address is routed to the IP-in-SCION tunnel:
192.168.0.1/32 dev scion-gateway proto sgrp metric 2
The route's metric is 2, corresponding to the priority of the matching domain2. It is important to
note that routing decisions use the "longest prefix match" algorithm. The metric is only used as a
tie-breaker when multiple routes exist for the exact same IP prefix; otherwise, the more specific
(longer) prefix always wins. Consequently, traffic to 192.168.0.1 is tunneled via SCION paths that
conform to the path filter defined in domain2.
Case 3: 172.20.5.128/25 is announced by ISD-AS 1-ff00:0:112
This prefix is processed similarly. It matches the second SGRP rule, which accepts prefixes in the
172.20.5.0/24 ge 24 le 32 range and also requires the origin to be ISD-AS 1-ff00:0:112.
The resulting route is added to the routing table with a metric of 1. The traffic will be tunneled
via SCION paths that conform to the path filter in domain1:
172.20.5.128/25 dev scion-gateway proto sgrp metric 1
Case 4: 100.35.0.0/24 is announced by ISD-AS 1-ff00:0:112
This prefix matches no SGRP rule. Therefore, it is dropped and not inserted into the local routing table.
If a packet is sent to an address in this range, such as 100.35.0.1, it will not match any
SGRP-generated rule. The longest prefix match algorithm will cause it to fall through to the host's
default route:
default via 172.30.200.1 dev enp5s0 proto dhcp metric 100
In this case, the packet is routed through the standard local network, not the IP-in-SCION tunnel.
Configuration examples
Examples of domain entities configuration
-
The example below shows a domain that accepts any IP prefix from any remote AS and announces any learned or configured IP prefixes to any remote AS.
{
"remote_isd_ases": [
{
"isd_as": "0-0",
"action": "ACCEPT",
"description": "Include all remote ASes",
"sequence_id": 0
}
],
"prefixes": {
"accept_filter": [
{
"prefixes": ["0.0.0.0/0, ::/0"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["0.0.0.0/0, ::/0"],
"action": "ACCEPT",
"sequence_id": 0
}
]
}
} -
The example below shows a domain that only accepts
1.0.0.0/8and2.0.0.0/8from ASes in ISD1and announces only prefixes in1.100.0.0/16. Furthermore, the domain only includes the local AS identity that is part of ISD1.{
"remote_isd_ases": [
{
"isd_as": "1-0",
"action": "ACCEPT",
"description": "Only ASes in ISD 1 are part of the domain",
"sequence_id": 0
}
],
"local_isd_ases": ["1-ff00:0:1"],
"prefixes": {
"accept_filter": [
{
"prefixes": ["1.0.0.0/8", "2.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["1.100.0.0/16"],
"action": "ACCEPT",
"sequence_id": 0
}
]
}
}
Examples of failover sequences
The example below shows a configuration with a single traffic policy that configures all traffic to only use paths within the Swiss isolation domain. Refer to the Examples of path filter definition and Examples of traffic matchers definition examples to see how the corresponding filters and matchers are defined.
{
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "provider_1_outgoing"
},
{
"sequence_id": 1,
"path_filter": "allow_all"
}
]
}
Examples of traffic policies
-
The example below shows a configuration with a single traffic policy that configures all traffic to only use paths within the Swiss isolation domain. Refer to the Examples of path filter definition and Examples of traffic matchers definition examples to see how the corresponding filters and matchers are defined.
{
"traffic_policies": [
{
"sequence_id": 0,
"name": "CH ISD only",
"description": "No traffic leaves the Swiss ISD.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "ch_isd_only"
}
]
}
]
} -
The example below shows a configuration with two traffic policies. The first policy chooses paths via provider 1 for priority traffic (defined as traffic tagged with the EF DSCP flag) and falls back to paths via provider 2 if no paths via provider 1 are available. The second policy chooses paths via provider 2 for all non-priority traffic. If no paths are available via provider 2 no fallback happens and traffic is dropped. Refer to the Examples of path filter definition and Examples of traffic matchers definition examples to see how the corresponding filters and matchers are defined.
{
"traffic_policies": [
{
"sequence_id": 0,
"name": "Priority traffic",
"description": "Priority traffic is sent preferred via provider 1",
"traffic_matcher": "ef_tagged",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "provider_1_outgoing"
},
{
"sequence_id": 1,
"path_filter": "allow_all"
}
]
},
{
"sequence_id": 1,
"name": "Non-priority traffic",
"description": "Non-priority traffic is sent preferred via provider 2",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "provider_2_outgoing"
}
]
}
]
}
Examples of path filter definition
{
"path_filters": [
{
"name": "allow_all",
"description": "Allow all paths",
"acl": [
"+ 0"
]
},
{
"name": "ch_isd_only",
"description": "Only allow paths within the Swiss ISD (64)",
"acl": [
"+ 64-0",
"-"
]
},
{
"name": "provider_1_outgoing",
"description": "Paths that leave the local AS through SCION interface 1",
"hop_pattern": "1-ff00:1:111#1 0*"
},
{
"name": "provider_1_incoming",
"description": "Paths that enter the local AS through SCION interface 1",
"hop_pattern": "0* 1-ff00:1:111#1"
},
{
"name": "provider_2_outgoing",
"description": "Paths that leave the local AS through SCION interface 2",
"hop_pattern": "1-ff00:1:111#2 0*"
}
{
"name": "provider_2_incoming",
"description": "Paths that enter the local AS through SCION interface 2",
"hop_pattern": "0* 1-ff00:1:111#2"
}
]
}
Examples of traffic matcher definition
{
"traffic_matchers": [
{
"name": "all",
"description": "All traffic",
"condition": "BOOL=true"
},
{
"name": "ef_tagged",
"description": "Traffic tagged as Expedited Forwarding",
"condition": "DSCP=0x2e"
},
{
"name": "site1_to_site2",
"description": "Traffic from Site 1 to Site 2.",
"condition": "ALL(SRC=10.1.0.0/24, DST=10.2.0.0/24)"
}
]
}
Examples of endpoint configuration
{
"endpoint": {
"enabled": true,
"ip": "10.8.0.1",
"enable_scion_rss": true
}
}
Examples of endpoint configuration with allowed interfaces
The example below shows an endpoint configuration that makes the Anapaya appliance reachable only via the SCION interface with ID 1.
{
"endpoint": {
"enabled": true,
"ip": "10.8.0.1",
"allowed_interfaces": [
{
"interfaces": [1],
"isd_as": "1-ff00:0:1"
}
],
"enable_scion_rss": true
}
}
Examples of remote endpoints configuration
{
"remotes": [
{
"isd_as": "1-ff00:0:1",
"description": "Branch office in Zurich"
},
{
"isd_as": "2-ff00:0:2",
"description": "Branch office in London"
}
]
}
Examples of static announcements configuration
{
"static_announcements": [
{
"next_hop_tracking": {
"target": "10.8.0.2"
},
"prefixes": ["10.8.0.2/32", "10.8.0.5/32"],
"sequence_id": 0
},
{
"prefixes": ["10.8.0.1/32"],
"sequence_id": 1
}
]
}
Use cases
This section contains common IP-in-SCION tunneling use cases and how they can be implemented using the Anapaya EDGE appliance's domain-based routing configuration. There are of course many more use cases and implementation options. The examples below are intended to illustrate common use cases and should serve the reader as a basis for implementation and adaptation.
Company WAN and public cloud SaaS access
In this use case, the operator has two Anapaya EDGE appliances (edge1 and edge2) in two local data center sites, each with one SCION access link to an upstream SCION ISP. This is a typical deployment for a regional or global headquarter site (HQ Zurich). The operator has to consider the company's remote sites (Branch Paris and Branch London) that are part of the company's wide area network (WAN). Furthermore, the operator wants to configure access to two Software-as-a-Service (SaaS) applications that run in a public cloud. As the public cloud is not directly SCION-enabled, the company makes use of two service providers that provide SCION-enabled access to the public cloud (Cloud Access Provider).
This use case can be naturally implemented with two domains: one domain for the company's WAN and a separate domain to configure public cloud access.
{
"domains": [
{
"name": "company_wan",
"description": "The company's wide area network."
},
{
"name": "public_cloud",
"description": "Access to SaaS applications in a public cloud."
}
]
}
Domain entities
he first step is to define what entities are part of the domains, i.e., announced and accepted IP prefixes and remote SCION ASes.
Let us first focus on the company_wan domain and assume the following
requirements:
- Possible IP ranges for the company's WAN:
10.0.0.0/8and192.168.0.0/16. - The local sites should only announce prefixes in
10.1.0.0/16and10.2.0.0/16 - Only the remote sites
2-ff00:0:2(Branch Paris) and2-ff00:0:3(Branch London) should be part of thecompany_wandomain. - IP-in-SCION tunnels should be established to both these branches from HQ Zurich.
Putting all this together, we get the following configuration for the company_wan domain:
{
"scion_tunneling": {
"domains": [
{
"name": "company_wan",
"description": "The company's wide area network.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["10.0.0.0/8", "192.168.0.0/16"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["10.1.0.0/16", "10.2.0.0/16"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"remote_isd_ases": [
{
"isd_as": "2-ff00:0:2",
"action": "ACCEPT",
"sequence_id": 0
},
{
"isd_as": "2-ff00:0:3",
"action": "ACCEPT",
"sequence_id": 1
}
]
}
],
"remotes": [
{
"isd_as": "2-ff00:0:2",
"description": "Branch office in Paris"
},
{
"isd_as": "2-ff00:0:3",
"description": "Branch office in London"
}
]
}
}
Next, the operator needs to define the what entities that are part of the public_cloud domain.
The following requirements are assumed:
- The public cloud is accessible via the Cloud Access Providers (CAP)
2-ff00:0:100,1-ff00:0:200, and2-ff00:0:200. Note that the latter two ASes are distinct on the SCION level, but obviously they belong to the same CAP. - The IP prefix assigned app1 is
1.0.1.0/24and the one assigned to app2 is1.0.2.0/24, therefore these prefixes must be accepted. - All internal hosts are NATed to an IP in
2.0.0.0/24, i.e., this is the IP prefix that needs to be announced in thepublic_clouddomain, such that the return traffic can be correctly routed from the public cloud to the local site.
Putting all this together, we get the following configuration for the public_cloud domain:
{
"scion_tunneling": {
"domains": [
{
"name": "public_cloud",
"description": "Access to SaaS applications in a public cloud.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["1.0.1.0/24", "1.0.2.0/24"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["2.0.0.0/24"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"remote_isd_ases": [
{
"isd_as": "1-ff00:0:100",
"action": "ACCEPT",
"sequence_id": 0
},
{
"isd_as": "0-ff00:0:200",
"action": "ACCEPT",
"sequence_id": 1
}
]
}
],
"remotes": [
{
"isd_as": "1-ff00:0:100",
"description": "Access provider 1 to public cloud"
},
{
"isd_as": "1-ff00:0:200",
"description": "Access provider 2 to public cloud (in ISD 1)"
},
{
"isd_as": "2-ff00:0:200",
"description": "Access provider 2 to public cloud (in ISD 2)"
}
]
}
}
Note the use of the ISD wildcard 0-ff00:0:200 in the remote_isd_ases section. This matches the
AS identifier ff00:0:200 in every ISD, however, since the operator only explicitly adds the ASes
1-ff00:0:200 and 2-ff00:0:200 in the remotes section, only those will be used to establish
tunnels with.
Announced prefixes
The prefixes defined in the announce_filter of the domains are not the prefixes that are
necessarily announced. Instead, these are filters applied on statically configured and dynamically
learned prefixes (via BGP). In this example, both Anapaya EDGE appliances are connected internally
to a BGP peer that announces prefixes to and accepts prefixes from the Anapaya EDGE appliances. To
learn how to configure BGP peers on an Anapaya EDGE appliance, please refer to the
BGP section.
Traffic policies
Besides defining the entities that are part of the domains, the operator also needs to configure at least one traffic policy per domain.
Often, a permissive traffic policy, i.e., a traffic policy that does not limit the set of acceptable SCION paths too much, is advantageous. The Anapaya EDGE appliance has a powerful path optimization engine that can ensure optimal path selection amongst the set of acceptable paths.
The requirements are as follows:
- Generally, reliability and performance should be maximized.
- app1 must only be accessible via paths within ISD 1 as it deals with highly sensitive and regulated data.
- There are no specific requirements for app2.
The operator has little influence over the traffic policies used by the remote endpoints. To ensure that the traffic for app1 also only uses paths within ISD 1 on the return path, the operator needs to coordinate that with the remote endpoint. If the operator also controls the remote endpoint then this is not an issue.
Given these requirements, the operator first needs to define three traffic matchers: one that matches traffic belonging to app1, one that matches traffic belonging to app2, and a third that matches any traffic.
{
"traffic_matchers": [
{
"name": "app1",
"description": "Traffic belonging to app1",
"condition": "DST=1.0.1.0/24"
},
{
"name": "app2",
"description": "Traffic belonging to app2",
"condition": "DST=1.0.2.0/24"
},
{
"name": "all",
"description": "All traffic",
"condition": "BOOL=true"
}
]
}
Furthermore, two path filters need to be defined - one that only allows paths within ISD 1 and one that allows all paths.
{
"path_filters": [
{
"name": "isd_1_only",
"description": "Only allow paths within ISD 1",
"acl": ["+ 1-0", "-"]
},
{
"name": "allow_all",
"description": "Allow all paths",
"acl": ["+ 0"]
}
]
}
Having defined the traffic matchers and path filters, the operator can now assemble the traffic policies for the domains.
{
"domains": [
{
"name": "company_wan",
"traffic_policies": [
{
"sequence_id": 0,
"description": "Allow all traffic to use all available paths.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "allow_all"
}
]
}
]
},
{
"name": "public_cloud",
"traffic_policies": [
{
"sequence_id": 0,
"description": "Traffic belonging to app1 can only use paths within ISD 1.",
"traffic_matcher": "app1",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_1_only"
}
]
},
{
"sequence_id": 1,
"description": "Traffic belonging to app2 can use all paths",
"traffic_matcher": "app2",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "allow_all"
}
]
}
]
}
]
}
Local endpoint
To finalize the IP-in-SCION tunneling configuration, the operator needs to configure the local
tunneling endpoint for each Anapaya EDGE appliance. This is the part of the configuration that will
be different for each Anapaya appliance. Besides the host address to which the tunneling endpoint is
bound, the operator also wants to control incoming traffic such that traffic coming via SCION
interface 1 is always handled by edge1, and traffic coming via SCION interface 2 is always handled
by edge2. To that end, the operator needs to specify allowed_interfaces accordingly.
{
"endpoint": {
"ip": "10.1.0.1",
"enabled": true
}
}
{
"endpoint": {
"ip": "10.1.0.2",
"allowed_interfaces": [
{
"interfaces": [2],
"isd_as": "1-ff00:1:1"
}
],
"enabled": true
}
}
Complete IP-in-SCION tunneling configuration
For completeness, the entire scion_tunneling configurations for edge1 is shown below. The one for edge2 only differs in the endpoint section.
IP-in-SCION tunneling configuration for edge1
{
"scion_tunneling": {
"endpoint": {
"ip": "10.1.0.1",
"enabled": true
},
"domains": [
{
"name": "company_wan",
"description": "The company's wide area network.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["10.0.0.0/8", "192.168.0.0/16"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["10.1.0.0/16", "10.2.0.0/16"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"remote_isd_ases": [
{
"isd_as": "2-ff00:0:2",
"action": "ACCEPT",
"sequence_id": 0
},
{
"isd_as": "2-ff00:0:3",
"action": "ACCEPT",
"sequence_id": 1
}
],
"traffic_policies": [
{
"sequence_id": 0,
"description": "Allow all traffic to use all available paths.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "allow_all"
}
]
}
]
},
{
"name": "public_cloud",
"description": "Access to SaaS applications in a public cloud.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["1.0.1.0/24", "1.0.2.0/24"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["2.0.0.0/24"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"remote_isd_ases": [
{
"isd_as": "1-ff00:0:100",
"action": "ACCEPT",
"sequence_id": 0
},
{
"isd_as": "0-ff00:0:200",
"action": "ACCEPT",
"sequence_id": 1
}
],
"traffic_policies": [
{
"sequence_id": 0,
"description": "Traffic belonging to app1 can only use paths within ISD 1.",
"traffic_matcher": "app1",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_1_only"
}
]
},
{
"sequence_id": 1,
"description": "Traffic belonging to app2 can use all paths",
"traffic_matcher": "app2",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "allow_all"
}
]
}
]
}
],
"remotes": [
{
"isd_as": "2-ff00:0:2",
"description": "Branch office in Paris"
},
{
"isd_as": "2-ff00:0:3",
"description": "Branch office in London"
},
{
"isd_as": "1-ff00:0:100",
"description": "Access provider 1 to public cloud"
},
{
"isd_as": "1-ff00:0:200",
"description": "Access provider 2 to public cloud (in ISD 1)"
},
{
"isd_as": "2-ff00:0:200",
"description": "Access provider 2 to public cloud (in ISD 2)"
}
],
"traffic_matchers": [
{
"name": "app1",
"description": "Traffic belonging to app1",
"condition": "DST=1.0.1.0/24"
},
{
"name": "app2",
"description": "Traffic belonging to app2",
"condition": "DST=1.0.2.0/24"
},
{
"name": "all",
"description": "All traffic",
"condition": "BOOL=true"
}
],
"path_filters": [
{
"name": "isd_1_only",
"description": "Only allow paths within ISD 1",
"acl": ["+ 1-0", "-"]
},
{
"name": "allow_all",
"description": "Allow all paths",
"acl": ["+ 0"]
}
]
}
}
Multi-EDGE setup
The example below shows a setup with two Anapaya EDGE appliances (edge1 and edge2), located at a different site but each providing a reduntant connection to the other site. The operator wants to ensure that the traffic is load balanced between the two Anapaya EDGE appliances according to the destination of the traffic. I.e., traffic destined to Zurich should be handled by edge1 and traffic destined to Lugano should be handled by edge2. In case one of the Anapaya appliances becomes unavailable, the other Anapaya appliance should take over the traffic for both sites.
To achieve this, the remote endpoints (edge3) need to configure the respective failover sequences for the different kind of traffic.
Domains
This configuration can be implemented with two domains: one domain for the Zurich site and one domain for the Lugano site, each with a failover sequence that ensures that the traffic prefers the EDGE appliance at the destination site.
The Zurich domain accepts the IP prefixes of the Zurich site (172.20.5.0/25) and the traffic
policy prefers all traffic paths that end at the interface of the Zurich site. The Lugano domain
is configured the same way, but with the accepted IP prefixes of the Lugano site
(172.20.5.128/25) and a traffic policy that prefers the paths that end at the interface of the
Lugano site.
{
"domains": [
{
"name": "Zurich",
"description": "Zurich side network.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["172.20.5.0/25"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic prefers paths to Zurich",
"traffic_matcher": "default",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "zurich-paths"
},
{
"sequence_id": 1,
"path_filter": "default"
}
]
}
]
},
{
"name": "Lugano",
"description": "Lugano side network.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["172.20.5.128/25"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"action": "ACCEPT",
"prefixes": ["0.0.0.0/0", "::/0"],
"sequence_id": 0
}
]
},
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic prefers paths to Lugano",
"traffic_matcher": "default",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "lugano-paths"
},
{
"sequence_id": 1,
"path_filter": "default"
}
]
}
]
}
]
}
{
"path_filters": [
{
"name": "zurich-paths",
"description": "Only allow paths to Zurich (interface 1)",
"hop_pattern": "0* 1-ff00:0:112#1"
},
{
"name": "lugano-paths",
"description": "Only allow paths to Lugano (interface 2)",
"hop_pattern": "0* 1-ff00:0:112#2"
},
{
"name": "default",
"acl": ["+"]
}
]
}
This desired behavior cannot be configured on the receiving side of the IP-in-SCION tunnel. Currently, only the sending side can define preferences of remote endpoints for the traffic.
The same setup can also be achieved with a single domain using two traffic matchers. Each traffic matcher would match the traffic for one of the sites. A different failover sequence would then be used for each traffic matcher to ensure that traffic prefers paths that end at the destined site.
Complete IP-in-SCION tunneling configuration
IP-in-SCION tunneling configuration for edge3
{
"scion_tunneling": {
"endpoint": {
"control_port": 40202,
"data_port": 40200,
"enable_scion_rss": true,
"enabled": true,
"ip": "172.20.4.2",
"probe_port": 40201
},
"domains": [
{
"name": "Zurich",
"description": "Zurich side network.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["172.20.5.0/25"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"action": "ACCEPT",
"prefixes": ["0.0.0.0/0", "::/0"],
"sequence_id": 0
}
]
},
"remote_isd_ases": [
{
"isd_as": "1-ff00:0:112",
"action": "ACCEPT",
"sequence_id": 0
}
],
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic prefers paths to Zurich",
"traffic_matcher": "default",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "zurich-paths"
},
{
"sequence_id": 1,
"path_filter": "default"
}
]
}
]
},
{
"name": "Lugano",
"description": "Lugano side network.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["172.20.5.128/25"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"action": "ACCEPT",
"prefixes": ["0.0.0.0/0", "::/0"],
"sequence_id": 0
}
]
},
"remote_isd_ases": [
{
"isd_as": "1-ff00:0:112",
"action": "ACCEPT",
"sequence_id": 0
}
],
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic prefers paths to Lugano",
"traffic_matcher": "default",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "lugano-paths"
},
{
"sequence_id": 1,
"path_filter": "default"
}
]
}
]
}
],
"path_filters": [
{
"name": "zurich-paths",
"description": "Only allow paths to Zurich (interface 1)",
"hop_pattern": "0* 1-ff00:0:112#1"
},
{
"name": "lugano-paths",
"description": "Only allow paths to Lugano (interface 2)",
"hop_pattern": "0* 1-ff00:0:112#2"
},
{
"name": "default",
"acl": ["+"]
}
],
"remotes": [
{
"isd_as": "1-ff00:0:112"
}
],
"static_announcements": [
{
"next_hop_tracking": {
"disabled": true
},
"prefixes": ["172.20.4.0/24"],
"sequence_id": 0
}
],
"traffic_matchers": [
{
"condition": "BOOL=true",
"name": "default"
}
]
}
}
Multi-ISD Anapaya EDGE setup
The example below shows a single Anapaya EDGE appliance (edge) that is part of two ISDs. One use case for this setup is having an Anapaya appliance that is part of an ISD that is completely disconnected from the rest of the public SCION Internet, but needs to be remotely managed. In this case, a second ISD that is connected to the public SCION Internet needs to be configured on the Anapaya appliance, such that it can be remotely managed. Another use case is if the Anapaya EDGE appliance has two upstream SCION ISPs that are in different ISDs themselves. This is only possible if the Anapaya EDGE appliance is also part of both those ISDs.
The following example shows an Anapaya EDGE appliance that is part of two separate and isolated ISDs (ISD 1 and ISD 2). The requirements are as follows:
- Only IP prefixes in
1.0.0.0/8are announced and accepted to and from ISD 1. - Only IP prefixes in
2.0.0.0/8are announced and accepted to and from ISD 2. - Only remote ASes that are part of the corresponding ISD can announce these prefixes to the local Anapaya EDGE appliance.
- The Anapaya EDGE appliance should use its local AS identity that corresponds the ISD when communicating with other IP-in-SCION tunneling endpoints in the same ISD.
- IP prefix
1.0.1.0/24should be announced to ISD 1 and2.0.1.0/24to ISD 2 if (and only if) internally the firewall at192.168.2.100is reachable.
Given these requirements, the operator defines two routing domains: one for ISD 1 and one for ISD 2.
{
"domains": [
{
"name": "isd_1",
"description": "Domain for communication in ISD 1."
},
{
"name": "isd_2",
"description": "Domain for communication in ISD 2."
}
]
}
Domain entities
The first step is to define what entities are part of the domains, i.e., announced and accepted IP prefixes and remote SCION ASes.
Let us first focus on the isd_1 domain. As described above, the operator wants to ensure that only
IP prefixes in 1.0.0.0/8 are announced to and accepted from remote IP-in-SCION tunneling endpoints
in ISD 1. Furthermore, the operator wants to limit the set of remote ASes to those that are part of
ISD 1 and use the local AS identity in ISD 1 for communication in this domain. Finally, the remote
site Remote 1 (1-ff00:0:2) needs to be explicitly added to the remotes section such that the
Anapaya EDGE appliance will establish IP-in-SCION tunnels to endpoints in Remote 1.
{
"scion_tunneling": {
"domains": [
{
"name": "isd_1",
"description": "Domain for communication in ISD 1.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["1.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["1.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"local_isd_ases": ["1-ff00:0:1"],
"remote_isd_ases": [
{
"isd_as": "1-0",
"action": "ACCEPT",
"sequence_id": 0
}
]
}
],
"remotes": [
{
"isd_as": "1-ff00:0:2",
"description": "Remote AS in ISD 1"
}
]
}
}
{
"scion_tunneling": {
"domains": [
{
"name": "isd_2",
"description": "Domain for communication in ISD 2.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["2.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["2.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"local_isd_ases": ["2-ff00:0:1"],
"remote_isd_ases": [
{
"isd_as": "2-0",
"action": "ACCEPT",
"sequence_id": 0
}
]
}
],
"remotes": [
{
"isd_as": "2-ff00:0:3",
"description": "Remote AS in ISD 2"
}
]
}
}
Announced prefixes
As described in the introduction, the Anapaya EDGE appliance should announce the local prefixes
1.0.1.0/24 in ISD 1 and 2.0.1.0/24 in ISD 2 if internally the firewall at 192.168.2.100 is
reachable. This can be accomplished by configuring the static_announcements section with
next_hop_tracking activated. Note that the static_anouncements section contains both IP
prefixes. The announce_filter in the corresponding domain configurations ensure that 1.0.1.0/24
is announce in ISD 1 and 2.0.1.0/24 in ISD 2.
{
"static_announcements": [
{
"next_hop_tracking": {
"target": "192.168.2.100"
},
"prefixes": ["1.0.1.0/24", "2.0.1.0/24"],
"sequence_id": 0
}
]
}
Traffic policies
Besides defining the entities that are part of the domains, the operator also needs to configure at least one traffic policy per domain.
Often, a permissive traffic policy, i.e., a traffic policy that does not limit the set of acceptable SCION paths too much, is advantageous. The Anapaya EDGE appliance has a powerful path optimization engine that can ensure optimal path selection amongst the set of acceptable paths.
In this example, the operator wants to ensure that traffic for destinations in ISD 1 only uses paths in ISD 1 and similarly for destinations in ISD 2. Besides that, the operator does not want to further limit the set of acceptable paths to give the Anapaya EDGE appliance the biggest possible path selection scope.
If the two ISDs are truly isolated from the rest of the SCION Internet, limiting the set of acceptable paths to those in the same ISD is not required as no other paths exist anyway. Nevertheless, it is good practice to be specific about the set of acceptable paths.
To implement these requirements, the operator first needs to define one traffic matcher to match all traffic and two path filters to match paths in ISD 1 and ISD 2 respectively.
{
"traffic_matchers": [
{
"name": "all",
"description": "All traffic",
"condition": "BOOL=true"
}
],
"path_filters": [
{
"name": "isd_1_only",
"description": "Only allow paths within ISD 1",
"acl": ["+ 1-0", "-"]
},
{
"name": "isd_2_only",
"description": "Only allow paths within ISD 2",
"acl": ["+ 2-0", "-"]
}
]
}
Using the defined traffic matcher and path filters, the operator can define the traffic policies for each domain:
{
"domains": [
{
"name": "isd_1",
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic must only use paths within ISD 1.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_1_only"
}
]
}
]
},
{
"name": "isd_2",
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic must only use paths within ISD 2.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_2_only"
}
]
}
]
}
]
}
Endpoint
To finalize the IP-in-SCION tunneling configuration, the operator needs to configure the local tunneling endpoint for the Anapaya EDGE appliance. Since the operator does not need to control incoming traffic, the only required configuration is to specify the host address to bind to and to enable the IP-in-SCION tunneling endpoint.
{
"endpoint": {
"ip": "192.168.2.1",
"enabled": true
}
}
Complete IP-in-SCION tunneling configuration
For completeness, the entire scion_tunneling configurations for edge is shown below.
IP-in-SCION tunneling configuration
{
"scion_tunneling": {
"endpoint": {
"ip": "192.168.2.1",
"enabled": true
},
"domains": [
{
"name": "isd_1",
"description": "Domain for communication in ISD 1.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["1.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["1.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"local_isd_ases": ["1-ff00:0:1"],
"remote_isd_ases": [
{
"isd_as": "1-0",
"action": "ACCEPT",
"sequence_id": 0
}
],
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic must only use paths within ISD 1.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_1_only"
}
]
}
]
},
{
"name": "isd_2",
"description": "Domain for communication in ISD 2.",
"prefixes": {
"accept_filter": [
{
"prefixes": ["2.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
],
"announce_filter": [
{
"prefixes": ["2.0.0.0/8"],
"action": "ACCEPT",
"sequence_id": 0
}
]
},
"local_isd_ases": ["2-ff00:0:1"],
"remote_isd_ases": [
{
"isd_as": "2-0",
"action": "ACCEPT",
"sequence_id": 0
}
],
"traffic_policies": [
{
"sequence_id": 0,
"description": "All traffic must only use paths within ISD 2.",
"traffic_matcher": "all",
"failover_sequence": [
{
"sequence_id": 0,
"path_filter": "isd_2_only"
}
]
}
]
}
],
"remotes": [
{
"isd_as": "1-ff00:0:2",
"description": "Remote AS in ISD 1"
},
{
"isd_as": "2-ff00:0:3",
"description": "Remote AS in ISD 2"
}
],
"traffic_matchers": [
{
"name": "all",
"description": "All traffic",
"condition": "BOOL=true"
}
],
"path_filters": [
{
"name": "isd_1_only",
"description": "Only allow paths within ISD 1",
"acl": ["+ 1-0", "-"]
},
{
"name": "isd_2_only",
"description": "Only allow paths within ISD 2",
"acl": ["+ 2-0", "-"]
}
],
"static_announcements": [
{
"next_hop_tracking": {
"target": "192.168.2.100"
},
"prefixes": ["1.0.1.0/24", "2.0.1.0/24"],
"sequence_id": 0
}
]
}
}
Configuring egress source NAT when public IP addresses are scarce
In some cases, the available IP addresses that an Anapaya appliance can announce to remote IP-in-SCION tunneling endpoints are limited, e.g., if a customer only has a single public IP address assigned by its ISP. To allow multiple internal hosts to share a single IP address, the Anapaya appliance can be configured to perform source NATing on the outgoing traffic. This way, the Anapaya appliance can use a single IP address to represent multiple internal hosts.
It is possible and also used in practice to announce private IP ranges to a remote IP-in-SCION tunneling endpoint. However, sometimes it is simpler to limit the set of announced prefixes to public IP ranges because then there is no coordination needed with the remote network to ensure that the announced IP ranges do not clash with the remote network's internal IP ranges.
Let us assume that the operator has only a single public IP address available 203.0.113.50/32
but wants to connect multiple internal hosts to the remote while only announcing the single public
IP address. Let us further assume that the internal hosts are in the range 192.168.1.0/24. The
operator can use the following configuration to achieve this:
{
"interfaces": {
"ethernets": [
{
"name": "lan0",
"addresses": ["192.168.1.1/24"],
"driver": "VPP_DPDK"
},
{
"name": "wan0",
"addresses": ["169.254.0.2/31"],
"driver": "VPP_DPDK"
}
],
"loopbacks": [
{
"name": "loop0",
"addresses": ["203.0.113.50/32"]
}
]
},
"nat": {
"snat": {
"address_pool": ["203.0.113.50/32"],
"exclude": [],
"interfaces": ["scion-gateway"]
}
}
}
The lan0 interface is configured with an address of the internal private IP range. Note that the
public IP address is added on a loopback interface and NOT on the wan0 interface. This is
because on the wan0 interface we have the IP underlay for the SCION access link (usually
provided by the upstream SCION ISP). The public IP address is only used for IP-in-SCION tunneling.
The NAT configuration specifies the public IP address that should be used for source NATing and the
interface on which the NAT should be applied to. In this case the address pool to use is the single
public IP address 203.0.113.50/32 and the NAT should be applied to the special scion-gateway
interface to apply it to all IP-in-SCION tunneling traffic.
When using egress NAT it is not possible by default to create IP connections from a host in a remote AS to a host in the local AS that would go through the IP-in-SCION tunnel. Using NAT means that only outgoing connections are possible. If incoming connections are needed, the Anapaya appliance needs to be configured to exclude the IP addresses that should not be NATed.
Configuring ingress NAT to collect users from the Anapaya GATE
In this use case, the operator wants to ensure that traffic streams coming from an IP-in-SCION tunnel from a remote Anapaya GATE are returned via the local Anapaya EDGE appliance, even though the remote IP prefixes are also reachable via the legacy Internet.
To address this use case, the Anapaya EDGE appliance can do NATing on the incoming traffic. Source addresses in packets arriving from the tunnel are rewritten by one of the addresses from the NAT's address pool.
The operator is responsible for setting up the static routes so that replies to the addresses in the NAT's address pool are routed back to the Anapaya appliance. This way the operator can ensure that replies to the packets arriving via the tunnel are routed back to the tunnel, even if the remote IP prefixes are also reachable via the legacy Internet.
If, on the other hand, a packet arrives through the Internet it keeps its original source address and the reply is routed back through the legacy Internet. Again, it is up to the operator to set up the routes in the local network accordingly.
{
"nat": {
"snat": {
"address_pool": ["192.168.0.1/32"],
"exclude": [],
"interfaces": ["eth0"]
}
}
}
Ingress NAT does not work with BGP enabled in the Anapaya appliance.
When using ingress NAT it is not possible by default to create connections from the local AS to the remote AS that would go through the IP-in-SCION tunnel. Using NAT means that there are no fixed IP addresses to connect to. If this is needed, a specific IP address, or a range of addresses, can be excluded from the NAT. The packet will then arrive from the tunnel with its original source address unmodified. In this case it's up to the operator to route the packets destined to that address to the SCION tunnel rather than routing them through the legacy Internet.