Skip to main content

Cluster

A SCION AS consists of one or more appliances that form a cluster to synchronize state. The appliances are the shards of the AS cluster and are referred to with a shard identifier. Each shard is an independent unit of the SCION AS and can function and fail independently of the other shards.

Note that this is an Anapaya-specific extension of how a SCION AS is internally implemented while fully adhering to the specification of the SCION protocol.

note

An appliance can be configured to take part in several ASes for instance to connect to multiple isolation domains. Hence, an appliance can be a member of several AS clusters. In each of the clusters it is identified by a shard id that is unique within the cluster.

The shard ids of the appliance are configured in the scion.ases section and referenced in the peer's cluster section.

Appliances synchronize two kinds of state:

  • SCION control-plane data Consists of path segments and beacons that are used to construct the ASes path database. The synchronization of control data happens only within an AS and is performed by the SCION control service. The control service synchronizes its path segments and beacons with the control services of all peers whose addresses it learns from the topology information. This happens regardless of whether the topology information is statically configured or synchronized.
  • Topology information Consists of addresses of SCION control services, IP-in-SCION tunneling endpoints and information about SCION links to neighbors. The synchronization of topology information is configured in the cluster section. Topology information can be automatically synchronized or configured statically. In both cases, each cluster member is configured with a list of all of its peers.

The Configuration reference section provides the full configuration reference. The sections following after provide descriptions of most relevant cluster configuration sections with more elaborated examples.

Configuration reference​

v0.40v0.39v0.38next

Anapaya appliance configuration (cluster only)

cluster object

The configuration for the appliance cluster.

features object

The list of feature that are announced to the peers. Note that the actually announced value can depend on whether what features is locally enabled and configured.

scion_rssboolean

Option to enable the announcement of support for the SCION RSS feature to the peers. If the local host does not support the SCION RSS feature, this option does not have any effect.

Default value: true
peers object[]

The list of peers in this cluster. This is used to configure the topology or the discovery of the topology of peer appliances in an organization.

  • Array [
    • descriptionstring

    Textual description for this peer.

    features object

    Configures the feature options of the peer. This field can not be set together with the synchronization field.

    scion_rssboolean

    Option to statically enable the SCION RSS feature. If set to true, the local router enables UDP source port entropy on the underlay for SCION packets forwarded to the peer, such that the peer can leverage RSS for SCION traffic. This can greatly improve throughput performance. This must only be set to true if the peer supports the SCION RSS feature.

    Default value: false
    namestring

    The name of this peer used to identify the peer. This can be any string but must be unique among all peers.

    scion object

    The relevant SCION configuration of the peer. This can be used to define the relevant SCION components on the peer appliance so that paths via the peer appliance can also be used.

    ases object[]

    The list of SCION ASes on the peer.

  • Array [
    • control object

    Configuration and state data for the control service in the peer.

    addressstring

    The address of the control service. The address must be specified as host:port.

    Example: 192.168.1.1:30100
    isd_asstring<isd-as>required

    ISD-AS number of the AS.

    Example: 1-ff00:0:110
    neighbors object[]

    The neighbors for the SCION AS in the peer.

  • Array [
    • interfaces object[]

    The list of interfaces on the peer for this neighbor AS.

  • Array [
    • interface_idinteger<uint16>required

    SCION interface identifier. It must be unique in the SCION AS.

    Possible values: >= 1 and <= 65535

    next_hopstring

    Internal address of the peer router that owns the interface.

    Example: 169.254.0.1:30100
    scion_mtuinteger<uint16>

    The maximum transmission unit in bytes for SCION packets. This represents the protocol data unit (PDU) of the SCION layer on this interface and is usually calculated as maximum Ethernet payload - IP Header - UDP Header.

    Default value: 1472
    Example: 1472
  • ]
    • neighbor_isd_asstring<isd-as>required

    ISD-AS number of the neighbor AS.

    Example: 2-ff00:0:210
    relationshipstringrequired

    The relationship to the neighbor AS. If the local AS is core, this value must either be CORE or CHILD. If the local is non-core, this value must either be PARENT, CHILD or PEER.

    Possible values: [CORE, CHILD, PARENT, PEER]

  • ]
    • shard_idinteger<uint32>

    The shard ID of the peers in the AS.

  • ]
    • scion_tunneling object

    The relevant SCION tunneling configuration of the peer. This is used so that all appliances can announce the full list of SCION tunneling endpoints in the AS to other ASes.

    endpoint object

    The SCION tunneling endpoint on the peer appliance.

    allowed_interfaces object[]

    The SCION interfaces for each SCION AS that is configured on the peer, that are allowed to be used by this IP-in-SCION tunneling endpoint. This can be used to control incoming traffic, e.g., if a tunnel endpoint should only be reachable via SCION interfaces 1 and 2, allowed-interfaces should list them explicitly. Remote tunnel endpoints will then only choose paths entering the respective local AS via SCION interface 1 or 2. If the IP-in-SCION tunneling endpoint on the peer appliance should be reachable via a SCION interface of another appliance, the allowed-interfaces list must be configured with the respective SCION interfaces. By default the list is empty, in this case the appliance will automatically configure the SCION interfaces that are configured on the peer as allowed-interfaces. Automatic configuration can be disabled by setting disable_auto_allowed_interfaces.

  • Array [
    • interfacesinteger<uint16>[]

    List of allowed interfaces for this SCION AS

    Example: [2,3]
    isd_asstring<isd-as>

    The SCION AS where the list of allowed interfaces applies. Packets to this IP-in-SCION tunnel endpoint in this SCION AS will only arrive on the listed interfaces.

  • ]
    • control_portinteger<uint16>

    Port number for control traffic. The control address is constructed from the IP address and this control port. The control address is used to exchange IP routing information as part of SGRP. If not set, or zero, the control port will be dynamically allocated.

    Example: 40201
    data_portinteger<uint16>

    Port number for data traffic. The data address is constructed from the IP address and this control port. The data address is used for the IP-in-SCION encapsulated traffic stream. If not set, or zero, the data port will be dynamically allocated.

    Example: 40200
    disable_auto_allowed_interfacesboolean

    Whether the automatic configuration of allowed interfaces should be disabled. When disabled, the IP-in-SCION tunneling endpoint of the peer will be reached by remote endpoints on all SCION interfaces of the locally configured AS. When enabled (default), the peer IP-in-SCION tunneling endpoint will only be reached by remote endpoints on the SCION interfaces that are configured on the peer appliance.

    ipstring<ip-address>

    IP address of the peer IP-in-SCION endpoint.

    Example: 192.168.1.100
    probe_portinteger<uint16>

    Port number for probing traffic. The probe address is constructed from the IP address and this probe port. The probe address is used by remote tunnel endpoints in their health probing. If not set, or zero, the probe port will be dynamically allocated.

    Example: 40202
    synchronization object

    The synchronization configuration for this peer. This can be used to configure the automatic synchronization of topology information and supported features. Automatic synchronization of topology and supported features is not recommended for EDGE deployments. Instead static configuration is recommended. This field can not be set together with the scion, scion-tunneling, and features field.

    addressstring

    The gRPC address of this peer, used for synchronization of appliance information

    Example: 192.168.1.1:30100
  • ]
    • synchronization object

    The configuration data necessary for the anapaya cluster synchronization. This determines how frequently this appliance synchronizes its local data with its peers, if synchronization is enabled.

    addressstring

    The address where peers can fetch topology information. If this is not set, topology information is not exposed to peers and should be statically configured on the peers.

    Example: 192.0.2.3:40000
    node_synchronization_intervalstring<duration-string>

    The interval between two consecutive topology synchronizations attempts to the cluster peers. Must only be set if dynamic topology discovery is enabled. It requires a unit suffix out of ['d', 'h', 'm', 's']. The encoding consists of a decimal number concatenated with a suffix; for example, '5s', '10m', '12h', and '1d'.

    Default value: 1m

    SCION control-plane data synchronization​

    SCION control-plane data is synchronized by the control services. The local control service addresses are configured in the scion.ases section. Refer to the documentation of the SCION section for a detailed description of this section. The addresses of remote control services are configured or synchronized through topology synchronization and no additional configuration is required.

    note

    Even if topology synchronization is disabled, the control service still dynamically synchronizes its SCION control-plane data (beacons and path segments) with all control services that are configured as peers in the same AS.

    Peers​

    The decision to configure topology information statically or to enable synchronization should be the same for all peers. In static configuration, all the topology information about peers must be explicitly stated in the configuration, and thus, the information must be known at the configuration time. In this mode, changing the topology information about peers requires changing the configuration. On the contrary, in the automatic mode, the peering appliances exchange their topology information using a protocol. Thus, only the synchronization endpoints must be configured in the configuration.

    Static topology configuration​

    Statically configuring the topology information is recommended for Anapaya EDGE appliances, where the configuration overhead is small and the topology does not change frequently.

    Loading...

    Automatic topology synchronization​

    Automatic topology synchronization is recommended for Anapaya CORE appliances. This way the configuration does not need to be updated when the topology changes for instance if a new SCION link is added.

    To configure a peer for topology synchronization only the endpoint needs to be configured through the synchronization.address. This field needs to match the cluster.synchronization.address configured in the peer's appliance configuration.

    Loading...