Appliance Release v0.42
This page contains the release notes for the v0.42 Anapaya appliance software release. The appliance software release is applicable for the following Anapaya products:
We recommend always upgrading to the latest available patch release. Please refer to Upgrade notes for steps to be taken when upgrading. For general information on how to upgrade your appliance, please refer to software updates.
Upgrade Notes
Refer to the Known issues for known issues in this release.
Starting with v0.42, the appliance enforces the software license. Make
sure that each appliance has a valid license before you upgrade. If an appliance has no valid
license and its trial period is over, the appliance is in the restricted-functionality state after
the upgrade. The appliance keeps the active configuration, but it refuses all new configurations.
Run appliance-cli info license before the upgrade. The command shows the license status. For more
information, refer to Software license enforcement.
-
This release introduces breaking changes. Please read the Breaking Changes section carefully before upgrading.
-
This release requires the installation of the anapaya-system package v2.20.0 (or later). The installer will automatically reject the installation if the required anapaya-system package is not available.
-
The appliance migrates the configuration only from v0.34 and later. To upgrade an appliance that runs v0.31, v0.32, or v0.33, first upgrade it to an intermediate release, for example v0.41.
-
The appliance now ships its logs with the OpenTelemetry collector instead of Promtail. If you send the appliance logs to Loki, enable structured metadata in Loki before the upgrade. Refer to Log shipping with OpenTelemetry.
-
The router now uses the new dispatch behavior by default. If your AS has more than one appliance and one of them runs a release before v0.39 (please note that any release prior to v0.39 is end of life), set the
disable-new-dispatch-behaviorfeature flag after the upgrade. Refer to New dispatch behavior is the default. -
The appliance now runs the endhost API itself. If you run an endhost API on the appliance, remove it before the upgrade. Refer to The appliance runs the endhost API.
-
The advertised routes of a shuttle server must not overlap. Upgrade the shuttle servers before the shuttle clients. Refer to Shuttle advertised routes must not overlap.
-
This software release also contains updated Grafana dashboards and alerting rules, please follow Recommended alerts and Recommended Grafana dashboards for more information on how to update your monitoring setup.
Features
Software license enforcement
Starting with v0.42, the appliance enforces the software license. The appliance checks each new configuration against the license and rejects configurations that do not comply. The appliance does not change or restrict the active configuration.
The APIs PUT /config, POST /configs, and POST /config/validate check the new configuration
against the license. The result of the check depends on the license
state:
| License state | New configuration |
|---|---|
trial-period | The appliance accepts all configurations. |
valid-license | The appliance refuses a configuration that the license does not allow. |
grace-period | The appliance refuses a configuration that the license does not allow. |
restricted-functionality | The appliance refuses all configurations. |
If the appliance refuses a configuration, the error lists each violation with the reason license restricted.
The appliance is in the restricted-functionality state in these cases:
- The trial period has ended and no license is installed.
- The license has expired and the grace period has ended.
These additions help you to monitor the license:
- The new health check "License covers the configured features" lists the parts of the active
configuration that the current license does not allow. The status of the check is
failingif the active configuration contains such parts. - The new metric
appliance_controller_enforcer_license_violationsshows the number of these parts. - The new alert
ManagementSoftwareLicenseViolationfires if the active configuration contains parts that the license does not allow. - The new alert
ManagementSoftwareLicenseExpiredfires if the license or the trial period has expired. In therestricted-functionalitystate, only this alert fires. appliance-cli info licenseshows the signing key version of the active license. When a release removes the support for a signing key version, the appliance does not accept licenses with this key version after the upgrade. Use this command before an upgrade to make sure that the new release accepts your license.
For more information on how to get and install a license, refer to License management.
Experimental Hummingbird support
Hummingbird support is an experimental feature. Use in production environments is not recommended without prior consultation with Anapaya Support.
Hummingbird is a SCION extension for bandwidth reservations. A Hummingbird path is a SCION path that can carry a flyover in a hop field. A flyover is a bandwidth reservation for that hop. This release adds Hummingbird support to the SCION router of the appliance.
The router processes the Hummingbird packets of each ISD-AS in one of these modes:
disabled: The router drops all Hummingbird packets.forwarding: The router forwards the Hummingbird packets without a flyover, and it drops the packets with a flyover. This is the default.flyover: The router verifies the flyover MACs and forwards the flyover packets as best-effort traffic.policing: The router verifies the flyover MACs and polices the flyover packets against their reservations. The router marks the packets that conform to their reservation as priority.
The router can also limit the traffic that it sends on an external SCION interface to an egress rate. Above the egress rate, the router drops all traffic, except for the BFD packets and the traffic that the Hummingbird policer marks as priority. Thus, the reserved traffic gets through a congested interface.
The appliance configuration does not contain the Hummingbird settings and the egress rate. In case you are interested in using Hummingbird, please contact Anapaya Support.
Log shipping to multiple destinations
The appliance can now ship its logs to more than one remote destination at the same time. Configure
the destinations as a list under management.telemetry.logging.destinations. Each destination has a
unique name, a type, and a configuration block for the type. These types are available:
LOKI: Loki, with the URL, basic authentication, and tenant ID.SYSLOG: Syslog over TCP, UDP, or TLS, with RFC5424.SPLUNK_HEC: The Splunk HTTP Event Collector.OTLP: OpenTelemetry over HTTP (HTTP_PROTOBUForHTTP_JSON).
You can use the same type for more than one destination. A destination is active by default. To
stop the log shipping to a destination and keep its configuration, set disabled: true.
For the configuration reference, refer to the management configuration.
Trust material management
The appliance API has new endpoints to remove SCION trust material:
DELETE /cppki/certificates/{chain_id}removes an AS certificate chain and its private key.DELETE /cppki/csrs/{csr_id}removes a certificate signing request and its private key.DELETE /cppki/trcs/isd{isd}removes all TRCs of an ISD. The control service then stops trusting this ISD. It deletes the path segments and beacons that pass through the ISD, and path lookups no longer return a path through the ISD.
The appliance protects the trust material that it needs to operate:
- To remove the last valid certificate chain of an AS, you must set the
forcequery parameter. Without a valid chain, the AS cannot renew its certificate. - To remove the TRCs of an ISD that the appliance is part of, you must set the
forcequery parameter. Without these TRCs, the control service cannot verify the control plane messages of the ISD, and the paths through the ISD stop working.
Use force with care. The gateway loads its certificate from the trust store again every few
minutes. If you remove the certificate chain that the gateway uses, the sessions to the remote
gateways stop.
If you remove a certificate signing request before the certificate is issued, the certificate is not usable when it arrives.
Configuration history metadata
The config API responses (GET /config, PUT /config, POST /configs, GET /configs, GET /configs/{version}) now include a config_metadata object. The object shows when, with which
software, and by whom a configuration version was saved:
versionandcreated_at: The configuration version and its creation time.release_versionandsystem_version: The software versions that were active when the configuration version was created.created_by: The authenticated API user.created_by_local_user: The operating system user that started a tool on the appliance.created_by_console_user: The user in CONSOLE. In this case,created_bycontains the service identity of CONSOLE.created_with: The user agent of the client, for exampleappliance-cli/<version>.
A field is absent if the appliance does not have the value. All user fields are absent for
configurations that were saved before v0.42. The appliance sets created_by from the API
authentication.
VRRP priority tracking
With fixed VRRP priorities, the appliance with the highest priority is the primary, even if it has no connectivity to the remote AS. A backup appliance that can forward the traffic is then not used.
You can now configure a tracker that checks if a route to an IP address exists through IP-in-SCION tunneling. If the route does not exist, the tracker lowers the VRRP priority of the appliance. Thus, the VRRP group selects a different primary.
Configure the trackers of a VRRP group under track.scion_tunneling. The target is the IP address
that the tracker checks. The priority_cost is the value that the tracker subtracts from the VRRP
priority if no route to the target exists. For example, this VRRP group has a priority of 80 while
no route to 104.20.26.136 via IP-in-SCION tunneling exists:
"vrrp": [
{
"addresses": ["10.10.0.254"],
"priority": 100,
"vrid": 1,
"track": {
"scion_tunneling": [
{
"name": "tracker1",
"target": "104.20.26.136",
"priority_cost": 20
}
]
}
}
]
The configuration validation makes sure that an IP-in-SCION tunneling configuration accepts a prefix
that contains target. The priority must be larger than the sum of the priority_cost values of
all trackers of the VRRP group. For more information, refer to Dynamic VRRP
priorities.
Use appliance-cli get debug/vpp/vrrp to see the state of the VRRP trackers.
Link coverage beacon selection
The control service uses a new algorithm, link_coverage, to select the beacons that it propagates
and registers. The algorithm keeps at least one beacon for each ingress interface. It then selects
the beacons that add the most links that no other selected beacon contains. Thus, more links of the
ISD appear in the path segments. The algorithm also balances the candidate beacons over the ingress
interfaces, so an interface with many beacons cannot hide the beacons of a different interface.
The link_coverage algorithm is the default. To select a different algorithm, use the
config.scion.ases.control.beacon_selection_algorithm field. The all_ingress algorithm now uses
the same balanced candidate set.
Experimental Endhost API
The appliance now serves the experimental endhost API. Endhost applications use the endhost API to find the SCION routers and to get SCION paths.
The endhost API serves each SCION AS that has scion.ases.control.enabled set to true.
By default, the appliance serves the endhost API only on a unix socket. Use these settings to change the endhost API:
- To expose the endhost API to the endhosts in the LAN, set
scion.endhost_api.addressto the listen address. - To restrict the access to your endhosts, add the source prefixes to
scion.endhost_api.allowed_sources, for example192.0.2.0/24. By default all sources that can reach the listening address are allowed. The appliance firewall applies this list only in the firewall modesAUTOandPREPEND. - To exclude a SCION AS, add the ISD-AS to
scion.endhost_api.excluded_isd_ases. - To disable the endhost API, set
scion.endhost_api.disabledtotrue.
If you already run an endhost API on the appliance, refer to The appliance runs the endhost API.
Other changes
-
The responses of
GET /network/shuttlesandGET /network/shuttles/{interface_name}now include the fielddisabled. For a disabled shuttle client, the response contains the name, the MTU, and the configured servers from the configuration. If the appliance cannot get the status of an enabled shuttle client,GET /network/shuttlesgives the reason in the fielderror. -
appliance-cli info shuttleshows the lineEnabledfor each shuttle client. If the appliance cannot get the status of a shuttle client, the command shows the reason in the lineError. Thus, you can see the difference between a disabled shuttle client, a shuttle client that cannot connect, and a shuttle client that does not respond. -
The responses of
GET /network/shuttlesandGET /network/shuttles/{interface_name}now include the fieldquic_version. The field shows the QUIC version of the connection to the shuttle server. The field is present only while the shuttle client is connected.
Breaking Changes
Software license enforcement
The appliance refuses new configurations that the license does not allow. In the
restricted-functionality state, the appliance refuses all new configurations. Refer to Software
license enforcement.
New dispatch behavior is the default
The router now uses the new dispatch behavior by default. For a packet to a host in the local AS, the router uses the SCION/UDP destination port as the underlay UDP destination port. Before this change, the router used the dispatcher port 30041 by default.
If your AS has more than one appliance and one of them runs a release before v0.39 (please note that
any release prior to v0.39 is end of life), set the disable-new-dispatch-behavior feature flag
on all upgraded appliances of the AS after the upgrade. Remove the flag after all appliances of the
AS run v0.39 or later.
"experiments": {
"features": [
{
"name": "disable-new-dispatch-behavior",
"value": "true"
}
]
}
The same change applies to an end host in the local AS. An end host that receives SCION packets only
on the dispatcher port 30041 also needs the disable-new-dispatch-behavior feature flag.
The appliance does not accept the enable-new-dispatch-behavior feature flag anymore. The
configuration migration to this release removes the flag.
Log shipping with OpenTelemetry
The appliance now ships its logs with the OpenTelemetry collector. The appliance controller removes Promtail automatically.
If you send the appliance logs to Loki, Loki must have structured metadata enabled:
limits_config:
allow_structured_metadata: true
We also recommend this configuration in the distributor section of the Loki configuration:
distributor:
otlp_config:
default_resource_attributes_as_index_labels:
- service.name
- hostname
- project
- job
- unit
- isd_as
With this configuration, Loki uses the hostname, project, job, unit, and isd_as resource
attributes as index labels. Thus, your existing Loki queries continue to work. Without this
configuration, you must change your queries. For example, the query
{job="control",isd_as="64-2_0_9f"} becomes {service_name="control"} | isd_as="64-2_0_9f".
For more information, refer to the Loki documentation on OpenTelemetry ingestion and structured metadata.
The ManagementLogSendError alert now monitors the OpenTelemetry collector. The Promtail alert has
the new name ManagementLogSendErrorLegacy. Keep it for appliances that run a release before v0.42.
Shuttle client disabled field
The disable field of a shuttle client interface now has the name disabled. All other fields that
turn off a feature use the name disabled. The appliance migrates the stored configuration during
the upgrade. A configuration that you apply after the upgrade must use disabled:
"shuttles": [
{
"name": "shuttle0",
"disabled": true,
"local": "1-ff00:0:111,172.20.2.3",
"servers": [{"endpoint": "[1-ff00:0:110,172.20.1.3]:52810"}]
}
]
Shuttle advertised routes must not overlap
The advertised routes of a shuttle server must not overlap. A shuttle client of this release rejects a route advertisement with overlapping routes, and then it cannot connect.
The advertised routes are the advertised_routes of the shuttle server, or the
allowed_destinations if advertised_routes is empty. The appliance also advertises the subnet of
the tunnel address of the shuttle server. The appliance now advertises this subnet only if no
route contains the subnet.
The appliance rejects a shuttle server configuration if the advertised routes overlap. Examples of configurations that the appliance rejects:
advertised_routescontains10.0.0.0/8and10.1.0.0/16.advertised_routescontains10.0.0.128/25, and the tunneladdressis10.0.0.1/24. The subnet10.0.0.0/24overlaps with the route, but the route does not contain the subnet.
A configuration with the tunnel address 10.200.0.1/24 and the route 10.0.0.0/8 is valid. The
appliance does not advertise 10.200.0.0/24, because 10.0.0.0/8 contains it.
A shuttle server of an older release advertises the subnet of the tunnel address also if a route
contains the subnet. A shuttle client of this release cannot connect to such a server. Upgrade the
shuttle servers before the shuttle clients.
Number of VPP worker threads
The system.vpp.cpu.workers setting now gives the exact number of VPP worker threads. Before this
release, on a host with hyper-threading and enough free physical cores, the appliance also gave each
worker its sibling core. For example, workers: 10 on a host with 16 physical cores gave 20 worker
threads.
On a host with hyper-threading and 14 or more physical cores, the default is now 8 worker threads
instead of 16. Each worker thread has its own physical core. If you set system.vpp.cpu.workers
explicitly, make sure that the value gives the number of worker threads that you need.
Stricter configuration validation
The configuration validation now rejects these configurations:
- A telemetry label value under
management.telemetry.labelsthat contains characters other than letters, digits, spaces, and_ - . : / # @. A value with a double quote, a backslash, or a closing bracket already made the telemetry configuration invalid. - A BGP neighbor with multihop BFD and without a BFD local address. FRR ignores such a BFD peer, so BFD did not run for this neighbor.
- A
flow-expiration-intervalof less than 15 seconds when flow metrics are enabled. The gateway did not start with such a value. - A GRE interface name that is not unique among all interfaces.
- A configuration with SCION tunneling and a VPP buffer size that is not larger than every interface MTU. SCION tunneling does not work correctly with such a configuration.
- A
management.health.addressthat uses the same IP address and port as a different service, for example the telemetry address or the endhost API address, or a health port in the reserved range 48000-48999. The two services could not both listen on the same address. - A shuttle server with overlapping advertised routes. Refer to Shuttle advertised routes must not overlap.
The appliance runs the endhost API
The appliance now runs the endhost API itself. An endhost API that you manually installed on the appliance conflicts with the endhost API of the appliance. Remove it before you upgrade the appliance.
Check if the endhost API is running on the appliance:
systemctl status endhost-api-server.service
To remove the endhost API, run these commands on the appliance:
sudo systemctl disable --now endhost-api-server.service
sudo rm /etc/systemd/system/endhost-api-server.service
sudo systemctl daemon-reload
sudo rm /usr/local/bin/endhost-api-server
sudo rm -r /etc/scion/endhost-api
After the upgrade, configure the endhost API in the appliance configuration:
- Set
scion.endhost_api.address. Update the applications that use the endhost API to use this address. - Set
scion.endhost_api.allowed_sourcesto restrict the access to specific IP prefixes. - Remove the firewall rule that you added manually for the endhost API.
v0.42.0
| Release date | Nov 18, 2026 |
| Download | Release portal |
Improvements
-
The shuttle client and server now support SCION paths with a small MTU. The size of the QUIC packets is the route MTU (the
mtuoption) plus 46 bytes. If this size is less than 1200 bytes, the shuttle client uses the SCION version of QUIC, which allows smaller QUIC packets. You can upgrade the shuttle clients and servers in any sequence. Other changes:- If an IP packet is too large for the tunnel, shuttle sends an ICMP "fragmentation needed" (IPv4) or "packet too big" (IPv6) message to the sender. Before, shuttle dropped the packet silently.
- The MTU of the TUN device is the route MTU, or 1280 bytes if the route MTU is smaller.
- The shuttle client and server do not start if the
mtuoption is missing, less than 100, or more than 1300.
-
The gateway IPFIX flow export now counts the traffic in both directions. For the flows from remote ASes,
src_isd_asis the remote ISD-AS anddst_isd_asis the local ISD-AS. -
The gateway sends the gateway discovery request directly to the discovery service address of the remote AS. The remote control service puts this address into its beacons. If a path does not contain this address, the gateway uses SVC resolution as before.
-
The gateway
/healthendpoint shows the new health checkGateway certificate for local AS available(check ID2010) for each local ISD-AS. The status isfailingif the gateway cannot load a valid certificate chain, anddegradedif the AS certificate expires within 48 hours. The status isnoticeif only a TRC in grace period can verify the AS certificate. -
The new
SCIONCryptoASCertificateExpiredandGatewayASCertificateExpiredalerts fire when the AS certificate has expired. TheSCIONCryptoASCertificateExpiresSoonandGatewayASCertificateExpiresSoonalerts now fire only while the certificate is still valid. -
Every alert now has a
runbook_urlannotation with the link to its entry on the runbook page. -
The Grafana dashboards now have a
Data Sourcetemplate variable for the Prometheus data source. You can import the dashboards into any Grafana instance. -
The control service health check
Certificate for local AS availableshows the expiry time of the AS certificate if the status isdegraded. -
The configuration validation now accepts BGP together with egress SNAT on the
scion-gatewayinterface. The validation still rejects BGP together with ingress SNAT. -
appliance-cli infoshows the configuration creation time and the license expiry time in UTC. -
FRR is updated to 10.6.2. The OpenTelemetry collector is updated to 0.165.0, and node-exporter is updated to v1.12.1.
-
The dependencies of the appliance are updated to fix known vulnerabilities. This includes the Go toolchain 1.27.4,
golang.org/x/net,golang.org/x/crypto,google.golang.org/grpc,github.com/go-jose/go-jose(v3 to v4),github.com/klauspost/compress,github.com/getkin/kin-openapi,rustls,hyper, and the distroless and Ubuntu base images.
Fixes
-
The
GET /debug/network/interfacesendpoint now lists the VLAN interfaces. Withforce-fetch=true, the endpoint now fetches the current interface state from Linux and from the dataplane. On a VRRP master, the response also contains the VRRP virtual addresses. -
The appliance now closes the QUIC connections to the hidden segment directory after each AS lease renewal and after each hidden segment registration. Before, the number of open connections and UDP sockets increased over time.
-
The appliance now adds the management API listening addresses to Caddy when the Caddy configuration has an empty listener list. Before, Caddy did not listen on the new addresses.
-
A change of
cluster.synchronization.addressnow also changes the source address of the requests to the peers. -
The appliance controller now stops correctly when cluster synchronization is on. Before, the stop could wait until systemd killed the process.
-
The appliance controller no longer reloads the gateway when the IP-in-SCION tunneling configuration does not change.
-
The AS certificate renewal now stops when the caller cancels it, for example when the client of
POST /cppki/certificates/renewdisconnects. -
The control service
/healthendpoint now reports the correct status for an AS certificate that only a TRC in grace period can verify. Before, the status was alwaysnotice, also for an expired certificate. The endpoint also reports the correct local ISD-AS instead of0-0if no valid signer exists. -
The configuration validation now checks the
control-portand theprobe-portof the IP-in-SCION endpoint, and reports the errors of all static announcements. -
dataplane-control no longer crashes when a configuration update replaces an interface address with an address in the same subnet, and the interface has static routes or a VRRP group.
-
The "Interfaces Config Applied" health check now fails if the last synchronization failed. The
dataplane_control_dataplane_sync_errormetric is now also set if dataplane-control cannot fetch the current dataplane state before a synchronization. -
The appliance controller now restarts dataplane-control each time that it starts the dataplane. Before, a configuration change that restarted the dataplane could make the next dataplane-control synchronization fail.
-
dataplane-control now synchronizes overlapping addresses on one interface correctly.
-
The router now applies the configured SCION IPFIX
flush_interval, and exports the IPFIX records for traffic on SCION links with a UDP socket. The router no longer crashes when SCION IPFIX is disabled on an interface. -
The dataplane now exports SCION IPFIX records in stateless mode, and keeps the IPFIX packets within the path MTU. The SCION IPFIX active flows metric is now 0 after you disable IPFIX on the last interface.
-
The gateway path monitor now keeps at least one monitored path for each local egress interface. Before, a failure of one interface could stop all monitored paths at the same time, and the announced prefixes flapped. The path monitor also analyzes all paths again when a local interface becomes usable again. Before, the path monitor waited for the next scheduled analysis, which can be one hour later.
-
The SCION daemon now looks up hidden segments at the hidden segment directory (HSD) after a reboot or an upgrade of the appliance. Before, the daemon could start before the internal interface of the appliance had its IP address. The HSD requests then failed with a QUIC timeout until a restart of the daemon.
-
appliance-cli info scionnow shows the AS lease at the HSD for each AS. The "AS leased at HSD" health check now has theisd_asfield in itsdataobject. -
Some debug endpoints of the appliance API caused a panic in the appliance controller if an optional part of the configuration was absent. These endpoints now properly return.
-
POST /tools/scion/showpathsnow requests fresh paths from the control service if therefreshparameter istrue. Before, the appliance ignored therefreshparameter. -
The appliance now configures a VF on a Mellanox NIC if the configured VF name is different from the Linux name of the VF. Before, the network configuration failed for such a VF.
-
The appliance now gives a Mellanox PF its Linux name back when the PF changes from a VPP interface to a Linux interface or to the parent of VFs. The appliance also restores the IPv6 and ARP settings of the PF. Before, the PF kept the name
_<name>, and the creation of the VFs failed. -
dataplane-control no longer crashes after a restart of the dataplane or of dataplane-control if a VF on a Mellanox NIC (mlx5) has a configured MAC address. The appliance now applies the configured MAC address correctly.
-
The shuttle client no longer stops its reconnect attempts permanently. Each connection attempt now has a timeout of 15 seconds.
-
The shuttle client and server now wait for the SCION daemon at startup. Before, they could exit on a fresh boot.
-
The shuttle client keeps its known paths until they expire, also if the SCION daemon briefly returns no path to the shuttle server.