Known issues
Here, you find a curated list of all currently known issues affecting the Anapaya appliance software. Some issues may span multiple software releases. The known issues that apply to the most recent minor release can be found on the v0.40 release page. To stay up-to-date, we recommend visiting the release pages of the appliance software that you have installed, or checking the Anapaya CONSOLE.
Each entry provides details about the affected versions, workarounds (if available), and the patch status. This information is also integrated with the Anapaya CONSOLE, making it easy to track and manage known issues directly from your environment.
Our goal is to give you clear visibility into any challenges you might encounter and help you plan accordingly.
Check affected
Not all known issues affect every appliance. For example, some issues are only relevant if BGP is configured on the appliance. To check if your appliance is affected by a specific known issue, you can use the CEL expression provided with each issue. For quick evaluation, copy the expression into CEL Expression, and your appliance configuration into Input on the CEL playground. Finally, hit Run to see if the expression evaluates to true. Run to see if the expression evaluates to true. For a more detailed guide, check out our tutorial on how to use the CEL playground.
If you are using the Anapaya CONSOLE, the software page will automatically show you which known issues affect your particular appliance based on the currently installed version and configuration on the device. You can also find a report of all appliances on the Reports Dashboard.
Conventions
Known issues are identified by a unique identifier (KNI-YYYY-SSSS), where:
YYYYrepresents the year the issue was first reported (e.g., 2025).SSSSis a sequential number assigned to each issue reported within that year. (e.g., 0001).
Known issues are categorized based on their severity:
- Critical: Issues that severely impact system security, stability, or core functionality. Immediate remediation is required to prevent or mitigate service disruption or compromise. For security-sensitive cases, full details may be shared only upon request.
- High : Issues that substantially affect key features, reliability, or performance but do not render the system inoperative. Timely resolution is strongly recommended to maintain expected functionality and user experience.
- Medium: Issues that moderately affect system behavior, performance, or usability. These should be addressed in the normal course of maintenance to prevent potential degradation or escalation.
- Low: Issues with minimal impact on system functionality or performance. They pose little to no operational risk and can be resolved at the next convenient opportunity.
Registry
KNI-2025-0001
Migration potentially produces invalid configuration, if:
- the username or password in the loki basic auth configuration is unset
- invalid features in the experiments sections are present
KNI-2025-0002
The appliance fails to configure interfaces, if the gateway field is used on an
interface with VPP driver variants. To resolve, please configure a 0.0.0.0/0 and/or
::/0 route instead of using the gateway field. In a future release, the gateway field
will be deprecated.
KNI-2025-0003
BGP BFD does not work with certain vendors (e.g. Cisco, Arista)
KNI-2025-0004
Appliances that are configured with BGP will sometimes not export all routes learned via SCION to their BGP peers due to a race condition. FRR, the third-party BGP daemon used, will not recover unless the affected route is flushed.
KNI-2025-0005
The IP-in-SCION tunneling component does not correctly free buffers on ingress stream handling which could trigger a buffer leak in rare conditions. This bug could be triggered for example, by a local misconfiguration of allowed interfaces.
Once the gateway is out of buffers, it needs to be reset to recover:
appliance-cli post debug/services/gateway/restart
KNI-2025-0006
The gateway_as_certificate_expiration_time_second metric is not correctly updated if
there are no reachable remote gateways
KNI-2025-0007
The IP-in-SCION tunneling component may crash if it is receiving specifically crafted SCION packets For more details, contact Anapaya support.
KNI-2025-0008
When you disable security (encryption) on all IP-in-SCION tunneling domains, after previously having it enabled on at least one domain, the IP-in-SCION tunneling component crashes repeatedly which leads to service interruption.
To recover, please restart the affected components:
appliance-cli post debug/service-groups/data-plane/restart
appliance-cli post debug/services/gateway/restart
If this does not solve the issue, please reboot the appliance.
KNI-2025-0009
Appliances that are connected to two BGP peers with the same BGP ASN, will sometimes lose routes in the kernel routing table from the BGP peers and therefore not export all routes to the SCION network.
Quick fix and workaround
Quick fix
In case you detected that your appliance learns prefixes via a BGP session but does not propagate them via SCION, restart the BGP daemon such that the routing table is rebuilt:
appliance-cli post debug/services/frr/restart
Workaround
A configuration workaround without the need to install the new patch release is also available.
Refer to the advanced FRR
configuration,
select the template according to your appliance version and add the following line to the
BGP configuration section after ipv6 forwarding.
no zebra nexthop kernel enable
The zebra configuration section should look like this:
! Zebra configuration
!
ip forwarding
ipv6 forwarding
no zebra nexthop kernel enable
!
Make sure to apply the configuration change and restart the BGP daemon afterwards:
appliance-cli put config/advanced/service-customization/frr/template < frr.conf.tmpl
appliance-cli post debug/services/frr/restart
KNI-2025-0010
Deleting a secret and adding another secret with the same secret id requires a restart of the appliance controller in order for the change to take effect.
KNI-2025-0011
BGPRouteMismatch alert does not work correctly. It fires whenever there are any non-BGP routes in the Linux routing table.
KNI-2025-0012
Loss of IPv6 link-local addresses and therefore IPv6 connectivity issues may occur after any configuration change, for the interfaces that have no explicit IPv6 link-local address configured.
To mitigate, follow these steps:
-
Explicitly configure an IPv6 link-local address for the interface.
-
Explicitly toggle Linux IPv6 link-local address generation off and on. Note that this needs to be repeated after every configuration change.
echo "1" > /proc/sys/net/ipv6/conf/<intf_name>/addr_gen_mode && \
echo "0" > /proc/sys/net/ipv6/conf/<intf_name>/addr_gen_mode
KNI-2025-0013
When connecting to multiple BGP peers with the same AS number, there is a risk that routes are incorrectly retracted from being announced via IP-in-SCION tunneling because of a bug in the IP-in-SCION component. This is different from KNI-2025-0009.
KNI-2025-0014
The IP-in-SCION tunneling component may crash if there are a lot of path switches and concurrent health API access.
KNI-2025-0015
The IP-in-SCION tunneling component logs excessively at info level with very large messages, making log viewing tools too slow to react.
Workaround
Set the log level to error for the gateway service:
appliance-cli services log level gateway error
KNI-2025-0016
Firewall rule generation skips SCION cluster synchronization addresses in certain cases.
When multiple SCION ASes are configured, and at least one AS is a IP-in-SCION tunneling only host (without control/router service), the firewall rules potentially does not allow traffic for the cluster synchronization addresses.
The mitigation is to manually prepend the required firewall rules for all the control service addresses. Visit the firewall configuration documentation for more information. An example rule to prepend could look like this:
{
"rule": "ip daddr 10.0.0.1 tcp dport 40000 counter name \"accept_scion\" accept"
}
KNI-2025-0017
Software upgrades from releases prior to v0.39 may fail due to a race between restarting the appliance-secretstore socket and running the appliance configuration migration in the appliance-controller.
The race can manifest itself in two different ways:
-
The configuration migration is interrupted after secrets have already been written to the secret store. The appliance-controller will attempt a second migration after the restart, but fail with the following error in the logs of the appliance controller (
journalctl -u appliance-controller):The secret id already exists at the requested version: <name>@1"Workaround
Since the installation has completed, but the configuration migration has failed, we can manually clean up the state and restart the appliance-controller:
systemctl stop appliance-secretstore
rm -rf /var/lib/appliance-secretstore/
systemctl start appliance-secretstore
systemctl restart appliance-controllerNote that this will delete all secrets stored in the appliance-secretstore. However, since the migration will be re-attempted, the secrets will be re-created.
-
The post installation action to restart the appliance-secretstore fails resulting in an automatic rollback of the upgrade. You can observe the following error in the logs of the appliance installer (
journalctl -u appliance-installer):{"Cmd":"systemctl","Args":["restart","appliance-secretstore.socket"]},"output":"Job failed.Workaround
Since the installation has rolled back, you can re-attempt the software installation. Because it is a race condition, it is likely to succeed on the second attempt.
In case the rollback was triggered during configuration migration, you might face the issue described in scenario 1 after the installation. In that case, please follow the workaround described above.
KNI-2024-0001
Dataplane not starting due to failing kernel setting
KNI-2024-0002
IP-in-SCION tunneling metrics incomplete
KNI-2024-0003
GATE flow exporter crashes when large amount of flows are added and deleted
KNI-2024-0004
Adding or deleting traffic matchers blackhole affected prefixes
KNI-2024-0005
Non-canonical prefixes can crash IP-in-SCION tunneling service
KNI-2024-0006
CA frontend consumes 100% CPU
KNI-2024-0007
Default buffer and worker allocation could lead to non-running dataplane
KNI-2024-0008
IP-in-SCION tunneling requires all TRCs for local ISDs to be present
KNI-2024-0009
IP-in-SCION path monitoring has stale path under race condition
KNI-2024-0010
IP-in-SCION takes up to 1h to recover from all dead paths to remote
KNI-2024-0011
SCION RSS not supported on interfaces with LINUX driver