Stretched Active/Standby Firewall#

I must define an ACI L4-7 Device with two Concrete Devices; One for the Active node and one for the Standby node. See ACI Multi-Pod and Service Node Integration, page=5. The active firewall is in pod A, the standby firewall is in pod B. Support east-west and north-south traffic flows. No risk of asymmetric traffic flows because there is only one active firewall at a time.

Impact of traffic hair-pinning#

Hair-pinning is a sub-optimal yet not avoidable traffic pattern. Beware of the latency impact of any hair-pinning traffic between pods due to traffic traversing the active firewall. The worst-case occurs when:

  • neither ingress or egress leaves learned about the remote endpoint (from their own perspectives of course),
  • both switches are in separate ACI pods,
  • and the active PBR node is not on the same pod as the egress leaf. Both North South and/or East West traffic flows may be impacted by hair pinning. => Provision enough bandwidth on the IPN network to accomodate such non-optimal traffic flows.

Design 1: The Active Firewall is the IP Default Gateway of the end user subnet#

sc1

Design 2: Two-arm Design with VRF Sandwich, L3Out Peering, no PBR#

This design is the application of the L4-7 service insertion in single-pod with VRF sandwich design to a multi-pod context. Two supported designs:

  • one L3Out connected to each service node in each pod.
  • a single L3Out connected to both service nodes across the multi-pod domain. Having L3Out with SVIs creates one stretched external bridge domain.

[!Important] VRF Sandwich or L3Out peering? Both concepts are required in this design option. #PersoNote Without L3Outs and VRF sandwich, Web EPG endpoints might reach the external networks through routing lookups in the fabric bypassing the firewall (if everything else is configured correctly).

See the document ACI Multi-Pod and Service Node Integration, page=19. in other way: sc2

The L3Out interface profiles of the service leaves and the firewall interfaces might be all in the same IP subnet, whether there is one firewall-internal and one firewall-external L3Out for both pods or per pod. #QA what good reason would make me put the L3Out interfaces and the firewall interfaces in different IP subnets? If not same IP subnet and ACI peers with the firewall using dynamic routing in the L3Out, then routing adjacencies are built between service leaves and the active firewall only, because the standby firewall is not participating in layer-3 operations. When a failover occurs, routing adjacencies must be established between AC border leaves and the new active firewall. If static routing is instead configured on the L3Out, then:

  • the service leaves are configured with a static route with next hop == the HA firewall vIP,
  • I must configure the secondary IP address on the service leaves and set a static route on the firewall (the active and the standby firewalls) with next hop the secondary IP address of the service leaves.
  • I must set IP SLA tracking on the static ip route with next hop the firewall vIP. Otherwise, ACI does not correctly detect the active firewall and since border leaves redistribute configured L3Out static routes in MP-BGP VPNv4, traffic from ACI to the outside may go to the wrong firewall. #QA how to configure IP SLA for static routing in L3Out? What behaviour does IP SLA influence?

VRF Sandwich and dedicated L3Outs per Pod#

In each pod: one L3Out for the firewall’s internal interface and another L3Out for the firewall’s external interface. #ciscoNOTrecommends design. Having L3Out with SVIs creates ‘disconnected’ local external bridge domains, one in each L3Out. #PersoNote There should be a HA link between the firewalls in case of a failover. And this HA link is not in the L3Out. If static routing is configured on the L3Out, then:

  • I must configure the service leaves with a static route with next hop == the HA firewall vIP,
  • I must configure the secondary IP address on the service leaves and set a static route on the firewall (the active and the standby firewalls) with next hop the secondary IP address of the service leaves.
  • I must set IP SLA tracking on the static ip route with next hop the firewall vIP. Otherwise, ACI does not correctly detect the active firewall and since border leaves redistribute configured L3Out static routes in MP-BGP VPNv4, traffic from ACI to the outside may go to the wrong firewall. #QA how to configure IP SLA for static routing in L3Out? What behaviour does IP SLA influence?

VRF Sandwich and stretched L3Outs between Pods#

One L3Out for the firewall’s internal interfaces and one L3Out for the firewalls’ external interfaces, across the multi-pod domain. If dynamic routing is deployed on the L3Outs, the service leaves establish routing adjacencies across the pods with the active service node only (since standby firewalls do not participate in the control plane). #CiscoRecommends activating Disable Remote Endpoint Learn to avoid problems when endpoints are initially attached on border leaves and then live-migrate. If static routing is configured on the L3Out, then:

  • the service leaves are configured with a static route with next hop == the HA firewall vIP,
  • I must configure the secondary IP address on the service leaves and set a static route on the firewall (the active and the standby firewalls) with next hop the secondary IP address of the service leaves.
  • no IP SLA tracking on the static route is required ( #QA since the external BD is stretched?)

Design 3: with PBR, no L3Out peering#

North-south traffic is redirected to the service node using PBR contracts. Rules of table 2 apply. Hair-pinning traffic is inevitable when one of the triplet (Provider EPG, Consumer EPG, PBR node)

with no VRF sandwiching#

Both one-arm and two-arm designs are supported. The PBR node interfaces might attach:

  • to the BDs of the (Provider, Consumer) pair, or
  • to service BDs.

for North/South traffic#

See the document ACI Multi-Pod and Service Node Integration, page=31.

Option 2: with symmetric PBR#

This design stipulates using PBR for both N/S and E/W traffic flows.

for North/South traffic#

See the document ACI Multi-Pod and Service Node Integration, page=58.

Requirements#
  • I need to direct traffic to the right active/passive firewall pair. For that, we need service graphs with PBR and the service devices need to be in routed mode.
  • Symmetric PBR. Otherwise host routes must be considered in the design.
  • The PBR policy contains as many destination groups as there are firewalls in the Active state, corresponding to the vMAC/vIP of the HA firewalls across the pods,
  • no L3Outs, no VRF sandwich.
  • I need two L4-7 Devices, each having two Concrete Devices. Both two-arm and one-arm designs are supported.

for East/West traffic#

requirements#

Symmetric PBR See the document ACI Multi-Pod and Service Node Integration, page=67.

Potential risks and solutions#

This design does not guarantee to maintain traffic symmetry if (Provider, Consumer) pair are in different pods when location-based PBR is enabled (one leg might traverse the HA firewall in pod A. The second leg might land on the HA firewall in pod B); The HA firewalls must support some kind of rerouting of traffic to maintain traffic symmetry. –> Use different interfaces for N/S and E/W. Apply one PBR policy for N/S and one PBR policy for E/W, both having Location-based PBR disabled.

[!NOTE] Symmetric PBR and Hashing Symmetric PBR ensures both communication legs of a traffic flow is steered to the same firewall. And load-balancing is part of symmetric PBR. Load-balancing of traffic is based on a hashing algorithm, which might select a PBR node that is not local to the ACI pod. –> Symmetric PBR may inherently introduce traffic hair-pinning.

Potential risks and solutions#

Risk Solution
hair-pinning traffic Activating Location-based PBR, i.e. the policy selects the local PBR nodes in the pod. But eliminating hair-pinning traffic 100% is not possible. Example: Endpoints A and B are in pods 1 and 2. For traffic from A to B, policy is applied at the egress leaf when border leaf in pod 2 does not know about B. So traffic hair-pins and PBR is applied at pod 1 although localtion-based PBR is applied in both pods. Ideally, the HA firewall systems shall be able to synchronize the connection state between them.
failure of a pod-local HA firewall leads to black-holed traffic because the traffic was originally steered at that HA firewall. The symmetric PBR hashing algorithm has a tracking mechanism that enables the leaf switch to automatically select a functioning PBR destination, despite of having location-based PBR enabled. See the document ACI Multi-Pod and Service Node Integration, page=64.

Option 3: with L3Out peering and PBR#

This design stipulates leveraging the routing semantics or VRF sandwiching to insert firewall inspection on N/S traffic (so no PBR), and PBR for E/W traffic. The same HA firewall can be used for inspecting North-South and East-West traffic flows:

  • ingress/egress North-South traffic into ACI traverses inevitably the firewall. So, North-South firewall inspection is done by way of regular layer-3 lookup.
  • East-West traffic leverages PBR.

North/South traffic#

Design 1#

See the document ACI Multi-Pod and Service Node Integration, page=70.

Only the inside interfaces of the firewalls are connected to ACI;

  • The firewall inside interfaces are connected using L3Out.
  • the firewall external interfaces are connected off-ACI to the WAN

Design 2: VRF sandwich#

  • Firewall inside interfaces are attached to ACI L3Out-A
  • Firewall outside interfaces are attached to ACI L3Out-B.

Risks and solutions#

Traffic hair-pinning –> implement host routing.

East/West traffic#

See the document ACI Multi-Pod and Service Node Integration, page=72.

#QA difference of this design for the E/W traffic with the design involving only symmetric PBR?