We wrote this blog as a refresher or quick guide based on the many resources and publications on the Internet. We all have different interpretations of the same objective, but sometimes, it is good to cross-check our interpretations with what others may think.

Cisco ACI for Enterprise Businesses in Data Centers | Article

Everyone is talking or using the Application Centric Infrastructure (ACI) in the networking world. Let’s begin with some questions.

What is Cisco ACI?

ACI stands for Application Centric Infrastructure and is a Cisco SDN solution for a Data Center environment. ACI is a way to create a common policy-based framework for the IT environment. Specifically across the Application, Networking and Security domains. Policy-Based is a set of guidelines or rules that determine a course of action. An example would be traffic going from a web server to the end host, which must pass through a firewall. Try to visualise, like QoS, Security and SLA’s

What are the key features/benefits?

• Automation
• Focus on Applications
• Integration Capabilities
• Virtualisation
• Container networking
• Orchestration
• Public Cloud networking

Why ACI?

• Leaf-Spine-Leaf Topology – Simple and Scalable
• ECMP – Routing Ethernet in Active/Active fashion
• East-West traffic optimisation, Anycast gateway on each Leaf
• Microsegmentation – Same Subnet? Not a problem at all!
• Security – White list policy by default

What are the ACI components?

• Switches -> Roles: Leaves and Spines
• Nexus 9K Modes: ACI for ACI and NX-OS for stand-alone usage
• Controllers: Application Policy Infrastructure Controller (APIC). UCS-C Server -> various capacity for different fabric size. Non-Cisco hardware is not allowed; it won’t work.

ACI Architecture

Please see the Figure below. The golden rule is that Spine Switches must be connected to all Leaf switches and vice-versa. However, Spine’s are not connected to each other and Leaves cannot connect either. Servers can only be connected to Leaves and NOT Spines. If a server is connected to the Spine – MCP (MisCabling Protocol) will detect that and stops the connection. LLDP (Link Layer Discovery Protocol) will disallow connections Spine <> Spine and Leaf <> Leaf

Spine-Leaf Topology

• 40 Gbps IP based Fabric with Integrated VXLAN overlay -> 100Gbbps capable
• Simple/Consistent/Scalable Fabric
• Composed of N9K devices, 9500 switches as the Spine’s (at least 2x for redundancy) used for Fabric Bandwidth
• Cisco 9300 switches at the Leaf layer (ToR – Top of the Rack). End devices, typically servers, VMWare chassis connect here.

ACI – Spine and Leaf Underlay Routing

• IS-IS (routing protocol) provides underlay routing
• In the scope are: IP unnumbered interfaces, L1 only (internal) connections, Advertises VTEP addresses, Generates multicast FTAG tree’s, Identifies and announced tunnels

What is VTEP?

Frame encapsulation is done by an entity known as a VXLAN Tunnel Endpoint (VTEP.) A VTEP has two logical interfaces: an uplink and a downlink. The uplink is responsible for receiving VXLAN frames and acts as a tunnel endpoint with an IP address used for routing VXLAN encapsulated frames (from Cisco Portal)

What is the APIC?

Application Policy Infrastructure Controller – APIC, is the main component of the ACI solution. It provides automation and management for the Cisco ACI fabric, policy enforcement, and health monitoring. The controller optimises performance and manages and operates a scalable multitenant Cisco ACI fabric.

• APIC is the Policy Controller in ACI
• Highly Redundant Cluster: typically three or more APIC’s for redundancy and quorum. They are NOT in Active/Standby configuration. They are in Active/Active deployment, and data is shared across the nodes. Each shard has 3x replica’s across controllers.
• APIC is NOT in control, or data plane of the fabric. Once the network environment is configured, and the APIC is down, it won’t affect the infrastructure. However, APIC is required for moves/adds/changes/deletions and any daily operations. So you have to have APIC in the long run. Your network can survive without it for a short while.

ACI – Fabric Discovery

• The APIC is responsible for: Fabric discovery and addressing, Image management, Topology and cabling validation.
• Fabric Discovery is made via Link Layer Discovery Protocol (LLDP), ACI specific TLV’s (OUI) and APIC management connection to infrastructure-vrf

Chicken or Egg? How do they discover each other?

ACI in the discovery process uses Intra-Fabric Messaging (IFM) method in which APIC and nodes exchange heartbeat messages. The technique used by the APIC to push policy to the fabric leaf nodes is called as IFM Process. In the final stage processes the discovery of the other leaf nodes and APICs in the cluster.

• Bootstrap API
• Leaf switch discovers APIC via LLDP, requests TEP address and boot file from APIC.
• Spine switch will find Leaf, requests TEP and boot file from APIC.
• Fabric now self-assembles
• When multiple APIC’s are discovered on the AV (Appliance Vector), they will form a resilient cluster.

What is Cisco ACI Tenant?

An ACI Tenant object model represents the highest-level object. Inside, you can differentiate between the objects that define the tenant networking, such as private networks (VRFs), bridge domains and subnets; and the objects that define the tenant policies such as application profiles and endpoint groups.

• Tenant – a logical unit for management
• Can be Customers, Business Units (BU’s) or Groups
• Allows for: Separate administration and data flows, Reusable IP address space, Distinct profile space.
• Three Default Tenants: Common – Provides common services to all tenants, Infra – used for All internal fabric comms, Mgmt – used for in-band and out-of-band management access policies.

Let’s build ACI like Lego Bricks

Context – a VRF within a Tenant

• Tenants may have one or more contexts, allows for IP address duplication

Bridge Domain – container for subnets

• These are necessarily VXLAN, using IRB functionality: Traffic within a BD is bridged, Traffic between BD’s is routed, Subnets are, therefore, irrelevant as traffic is routed based on ./32 host routes.
• Layer 2 flooding is disabled by default; it can be enabled within Bridge Domain for ARP, DHCP and integration of CE.

How to Manage, OOB Access?

Managing the Fabric, Cisco Nexus 9K Mgmt scope

• In-band, via the infra and management VRF’s, Console ports, Out-of-Band dedicated Management Port (like other Nexus devices, N5k and N7k)
• APIC Mgmt scope; Fabric ports (2x data), OOB Mgmt, Console ethernet, CIMC/IPMI

How does ACI Forwarding in the fabric?

In a nutshell, if a server connected to a Leaf switch wants to communicate with the other server somewhere else on the LAN, Leaf will do a lookup its ‘Local Station Table’ for a VTEP (Virtual Tunnel Endpoint). If it can’t find it there, it will try ‘Global Station Table’. Still, if it can’t find it there either from previous communications, it will ask the Spine switch. The Spine(s) know everything, and they will see a VTEP entry to forward the traffic to the destination.

Forwarding, Channeling your inner LISP.

• Layer 2 and Layer 3 is forwarded based on the destination IP, Intra and Inter Subnet.
• Each Leaf switch has 2x forwarding tables: Global Station Table -> Cache of Fabric endpoints, Local Station Table -> Hosts directly attached to Leaf, or off of Leaf, do ‘show endpoint’ in CLI.

Pervasive SVI Gateway

• No HSRP or VRRP, Available on all Leafs (where endpoints reside), Similar to the Distributed IP AnyCast GW in VXLAN eVPN

Management Protocols and Interface Policies for ACI

• Cisco Discovery Protocol (CDP) – default policy is ‘off’ -> used in ‘interface policies’
• Link Layer Discovery Protocol (LLDP) – default policy is ‘enabled’ -> used in ‘interface policies’
• Network Tim Protocol (NTP) – you can use in-band or out-of-band NTP, depending on the MGMT scheme the fabric utilises
• Domain Name Services (DNS) – useful and can be necessary for hostname to IP address resolution

ACI, Fabric Access Policies

VLAN pools represent blocks of traffic VLAN identifiers. A VLAN pool is a shared resource and can be consumed by multiple domains such as VMM domains and Layer 4 to Layer 7 services.
Each pool has an allocation type (static or dynamic), defined at the time of its creation. The allocation type determines whether the identifiers contained in it will be used for automatic assignment by the APIC (dynamic) or set explicitly by the administrator (static). By default, all blocks contained within a VLAN pool have the same allocation type as the pool, but users can change the allocation type for encapsulation blocks contained in dynamic pools to static.

• Namespace policy defines ID ranges used for VLAN encapsulation. Specifies the Vlan’s that may be used by a domain (kind of like an ‘allowed list’). 1x Vlan pool per domain
• 2x Modes of Operations: Static allocation – Used with bare-metal servers, Layer 2 / Layer 3 handoffs for actions like ‘static path bindings’, Dynamic allocation – APIC dynamically pulls a Vlan out of the pool (familiar with VMM implementations)

The ACI fabric can automatically assign VLAN IDs from VLAN pools. It saves a tremendous amount of time, compared with trunking down VLANs in a traditional Data Center.

The Domains – Fabric Access Policies

Domains act as the glue between the configuration done in the fabric tab to the policy model and endpoint group configuration found in the tenant pane. The fabric operator creates the domains, and the tenant administrators associate domains to endpoint groups.

• They are called  Domains, because ‘how’ devices/elements connect to the fabric.
• Physical – used for Bare-Metal hosts/servers.
• External Bridged – used for external Layer 2 connections, to an Outside Switched Network
• External Routed – used to connect to an external Layer 3 device for routing into/out of the fabric.
• VMM – used to connect to a hypervisor controlled environment like vCenter, OpenStack, o MS SCVMM

Attachable Access Entity Profile (AAEP) or (AEP)

An Attachable Entity Profile (AEP) represents a group of external entities with similar infrastructure policy requirements. The infrastructure policies consist of physical interface policies that configure various protocol options, such as Cisco Discovery Protocol (CDP), Link Layer DiscoveryProtocol (LLDP), or Link Aggregation Control Protocol (LACP).
An AEP is required to deploy VLAN pools on leaf switches. Encapsulation blocks (and associated VLANs) are reusable across leaf switches. An AEP implicitly provides the scope of the VLAN pool to the physical infrastructure.

• You will typically have one AEP per tenant.
• A group of ‘external’ entities with a similar policy, Required to deploy VLAN pool on Leafs, Defines the range, but does NOT provision
• Brings together the interfaces and VLAN’s so that the APIC knows where to deploy VLANs (i.e. what Leaf switches to push VLAN’s too)
• AAEP’s contain Domains and are
• held by Interface Policy Groups

ACI Endpoint Groups (EPG’s)

Endpoint groups (EPGs) are used to create logical groupings of hosts or servers that perform similar functions within the fabric and that will share similar policies. Each endpoint group created can have a unique monitoring policy or QoS policy and are associated with a bridge domain.

• EPG’s are groups of applications and/or entities independent of networking formula (i.e. VLAN’s, IP’s, etc…)
• Normally similar in nature (i.e. web, database, application servers)
• Group of endpoints that require similar policy: Outside networks, Groups of Servers/Applications, Network services, Storage devices
• Types of EPG’s include: Application EPG, Layer 2 External EPG, Layer 3 External EPG, Management EPG (Mgmt, OOB and Inbound)

• EPG’s are flexible and extendable
• EPG’s are the policy enforcement point for the group objects
• The policy is NOT enforced by subnet(s)
• IP address changes will not affect policy unless the endpoint is defined by IP address
• Nodes within an EPG can communicate.
• Nodes between EPGs must have a ‘contract’ in place in order to communicate.

Contracts – connecting all together

• Contracts dictate how EPG’s inter-communicate, define inbound/outbound permit and denies, QoS, redirects and service graphs
• Work in a provider/consumer model; an EPG can provide a contract that another will consume.

How to Get Started Leveraging AI?

New innovative AI technology can be overwhelming—we can help you here! Using our AI solutions to Extract, Comprehend, Analyse, Review, Compare, Explain, and Interpret information from the most complex, lengthy documents, we can take you on a new path, guide you, show you how it is done, and support you all the way.
Start your FREE trial! No Credit Card Required, Full Access to our Cloud Software, Cancel at any time.
We offer bespoke AI solutions ‘Multiple Document Comparison‘ and ‘Show Highlights‘

Schedule a FREE Demo!

Now you know how it is done, make a start!

Download Instructions on how to use our aiMDC (AI Multiple Document Comparison) PDF File.

Decoding Documents: v500 Systems’ Show Highlights Delivers Clarity in Seconds, powered by AI (Video)

AI Document Compering (Data Review) – Asking Complex Questions regarding Commercial Lease Agreement (Video)

v500 Systems | AI for the Minds | YouTube Channel

Pricing and AI Value

‘AI Show Highlights’ | ‘AI Document Comparison’

Let Us Handle Your Complex Document Reviews

Explore our Case Studies and other engaging Blog Posts:

Power of AI in Multiple Document Comparisons in the Insurance Sector

How do we extract critical information from a Public Company Annual Report?

How AI Multiple Document Comparison Benefits Core Sectors Across the Globe

Manual vs AI Automated: Unlocking the Secrets of Multiple Document Comparison

Assisting forward-thinking professionals and SMBs in unlocking document information by leveraging AI. Our Vision and Mission

Maksymilian Czarnecki

The Blog Post, originally penned in English, underwent a magical metamorphosis into Arabic, Chinese, Danish, Dutch, Finnish, French, German, Hindi, Hungarian, Italian, Japanese, Polish, Portuguese, Spanish, Swedish, and Turkish language. If any subtle content lost its sparkle, let’s summon back the original English spark.