Data Center Structure Management and Automation with Cisco DCNM and Spine‑Leaf Architectures

Building a data center has no single method, and managing its structure also lacks a one‑size‑fits‑all solution; Cisco, third‑party, and open‑source tools provide diverse capabilities for monitoring, managing, automating, and troubleshooting data‑center infrastructure.

Cisco Data Center Network Manager (DCNM)

Cisco DCNM is a unified management system for data‑center structures, enabling configuration, monitoring, and fault isolation. It can be installed in four modes: Classic LAN (for traditional Nexus designs), Media Controller (IP‑based media solutions, not covered here), SAN Controller (for MDS switches, not covered here), and LAN Structure (providing a structure generator for automated VXLAN EVPN deployment, overlay provisioning, end‑to‑end flow tracking, alarms, troubleshooting, compliance, and device lifecycle management).

DCNM version 11.2 adds support for Cisco Network Insights applications, namely Network Insights‑Advisor (NIA) for proactive monitoring and issue prevention, and Network Insights‑Resources (NIR) for inventory and activity overview.

Conclusion

The document reviews several Cisco spine‑and‑leaf designs, highlighting key technical components and design considerations for each architecture.

Cisco FabricPath spine‑leaf networks are proprietary, offering simplicity, flexibility, stability, scalability, fast convergence, and Layer 2 multipath, but rely on flood‑and‑learn mechanisms that can cause broadcast overhead as host counts grow.

Cisco VXLAN flood‑and‑learn spine‑leaf networks conform to IETF VXLAN (RFC 7348) and also use flood‑and‑learn, facing similar scalability challenges.

Cisco VXLAN MP‑BGP EVPN spine‑leaf architecture uses MP‑BGP EVPN as the control plane, aligning with RFC 7348 and RFC 8365, providing separation of control and data planes, distributed anycast gateways, ARP suppression, and robust multi‑tenant support.

MP‑BGP EVPN is an industry‑standard protocol enabling multi‑vendor interoperability.

It allows control‑plane learning of Layer 2/3 reachability, supporting scalable multi‑tenant VXLAN overlays.

Leverages a decade‑old MP‑BGP VPN foundation for large‑scale deployments.

EVPN address families carry both Layer 2 and Layer 3 reachability, enabling integrated bridging and routing.

Protocol‑based MAC‑to‑IP routing and ARP suppression reduce network flooding.

Distributed anycast on leaf ToR switches optimizes east‑west and north‑south traffic and supports workload mobility.

Provides VTEP peer discovery and authentication, lowering malicious VTEP risk.

Enables active‑active multi‑addressing at Layer 2.

Management tools simplify visibility, troubleshooting, and automation of structural components and tenant networks.

The following table compares the four Cisco spine‑and‑leaf architectures discussed: FabricPath, VXLAN flood‑and‑learn, VXLAN MP‑BGP EVPN, and MSDC Layer 3.

Cisco Spine-and-Leaf Layer 2 and Layer 3 Fabric

Cisco FabricPath

Cisco VXLAN Flood and Learn

Cisco VXLAN MP‑BGP EVPN

Cisco MSDC Layer 3

Transport medium requirement

Layer 1

Layer 3

Layer 3

Layer 3

Encapsulation

FabricPath (MAC‑in‑MAC frame encapsulation)

VXLAN (MAC‑in‑IP packet encapsulation)

VXLAN (MAC‑in‑IP packet encapsulation)

–

Unique node identifier

FabricPath switch ID

VTEP

VTEP

Layer 3 IP address or loopback address

End‑host detection

Flood and learn

Flood and learn

Localized flood and learn with ARP suppression

None (localized IP subnet)

Silent host discovery

Yes

Yes

Yes

No

End‑host reachability and distribution

Flood and learn plus conversational learning

Flood and learn

MP‑BGP EVPN

Unicast routing protocol (eBGP)

Broadcast and unknown unicast traffic

Flood by FabricPath IS‑IS multidestination tree

Forwarded by underlay PIM or ingress replication (Note: Ingress‑replication is supported only on Cisco Nexus 9000 Series Switches.)

Forwarded by underlay PIM or ingress replication (Note: Ingress replication is supported only on Cisco Nexus 9000 Series Switches.)

Stops at leaf ToR switch

Underlay control plane

FabricPath IS‑IS

Any unicast routing protocol (static, OSPF, IS‑IS, eBGP, etc.)

Any unicast routing protocol (static, OSPF, IS‑IS, eBGP, etc.)

Unicast routing protocol (eBGP)

Overlay control plane

–

–

MP‑BGP EVPN

–

Layer 3 gateway

● Internal and external routing at border spine ● Internal and external routing at border leaf ● Up to 4 FabricPath anycast gateways supported

● Internal and external routing at spine VTEP ● Internal and external routing at border leaf VTEP ● Up to 2 active‑active gateways with vPC supported

● Distributed anycast gateway on leaf ToR switch for inter‑VXLAN routing ● Border leaf switch for external routing (Note: The spine switch only needs to run BGP‑EVPN control plane and IP routing.) ● Border spine switch for external routing (Note: The spine switch needs to support VXLAN routing on hardware.)

● Leaf ToR switch for internal routing ● Border leaf switch for external routing

Layer 2 VXLAN gateway

–

Leaf ToR switch

Leaf ToR switch

–

Multicast traffic

Supports: ● Layer 2 multicast traffic (forwarded by multidestination tree) ● Layer 3 IP multicast traffic (forwarded by Layer 3 PIM)

Supports: ● Layer 2 multicast traffic (forwarded by underlay PIM) ● Layer 3 IP multicast traffic (forwarded by Layer 3 PIM)

Supports: ● Layer 2 multicast traffic (forwarded by underlay PIM or ingress replication (Note: Ingress‑replication is supported only on Cisco Nexus 9000 Series Switches.)) ● Layer 3 IP multicast traffic (forwarded by Layer 3 PIM‑based multicast routing on external router or Tenant Routed Multicast (TRM) (Note: TRM is supported on Cisco Nexus 9000 Cloud Scale Series Switches))

Supports: ● Layer 3 IP multicast traffic

Multi‑tenancy

● Layer 2 multitenancy with VN‑segment ● Layer 3 multitenancy with VRF‑lite

● Layer 2 multitenancy with VNI ● Layer 3 multitenancy with VRF‑lite

● Support for both Layer 2 multitenancy and Layer 3 multitenancy

No

Standard reference

TRILL‑based (Cisco proprietary)

RFC 7348

RFC 7348 and RFC8365 (previously draft‑ietf‑bess‑evpn‑overlay)

Routing protocol

Supported hardware

● Cisco Nexus 7000 Series Switches including the Cisco Nexus 7700 platform switches ● Cisco Nexus 5500 and 5600 platform switches ● Cisco Nexus 6000 Series Switches

● Cisco Nexus 7000 Series Switches including the Cisco Nexus 7700 platform switches ● Cisco Nexus 9000 Series Switches

● Cisco Nexus 7000 Series Switches including the Cisco Nexus 7700 platform switches ● Cisco Nexus 9000 Series Switches

● Cisco Nexus 7000 Series Switches including the Cisco Nexus 7700 platform switches ● Cisco Nexus 3000 Series Switches ● Cisco Nexus 9000 Series Switches