Overlay Networking

How to integrate Overlay Networking and the Physical Network

An overlay network uses modern tunneling protocols to connect software Network Agents in Hypervisors or Operating Systems. Today, these Network Agents are little more than “robot patch panels” (you probably call them vSwitch’s) but in the near future these agents will be complete networking devices performing switching, routing & filtering inside your server.

What Matters Next

Overlay Networking is still early in the technology adoption cycle but deployed by Cloud Companies such as Microsoft Azure, Rackspace & Joyent. It’s not an idea, it’s a foundational technology that means that networks can really support multi-tenancy for hosting providers. In the near future, Overlay Networks will completely change the way Enterprise Security operates by providing a way to create & manage unlimited network zones from a physical network without the hassle of MPLS or the expense of firewalls. This is huge change in Infrastructure Security and will obsolete many firewall products.

Overlay Networking is a “real” technology that appears to have plenty of momentum in the market, including new players like VMware which suggests that it is past the “Works in PowerPoint” phase. The real value of Overlay Networking is that networking moves away from “dumb connectivity” & into “network services”. Today, networking is a “cost center” that is must have to provide business services in the same way that power to a data center is a must have.

 Trust is in Short Supply

What matters for customers is that the Tunnel Fabric created by Overlay Networking must be reliable & trustworthy. For the last thirty years the networking industry has been overcoming the physical challenges of forwarding performance, density, & pricing. Let’s take the view that these problems have largely been solved & point out that networking vendors are looking for revenue growth by adding non-core networking functions to existing devices.

The result is that network engineers are well practiced in looking closely at hardware as key solution criteria for networking. In Overlay Networking, the physical network becomes less important from a service delivery perspective because business value is derived from the overlay through service creation & management. This conflict in perspective is creating cognitive dissonance among many networking professional who are not ready to adapt to the idea that networking can be a software centric technology. More importantly, hardware networking requires a lot of attention to maintain skills & competency. This type of “chin down” work doesn’t allow much energy for “chin up” activities to look for other, better technologies.

The Case for Integration & Dependency

What happens to the Overlay Network if the physical network is experiencing packet loss, jitter or some form of “negative service” condition? The “Case for Integration” states that the Overlay network (of whatever form) must have some awareness of the physical network condition. In normal operation, when a link in an ECMP path fails, the routing flaps in an L3 core & recovers the path. Or in a Layer 2 MLAG network there might be congestion on an uplink due to failure of an Ethernet connection in the bundle or congestion in a single link in the path.

An overlay network tunnel has no state in the physical network. The physical network has no awareness or visibility of the Overlay Flow that passes over the network. VXLAN is just another UDP/IP packet heading on down the network & any packet loss requires a retransmission.

If this vision of the network fills you with apocalyptic fear then some form integration between the physical network & the overlay network is needed to make you happy. Consider the following diagram that represents two VMs communicating via the Overlay Network & the potential path through the Underlay Network.

Network is not simple or necessarily easily defined. A network with only hypervisors (VMware vCloud, OpenStack etc) could have a high level of controller. But in more common networks, there may be hundreds of traffic sources in the Underlay Networks that could generate data and create a more complex system.

The various technologies used for Ethernet Fabrics such Clos Based Leaf/Spine for L2 or L3, MLAG or even STP in the Data Centre LAN mean that data flows are load balanced across available paths using different mechanisms. Predicting traffic is not practical (although technologies like Juniper QFabric are an attempt in this direction).

It’s fairly easy to develop concerns about these problems even when those concerns are completely without basis in fact. Don’t let vendors or colleagues attempt to undermine the future with Fear, Uncertainty and Doubt. These are exactly the same technical events that happens in  your network today.

The IP protocol is designed to drop packets and frames as needed and networks are expected to drop data by the applications. The point of careful design to only drop a minimal percentage at an acceptable rate over time.

 How Could Network/Overlay Integration work ?

In a controller-based Overlay Network, it would be possible to implement an application that “listens” to the network itself. An OSPF software agent on the controller configured as a neighbour in “listen” mode can gather a comprehensive view of IP paths and network change from the OSPF state database. Similarly, a Spanning Tree listener could receive Topology Change Notifications & gain insight on the Ethernet stability (but not much on the network graph).

More comprehensive approach would be to create software applications on a controller that would gather information from each device in the physical network. A starting point would be to use SNMP for basic information. A vendor might develop a proprietary API  that would expose a wide range of other information. Collate that information into a network graph, invest a large sum into software development & then you will have a Network Controller system that is tightly integrated into the Network Fabric. Then have the Network Controller also interface to the Network Agents.

To integrate an overlay would imply that there is a “feedback loop”between the physical network & overlay network in some form or other.

Another option is to build an Ethernet Fabric that is single network. Something like what Juniper QFabric had done to create a comprehensive end-to-end Ethernet system that acts like a communist dictator for a socialist network.

The Case For Abstraction & Isolation

The IP protocol has proven itself to be robust, reliable & workable for a wide range of solutions. Twenty years ago, all telephones used dedicated circuits to guarantee the quality of the call.

Try using Skype & rarely have problems with a call. An Overlay Network built on tunnels will be guaranteed to work without any interaction with the physical network provided there is enough bandwidth in the underlying network.

And if there is some packet loss, the Network Agents will resend it the data.