BGP and MPLS-Based VPNs

　　Introduction

　　Two months ago we started looking at MPLS, Multiprotocol Label Switching, and continued with MPLS last month. This technology is very hot among large Enterprises and Service Providers. If you missed them or want an electronic copy of them, the previous articles can be found at:

　　A caution: this is all fairly new stuff, I do not have equipment available to test it with (nor time), and am piecing together information from various sources. Thus the configurations are my best effort but are not guaranteed accurate.

　　What is a VPN?

　　A Virtual Private Network or VPN is a network implemented using a shared network infrastructure but so as to provide the security and privacy of a private leased-line network. Older examples would be Frame Relay and ATM. Lately VPN has come to more often refer to IPSec tunnels over the Internet, or perhaps PPTP or L2TP dial VPN connectivity across a shared internetwork.

　　For our purposes in this article, the VPNs will be IP networks where the WAN core of a corporate network has been outsourced to a Service Provider. The IP VPN connectivity is provided across a shared IP network belonging to the Service Provider. It will turn out the the BGP and MPLS-based VPNs we will talk about are powerful enough to provide secure connectivity (and relatively simple configuration) for both intranets and extranets.

　　Terminology:

　　Intranet -- VPN interconnecting corporate sites

　　Extranet -- VPN connecting corporate site or sites to external business partners or suppliers. The Internet is the ultimate insecure Extranet VPN.

　　Customer Edge (CE) router -- a router at a customer site that connects to the Service Provider (via one or more Provider Edge routers)

　　Provider Edge (PE) router -- a router in the Service Provider network to which Customer Edge Routers connect

　　Provider Core (Core) router -- a router in the Service Provider network interconnecting Provider Edge routers but, generally, not itself a Provider Edge Router

　　Entry and Exit PE routers -- the PE routers by which a packet enters and exits the Service Provider network

　　In the figure, imagine the red routers are connected with one VPN, and the blue ones with another. (I tried to draw in some lines to suggest connectivity, but things rapidly got rather cluttered). An extranet is where some red routers connect to some blue routers. The red path with arrow shows traffic from the bottom red CE router to the top one. The first (bottom) gray provider router is the entry PE router, and the final gray provider router is the exit PE router (terms used below).

　　Understanding MPLS-Based VPNs

　　I've been thinking of MPLS-based VPNs as basically using long IP addresses. That isn't exactly what's going on, but it is a key part of it.

　　Each site belongs to a VPN, which has a number. In the Cisco implementation, this number is configured as the 8 byte Route Distinguisher (RD). The route distinguisher number is used to prefix the IP addresses for the site. It is configured on the interface (or subinterface) connecting to the site. This gives us a way to tell duplicate private addresses apart, to distinguish them. For example, subnet 10.1.1.0 for VPN 23 is different than subnet 10.1.1.0 for VPN 109: from the MPLS VPN provider's point of view they are really 23:10.1.1.0 and 109:10.1.1.0, which are quite different. Putting the 8 byte route distinguisher in front of a 4 byte IP address gives us a 12 byte routing prefix. We regard these as the VPN-IPv4 family of addresses.

　　The multiprotocol extension to BGP4, MBGP, was invented to carry such routing information between peer routers. So once we think in terms of routing 12 byte prefixes, there is a natural way to propagate the information. For security and scalability, MBGP only propagates information about a VPN to other routers that have interfaces with the same route distinguisher value. That reduces the chance of accidentally leaking information about Customer A to Customer B (quite easily done with routing distribute lists in a tunneling approach, or with route maps or distribute lists or prefix lists and ordinary BGP). It also means that each PE router only tracks routes for the customers connected to that one PE router, not for the entire set of long prefixes for all sites and customers connected to the Service Provider. Scalability!

　　Another aspect of this is that core routers, not being connected to CE routers, don't learn VPN-IPv4 routes. We'll come back to this idea in a moment. This is desirable: it turns out we only need to run an IGP (Internal Gateway Protocol), so that core routers have routes to all PE routers. And from our prior discussions about MPLS, we suspect the IGP might be OSPF or IS-IS, to allow implementation of MPLS Traffic Engineering. Only tracking routes to PE routers keeps the core extremely scalable, and greatly simplifies the size of routing tables for core routers. This too enhances scalability!

　　So what we've got so far is long addresses, and tracking routing that builds in the VPN ID or route distinguisher as part of the routing prefix. The PE routers that share the long prefix routing information are all speaking MBGP, all within the same AS -- hence internal MBPG, or iMBGP. This behaves very much like ordinary BGP. Well, when iBGP speaking routers propagate routes, they also propagate attributes. One key attribute for Service Providers is the next hop attribute. For iBGP-speaking routers, the next hop is generally the exit point from the Service Provider network, the exit point used to reach the advertised destination prefix.

　　If we were to actually route based on the long addresses, we'd have to forward the packets hop by hop and do a routing lookup at each PE or core router between the entry PE router and the exit PE router. The problem with that is, we would then have to convert our IP header to use our longer addresses at the entry PE router, we'd have to have internal core routers that knew how to forward this new network-layer protocol, and then we'd have to strip out the longer addressing information at the exit PE router. This probably sounds sort of like what MPLS already does with labels -- but now we'd be doing it with actual network layer headers. Some readers might be thinking "aha! IPv6! Tunneling IPv4!". Nice thoughts, but ... WRONG!

　　I suppose the network layer code could have been written to support this, or IPv6 could have been used for a form of tunneling. But all of that would have cost time and work and money. Instead, the Cisco engineers who came up with this had a very clever idea. MPLS!

　　All that the entry PE routers need to do to packets is somehow deliver them to the appropriate exit PE router, the next hop known via the mandatory MBGP next hop attribute. But with MPLS and any IGP carrying routes to the PE routers, we will already have an MPLS Label Switch Path (LSP) from the entry PE to each possible exit PE! And that does it.

　　When a packet comes in, we look up the long (VPN) destination prefix in the MBGP routing information base (RIB). That tells us the next hop router, the exit PE router. We would normally look up how to get to that router in the IGP, and determine the IP next hop. But this gets short-circuited by MPLS: we find we have a label available for an LSP that delivers packets very efficiently to the MBGP next hop router, the exit PE router. And (here's the clever part) if we use the LSP, the core routers in the core never have to examine IP addresses or headers, they just use the labels to forward the packet!

　　So MPLS LSPs act as tunnels through the Service Provider core, meaning we can get away with an IGP in the SP core, and thus the SP core routers can remain ignorant of the many, many possibl