Internet Trends Driving Scale
The internet is a constantly evolving network. While its primary purpose continues to be that of providing global connectivity, we are in the digital era where much of the growth is occurring at the edge. There are several reasons for edge networks to be richly connected – delivering content over optimal paths, being geographically close to the consumer for better user experience, continuous growth of augmented reality (AR) and virtual reality (VR) applications over 5G connectivity demanding low latency, etc.
In order to increase their presence at the edge, cloud, content and service providers are growing points of presence (PoPs) globally and overall the number of internet edge routers has been on a constant rise. As the number of peering routers in a customer’s network grows, the number of paths a customer can exercise for internet traffic grows in a commensurate manner. Peering routers speak eBGP with external routers and participate in the customer’s iBGP mesh of internal routers. The growing number of edge routers means a growing number of BGP speaking nodes, sessions, and paths. Moreover, customers prefer complete visibility and control over these paths with the ability to influence and engineer traffic using their own “routing controller” logic instead of just using the protocol best path.
Leaf-Spine Clos architectures are standard in data centers due to their non-blocking, scale-out, any-to-any, resilient, connectivity characteristics. This architecture is now permeating the internet provider networks replacing the rigid, traditional router chassis with a more cost-effective, scale-out design where capacity can be added more seamlessly to meet growing demand. But this has resulted in the number of BGP (and IGP) nodes and sessions in the network multiplying and the number of paths increasing dramatically due to the ECMP nature of Clos.
Role of Route Reflectors in the Internet
Route reflectors (RRs) are an inherent part of most large-scale BGP network architectures. To avoid a full mesh of iBGP sessions, most BGP networks are deployed with one or more levels of RRs that
- ingest all the paths from client routers,
- typically execute a “best path selection”, and then
- advertise the best path northward and back to all the clients. This operation is often compute-intensive and is at the vortex of scale explosion as the number of RR clients, number of internet prefixes, and number of paths keeps growing, as noted above.
In addition to IPv4 and IPv6 prefixes, RRs often handle other BGP address families (AFs) such as VPNv4/VPNv6, BGP labeled unicast (LU); etc. RRs play a fundamental role in overall network convergence, especially in very large networks with increasing BGP speakers (iBGP and eBGP RR clients). When customers enable features such as ADD_PATHS to get visibility into all BGP paths, it becomes imperative to maintain the convergence characteristics at scale. Balancing scale-up of RRs with scale-out methods like sharding (across and/or within AFs) and leveraging multiple CPU cores is essential.
An RR today, is not limited to best path computation. Its routing state database built using standards-based BGP protocol can be harnessed in several ways, from enhancing applications to be more routing aware to building monitoring systems to enabling routing security, to name just a few. Furthermore, RRs are also the natural point to implement network routing policy to influence best path selection based on several types of matches (e.g., prefix lists, community, AS paths, etc.) and to control outgoing attributes in advertised routes. This centralized policy processing not only substantiates the need for compute-intensive operations, especially at large scale, but also underlines the importance of programmability in modern-day RRs.
ArcRR – a Modern-Day Route Reflector and More
In the past few years, several customers have been steadily moving away from dedicated, in-path, hardware route reflectors to out-of-path, software virtual RRs in order to leverage commodity, compute hardware with lots of cores at their disposal. This trend also facilitates the separation of control plane functionality from routers that are in the data path, keeps the core routers BGP-free, and enables flexible scale-out designs. So far, the only choices that customers have had are
- to utilize either heavy, legacy router software packaged as VMs (/containers) or
- to use open source implementations like Bird or Go BGP that have neither the appropriate scale nor the required performance metrics.
ArcRR, on the other hand, is a purpose-built, lean and composable, high-performance route reflector. It leverages the 64-bit, multi-threaded ArcOS implementation to drive best-in-class scale and convergence. It comes in flexible form factors and can be deployed over bare metal servers, as virtual machines (KVM, ESXi, Virtual Box, etc.), or as Docker containers.
ArcRR is built for networks with very large path scale that demand rapid convergence. It is built for the modern-day network operator who can leverage its rich streaming telemetry capabilities to build a BGP monitoring system. It is built for the provider of today who has standardized on a fully automated framework for all things networking using OpenConfig/YANG models. It is built for the security buffs who can leverage the integration of ArcRR with the Arrcus RPKI-based Route Origin Validation (ROV) solution to assess the validity of routes received and for detecting anomalies, thereby improving the security of their network. Finally, as routing controllers gain more prominence in edge routing and traffic engineering, ArcRR is built to integrate into these modern ecosystems.
Learn More
Check out the ArcRR At-A-Glance and/or visit us at https://www.arrcus.com to learn about how we can help address your evolving route reflector needs with ArcRR.