Border Gateway Protocol

Border Gateway Protocol (BGP) is today’s Internet routing protocol.

It is a path vector protocol, which selects the route based on some static metrics (ie. as path length, local preference).

However none of these metrics reflect the actual performance, availability or cost of the path. Read more

Non-Stop Internet

Non-Stop Internet (NSI) technology has been created by BORDER 6 engineering team to improve and automate the BGP management for multi-homed networks. Since the design face, up till now we put a lot of emphasis to develop rock solid software for critical part of the infrastructure, which is the WAN edge.

The NSI executes thousands of performance and availability tests per minute. Measurements are done automatically for most important distant subnets though plug&play probing mechanism. The default probe type is the TCP syn stealth test. The  locally installed probe sends constantly TCP SYN packets to destination IP addresses and waits for the TCP SYN ACK packet. All of it is done via all of the configured Transits or Internet Exchange points (IX’es). Probing helps to detect the broken, packet loss paths and automatically reroute the traffic to the stable upstream. Further more, the time gap between both packets (latency) gives overview of the link performance, which allows NSI to determine best route per destination and confront it with the BGP decision in real time. If the optimization engine is enabled then the traffic is moved to the fastest and more reliable link.

The platform gives detailed insight and visibility of the traffic flows. Who are the top talkers (per ASN or Prefix)? Which protocols are used at most within the network? How the traffic is balanced between all of the up links? Which is the best transit for given time period? All of this information can be extracted through multiple reports. This part of our software plays important role in improving peering policy or gives a valuable hint when changing the transit or its capacity. Instead of blind trusting the provider the NSI provides full and transparent view over the BGP-WAN edge.

Automated approach for the software defined WAN management, can also safe significant amount of your budget. Control of the transit commits is the key feature here. Cost driven optimization selects not only the best performing path, but also takes to account it’s cost. If the performance measurements return  acceptable values, then the core engine will select the cheapest route. Other cost reduction can be achieved by the dynamic FIB table size control. In that case the NSI platform acts as a BGP controller, which injects only number of routes, that will cover ~98% of the overall traffic (the rest is routed then via default gateways). Using only these top prefixes and dynamically updating them by the controller, you can use switches with L3 BGP functionality instead of expensive, dedicated routers.

Border Gateway Protocol

Border Gateway Protocol (BGP) is the protocol of the Internet today. It is a path vector protocol, which selects the route based on some static metrics (ie. as path length, local preference). The problem is that none of these metrics reflect the actual performance or cost of the path. Whether a route is short or long distance, BGP is not able to make any difference. Whether a path is properly forwarding packets or dropping some of them, again this protocol fails in detecting such quality problems. This paradigm results into the inability to detect failures and poor performance:

  • Long paths Large delays cause frustration to web users. On business-to-consumer applications this directly affects the actual sales rates. It severely impacts the perceived quality of voice communication and leads users to shorten voice calls. Finally it results in very noticeable performance problems with gaming applications.
  • Packet loss Low packet loss rates usually produce same perception as high delays. However packet loss turns into a bigger issue with video applications such as streaming and videoconferencing.
  • Broken links Routing equipments sometimes undergo inconsistencies of the routing and control planes. In this situation, the BGP mechanism sees the network as properly functioning while the interfaces and routing process are actually not forwarding packets. In this case, the users cannot access their applications anymore until a manual action is executed on the equipments to restore the function of the network. Similar situations are also noticed when operators implement wrong access-lists or flood protection mechanisms actually block legitimate user traffic.

Detecting failures and measuring Internet performance require application traffic to flow back and forth to remote peers and from remote users. Our software executes thousands of tests per minutes. It creates a full description of the customer network connectivity and notices all changes almost instantaneously. Once properly configured via Graphical User Interface, these automated probings do not require any manual intervention. Below you will find some detailed information regarding our technology and it’s features.

General Architecture

The NSI is made of two modules: NSI Device and NSI Server. The NSI Device (aka Probe) is responsible for executing all of the performance measurements, retrieve and aggregate the flow information from edge routers, poll the SNMP information. The information is reported then to the NSI Server. Multiple NSI Devices can be deployed, i.e. per data-center location, if need. Our support team will provide you the best design.

The NSI Server is the central point of global WAN management. It holds the databases, provides the graphical interface and executes actions to improve the global routing policy.

The system intelligence is the Routing Decision Engine (RDE). It is responsible for the automated cost and performance optimization. Equipped with additional tools like: Virtual Router (VR), ultra fast Border 6 Flow Collector, WAN Monitoring (WAN-MON) and additional tools.

The NSI platform supports installation in virtual environment (KVM, VMware), on custom hardware platform or optionally is delivered as the BORDER 6 appliance. The operating system is based on Linux kernel, but all of the modules, like flow collector, BGP daemon, where developed by BORDER 6 R&D team to provide robust performance and stability.

Virtual Router (VR) – FIB optimization

How big is the Internet today? Well, the full “map” consist of around 530k prefixes. The edge router will load this number of networks to the FIB table. But do you exchange the data with all of the prefixes? What number of prefixes would cover most of your traffic? This can be extracted via our reporting module, the Flow Collector. But NSI can do more with its Virtual Router functionality (VR).

The scenario is very simple. The BGP edge devices would announce the given prefixes plus it will accept only default route from the up-streams. NSI platform, software defined BGP controller in that case, will get the full BGP feed from each transit. Thanks to the RDE analysis, the controller will inject to the edge gear the full set of Managed Subnets, so the TOP destination prefixes that exchange most of the traffic. This feature allows you to safe budged when planning new router purchase. You can change the expensive router option with smart switches that support layer 3 BGP functionality (i.e. Cumulus Networks).

Image

Routing Decision Engine (RDE) – Performance and Cost optimization

The RDE it is the control plane for the agile BGP multi-homed setup that NSI platform provides. The way it works can be divide into three steps:

1/ Managed Subnets selection,

2/ Constant performance analysis,

3/ Optimization and monitoring.

The First step is about getting the statistical information from the traffic flows and then selecting set of prefixes which exchange most of the traffic with the multi-homed network. This set of destinations is called “Managed Subnets”. For each of the Managed Subnet prefixes, the engine will assign one or two probing IP addresses. This IP addresses would be used as the target destinations for the measurements. The Managed Networks set is build automatically, but it can be adjust depending on the requirement. For example the administrator might want to select manually prefixes which should be included within this set, no matter what value of the traffic they are exchanging.

In the second step NSI Server passes the networks with their probing information to NSI Device. This will start the process of executing the measurements by the probe. By default NSI uses the SYN Stealth for probing. It sends TCP SYN packet to the given destination on specific port and waits for TCP SYN ACK packet. The time gap between both packets is the measured latency.

Another metric it provides is packet loss ratio. By default each probe is executed every 240 seconds with 10% tolerance, via all of the configured transits and internet exchange points (IX’es). This setting can be adjusted per prefix or globally for all tests. The RDE probing engine also provides two other probe types: the UDP probe and the ICMP probe. All probing implementation has plug&play configuration approach, but can by changed later to provide maximum flexibility.

prefix

In the final step the RDE is comparing the BGP choice, path selected by the BGP protocol (CURRENT transit) with the BEST transit (the optimal performance). If BEST transit is different vs. the one selected by the BGP protocol then the system will generate routing order. The routing order selects a next-hop for the destination pointing to the fastest and/or most reliable upstream. Depending which RDE mode is enabled, the orders are executed automatically (automatic optimization) or wait for the administrator validation (semi-automatic mode).

rde

The best path selection algorithm also takes to account the cost of the configured transit. So just after selecting the BEST transit it will look at the statistics and check if the traffic sent over the transit will go over the committed bandwidth (CDR). This allows to control the transit burst in cost effective way.

BORDER 6 Flow Collector

The automated routing optimization done by RDE is great, but it is also important to have detailed insight information on the network traffic nature. NSI flow collector was developed by BORDER R&D team to meet the specific performance and scalability requirements. It supports NetFlow, sFlow, jFlow and IPFIX protocols. The collector stores data in PostgreSQL database. The collected information is presented through different variants of reports which help to improve the peering policy, adjust the inter connectivity. That view can be strong argument for future transit selection. Some of the included reports are:

1/ TOP talkers (ASN/Prefix);

2/ TOP protocols;

3/ Traffic load balancing stats (global, per transit or IX, per geographical location);

4/ Transit usage statistics, including 95th percentile calculations;

5/ IX traffic statistics per peer;

6/ Monthly billing information;

7/ Real-Time traffic analysis;

8/ Geo-location probes presentation, traffic distribution view (per ASN or Prefix);

On the top of the flow information the collector stores also historical information regarding probing measurements. The administrator can check the overall performance overview of the transit (general, per prefix, ASN) over custom time period, monthly or weekly. It is valuable tool for monitoring the SLA’s with operators. All of this statistics can be exported as pdf or csv file. Each of the reports can be also transformed as widget for real-time monitoring purpose.

WAN Monitoring (WAN-MON)

Monitoring specific prefixes might be very important for some of the setups. Specially when the include VPN infrastructure. Having and detailed information about path changes for specific prefixes is crucial. This helps applies pro-active actions when something starts to go wrong with your transit provider. The WAN-MON module will monitor and archive all the BGP and performance information per prefix and notify administrator is some specific scenario occurs. It might be the as-path change, the as path length reaches critical value or the RTD, packet loss for specific prefix goes over the configured the threshold.

Additional tools

NSI platform includes of large set of notification that are triggered on some specific situation. It can be BGP session state change, DDoS detection or for example transit Blackout. Custom notification could be configured depending on the need. Per each of the notification so called smart action can be added. It can be custom script execution, BGP order, mail or sms announcement. All of the software defined action can automated the whole engineering at the edge and lower the operational cost.

When it comes to troubleshooting NSI can help with rich set of tools. Visualization of the multi-transit traceroute, MTU discovery, manual probing execution are just few examples.

Licensing model.

License type is based on the outbound traffic flow (95th percentile), aggregated for all the transits. It starts with the license for traffic of 200 Mbps and ends with the unlimited license.

BORDER 6 license comes with two flavors: FULL and LIGHT. Light version does not include the automated route injection for the optimization purpose.

The license includes 24/7 support and feature updates during the license period.