Skip to content 
This is the second blog post in the “Massive Scale” series. In the previous blog post, we touched upon the importance of a massive networking scale and how each ATOM functionality has been designed with scale in mind. This blog will examine the scale of Anuta ATOM workflow automation.
New networking technologies have fed a continuous stream of innovative, bandwidth-hungry applications. This has compelled network architects to design and enforce laborious methods-of-procedures (MOPs) to execute complex operations. Some of these tasks are stateless or one-shot operations such as software upgrades, while others are stateful and require service lifecycle management such as L2VPNs. ATOM’s workflow automation and service orchestration work together and help operators to design and execute their flows and services on a large scale. In this blog post, we will discover the Anuta ATOM’s workflow automation scale capabilities.
Workflow Automation at Scale
ATOM’s workflow automation capabilities enable network administrators to automate manual processes and methods of procedures. These tasks often include pre-checks, post-checks, approval chains, and integrations to OSS/BSS, ticketing/billing, and ITSM solutions. Large networks typically have complex workflows consisting of hundreds of commands, integration to tens of network elements, and complex business logic to bind the various elements together. ATOM workflow automation has been designed not only to facilitate simple use cases, but also design and execute the most complicated use cases at scale.
ATOM workflow automation has been deployed by various customers to automate their MOPs and execute them at scale. The following two case studies provide a glimpse into ATOM’s massive scalability.
Software Upgrade of 20,000 BNG Routers for Large Service Provider Networks
A large service provider based in Europe approached Anuta Networks with the task of improving the stability of their network. This particular service provider performs critical software upgrades of 2,000 Juniper MX BNG devices within a designated change window. Their current process consists mostly of manual operations and scripts. On its best day, the organization was able to upgrade the software of 50 devices in 1 hour by employing four engineers.
Recently, the same service provider executed a complex In-Service-Software-Upgrade (ISSU) method-of-procedure (MOP) consisting of 49 prechecks, 49 post-checks, and 49 rollback procedures.
Their MOP included the following sequence of procedures:
- Disk space check to validate sufficient space exists to perform the upgrade.
- Configuration backup.
- 49 pre-checks to determine correct conditions for an upgrade.
- A request for approval from an administrator before the upgrade.
- Generation of a transient configuration.
- Performance of a vendor-approved ISSU Upgrade:
- Perform JUNOS Upgrade
- Perform JSU Upgrade
- On any error, perform a rollback consisting of 49 commands
- Perform post-validation.
- On Error, perform a rollback consisting of 49 commands.
- Perform an ICCEA upgrade.
- Rollback transient configuration.
- Perform a JSU upgrade.
- Execute post-checks of 49 commands and on error rollback.
- Clean up upgrade scripts
Every engineer must synchronously operate numerous command-line interfaces, track errors and alerts, perform rollbacks, and restart failed flows in the current process. Not only was the process cumbersome, but it also introduced numerous manual errors, leading to substantial time and resource 
waste.
The service provider automated the entire end-to-end method-of-procedure using Anuta Networks’ ATOM workflow automation capability. ATOM’s workflow automation runs a BPMN engine that can process thousands of workflows simultaneously. In the service provider environment, ATOM has been tested to perform an astounding 2962 workflows/hour. Each workflow performs all steps in the MOP listed above.
As a result, ATOM has been validated to upgrade 20,000 Juniper MX devices (10x more than in customer network) in a simulated environment in approximately 6 hours and 45 minutes. Executing the most complex of workflows in a short period has been the strength of the ATOM platform.
IPV4 to IPV6 Network Migration for ~10,000 devices at Tier-1 Media Entertainment Company
One of the world’s largest media, entertainment, and amusement park companies approached Anuta Networks to help them accelerate their IPv6 transformation. The conglomerate was preparing to launch its own Content Delivery Network (CDN) and transition to IPv6 to ensure future scalability. Their network infrastructure includes switches, routers, and firewalls from multiple vendors – Cisco Systems, Juniper Networks, Palo Alto, and F5. encompassing eight different platforms. The customer had a total of ~10,000 devices that spanned various regions across the United States further complicating matters..
Migrating, a multi-vendor and multi-region network from IPV4 to IPV6, is complex and involves the following procedure.
Phase A: Configure IPV6 interfaces.
- Execute and Validate Pre-Checks for all eight device platforms. Pre-Checks for each platform vary and range from 15 to 20 commands, including interface checks, BGP/isis checks, and VRF and other routing checks: (Specific to each vendor platform)
- On any error, automatically log the issue to ServiceNow and rollback all operations.
- Connect to ATOM IPAM and retrieve free IP addresses.
- Generate interface specific commands; (Specific to each vendor platform)
- Get approval from the network administrator.
- Upon the network administrator’s approval, provision interface commands on the device; (Specific to each vendor platform)
- Run Post Checks for the device. Post-Checks for each platform vary and range from 15 to 20 commands, including interface checks, BGP/isis checks, and VRF and other routing checks: (Specific to each vendor platform)
- Upon discovering any error in provisioning, automatically log the issue to ServiceNow and rollback all operations.
Phase B: Configure IPV6 routes
- Execute and validate pre-checks for all eight device platforms. Pre-checks vary and range from 15 to 20 commands, including interface checks, BGP/isis checks, and VRF and other routing checks: (Specific to each vendor platform)
- Upon discovery of any error, automatically log the issue to ServiceNow and rollback all operations.
- Retrieve and verify interface IP addresses and BGP configurations (Specific to each vendor platform);
- Generate eBGP and iBGP configuration; (Specific to each vendor platform)
- Get approval from the network administrator.
- Upon the network administrator’s approval, provision interface commands on the device.
- Run post checks for the device. Post-checks for each vary and range from 15 to 20 commands, including interface checks, BGP/isis checks, and VRF and other routing checks: (Specific to each vendor platform)
- Upon discovery of any error in provisioning, automatically log the issue to ServiceNow and rollback all operations.
The customer has to perform Phase A on all multi-vendor and multi-regional devices before executing Phase B. By automating this complicated procedure with ATOM, the media entertainment company increased its productivity by 35 times. The customer could migrate only eight devices per week without automation. After automating the entire MOP using ATOM workflow automation functionality, the enterprise can now migrate 280 devices per week.
ATOM also has a comprehensive workflow logging and monitoring functionality that helps operators pinpoint and debug any workflow. All workflows and sub-workflows are listed in a hierarchical format for easy viewing and traceability. The ATOM platform consists of a horizontally scalable architecture (stay tuned for more specifics about the ATOM architecture in a future blog post).
We hope you found this blog informative. Our next blog will focus on ATOM’s scale for service orchestration and lifecycle management, so come back soon!
