Workload automation (WLA) refers to IT tools that manage automated processes according to compute requirements. Users can schedule unattended jobs, execute jobs ad hoc, and allocate compute resources based on job requirements and the capacity of those resources. Because of this, WLA tools can be used to optimize both processes and the use of resources, preventing job delays and increasing IT efficiency.
WLA tools support both date/time scheduling and event-based scheduling, enabling users to pass data and dependencies downstream based on IT and business events. As opposed to traditional job scheduling solutions, which are often native, WLA tools are designed to support cross-platform processes regardless of operating system or underlying technology.
Workload automation products can offer a wide range of capabilities that extend across use cases, including:
- Secure file transfers
- IT automation/IT operations
- Business process automation
- Digital infrastructure
- Cloud computing
- Data center/ETL
- Big data/Hadoop
- And more
A Quick History Of Workload Automation Solutions
Workload automation software has its roots in batch processing. For much of the 20th century, batch jobs were processed overnight when compute resources weren’t occupied by daytime operations.
Batch processing became more complex over time, including with the introduction of multiprogramming. Meanwhile, IT teams became responsible for more jobs and more data, making overnight batch windows inadequate.
At the same time, there was a growing demand for jobs to be run in real-time, which traditional schedulers couldn’t support. As a result, IT vendors began to offer WLA platforms that supported stream processing, transactional processing, and event-based automation. Users could create jobs that would execute automatically when specified conditions were met.
There was also a need to better manage IT resources —as IT became busier, job delays became increasingly common. WLA tools were used to address this by scheduling jobs based on available CPU and memory. This helped reduce backlogs and bottlenecks, reducing delays.
As environments became more heterogeneous in the 2000s, WLA vendors began offering more integrations with their product offerings, allowing users to assemble cross-platform processes that minimized human intervention.
Modern Workload Automation Software Capabilities
WLA platforms have continued to evolve as the demands placed on IT have grown and shifted.
Today, most WLA solutions provide a variety of features and capabilities that enable IT to develop, deploy, monitor, and maintain cross-platform processes. This includes the use of:
- Alerts
- Variables
- Job constraints
- Auditing
- Change management
- Dependency mapping
- Resource provisioning
- And more
By 2010, WLA solutions had become commonplace. In 2014, Gartner estimated that most large enterprises were using at least three WLA tools. Many of these tools were in silos, deployed within departments where they were used for specific environments. As a result, WLA was understood to be a subset of an organization’s larger automation strategy, deployed alongside IT process automation tools and business process automation tools, for example.
WLA was a mature market and innovation was thought to have peaked. In 2014, Gartner released its final Magic Quadrant for Workload Automation, and removed WLA from its Hype Cycle charts (WLA had progressed beyond Gartner’s Plateau of Productivity).
But that’s just the beginning of a new story.
Gartner Brings Back Workload Automation, Sort Of
The WLA market has continued to evolve over the last several years. Vendors have consolidated (CA acquired Automic and then Broadcom acquired CA, for example) and have taken their product offerings in different directions. Several key vendors in the WLA market have continued to build out the capabilities of their platforms in order to better support process orchestration.
This led Gartner to introduce its first Market Guide for Service Orchestration and Automation Platforms in 2020.
What are service orchestration and automation platforms (SOAPs)?
“SOAPs do not represent a new market, rather a transformation of a mature market for workload automation tools to meet modern infrastructure, application and data requirements. SOAPs represent the evolution of workload automation tools that aim to manage and automate a complete business process. These tools embrace the shift from time-based scheduling to event-based workflow orchestration, thus making them better suited for real-time data processing needs.”
-Gartner, Market Guide for Service Orchestration and Automation Platforms
WLA platforms had evolved right along with IT.
Don’t Settle For Unreliable Automation
Improve reliability and extensibility across your environment with the right workload automation strategies and tools.
Trends Impacting Workload Automation (and SOAPs)
Market-leading WLA platforms continue to evolve to keep pace with IT’s needs. Vendors have yet to adopt Gartner’s SOAP phrasing, and often market their products as automation and orchestration platforms. Again, this includes a segment of the WLA market, as many vendors have not kept pace with business and IT trends.
There are four main trends that market-leading WLA platforms are designed to help IT address:
- Proliferation of disparate tools and distributed environments
- Rising importance of multi-cloud and hybrid-cloud environments
- The shift towards DevOps and agile methodologies
- IT’s increased role in providing services to external customers
Proliferation Of Disparate Tools
IT environments are becoming more complex as organizations implement a wide range of enterprise applications and technologies to meet business needs. MuleSoft’s 2020 Connectivity Report found that 85% of organizations consider integration challenges to be a major roadblock to transformation, underscoring the difficulties IT teams can have in managing environments with disparate systems and applications.
WLA platforms can provide a variety of ways to integrate tools and systems, including through direct integrations and universal connectors. WLA platforms can also enable users to load and execute APIs (including WSDLs, SOAP services, .Net assemblies, stored procedures, and command lines).
Today, WLA platforms can also provide REST API adapters which can be used to quickly and reliably incorporate virtually any technology into end-to-end processes. These adapters are increasingly simple to use as new features are added, such as reusable templates, wizards, and testing.
As organizations become more reliant on cloud-based infrastructure and applications, the use of APIs to integrate and orchestrate processes will increase.
Multi-Cloud And Hybrid Cloud Environments
Forrester expects half of enterprises to make cloud-native transformation a top priority in 2021. Many of these organizations will rely on multiple cloud vendors in order to leverage a wider range of tools and services. At the same time, many organizations will continue to maintain legacy mainframes and other on-premises systems, including private cloud.
As a result, organizations are moving towards orchestration and integration hubs that can seamlessly manage data across multi-cloud and hybrid environments. WLA solutions often provide direct integrations with Microsoft Azure, Amazon AWS, and VMware, among other providers. WLA can also be deployed in cloud environments, on-premises, or in hybrid configurations, simplifying management and compliance requirements for cross-platform processes.
The large number of VMs and cloud-based servers is also of growing concern for IT teams —there can be hundreds of virtual or cloud-based resources in a given environment, making manual management virtually impossible.
To help address this, WLA platforms can be directed to dynamically manage workloads and compute resources. Using machine learning algorithms, a WLA platform can schedule jobs and allocate resources based on historical trends and real-time needs. This includes rerouting jobs, cordoning resources, and provisioning/deprovisioning resources in real time for dynamic, needs-based scalability in multi-cloud environments.
This makes it possible for IT environments to spin machines up and down based on demand so that organizations aren’t paying for idle machine resources.
DevOps And Agile Methodologies
There’s a growing trend to spend more time in development and less time maintaining processes and assets. Part of this is because businesses need to quickly adapt to shifting market demands, especially in light of the COVID-19 crisis. Digital services and digitally augmented operations are critical to keeping many organizations afloat.
To meet these challenges, WLA platforms are providing tools that accelerate and streamline development and operations. This includes drag-and-drop workflow designers that abstract away complex code, visual maps that lay-out dependencies, and features that help drive collaboration, such as Check-Out | Check-In, change management, and testing functionality.
WLA platforms can also support PowerShell, command lines, and, as we covered, APIs. Scripts can be maintained within the platform for easy integration into cross-platform processes, while documentation and auditing provide security and prevent knowledge loss.
Meanwhile, these same WLA platforms are helping IT teams automate operations using real-time monitoring, alerting, auto-remediation, and even pre-emptive error detection. The goal for WLA vendors is to reduce the time IT teams spend maintaining, in order to increase the time IT teams can spend developing and iterating services.
Expanding Role Of IT
IT’s role within the organization has been expanding, and continues to expand at an accelerated pace. Gartner analysts expect that, by the end of 2022, 70% of customer experience (CX) programs will rely directly on IT. As a result, CIOs and IT leaders are being given greater responsibility in overseeing CX programs and in managing day-to-day operations.
In order to help organizations pivot and adapt, WLA vendors are building-out capabilities that enable IT to rapidly integrate news tools and technologies; to seamlessly manage data across silos and environments; and to accelerate development and innovation.
Today’s market-leading WLA platforms are advanced, intelligent automation and orchestration solutions that have evolved far beyond traditional scheduling and automation tools. Today, many WLA vendors support mobile and web applications that enable IT users to manage their environments from a range of devices and browsers, regardless of whether they’re in the office. Self-service portals are also available, enabling business users to trigger and monitor ad hoc processes without having to open IT tickets, saving IT time while improving SLAs.
WLA vendors are pushing the limits of extensible automation, going far beyond traditional job schedulers to drive digital transformation and set up IT for long-term success.
Ready To See How We Make Workload Automation Easy?
Schedule a demo to watch our experts run jobs that match your use cases in ActiveBatch. Get your questions answered and learn how easy it is to build and maintain your jobs in ActiveBatch.
