"We have always been at war with bureaucracy"​ - Agile, Business Process Reengineering, and Automation

26 May 2019

“The past was alterable. The past never had been altered. Oceania was at war with Eastasia. Oceania had always been at war with Eastasia.”

There is a war being fought daily in our organizations. Thousands of men and women are engaged in a struggle with the evil empire - bureaucracy. Their weapon of choice - agile mindsets and techniques. Apparently.

Lets define bureaucracy as the use of tightly and centrally defined rules, procedures and roles to manage work. Lets define agile as an approach to work that focuses on small, autonomous, cross-functional teams delivering defined outputs to customers in time-boxed, short-duration, iterative cycles. A number of writers (Steve Denning, Darrell Rigby) have explicitly positioned the agile movement against bureaucracy. But some of these arguments feel familiar.

"The usual methods for boosting performance—process rationalization and automation—haven’t yielded the dramatic improvements companies need. In particular, heavy investments in information technology have delivered disappointing results—largely because companies tend to use technology to mechanize old ways of doing business. They leave the existing processes intact and use computers simply to speed them up.

But speeding up those processes cannot address their fundamental performance deficiencies. Many of our job designs, work flows, control mechanisms, and organizational structures came of age in a different competitive environment and before the advent of the computer. They are geared toward efficiency and control. Yet the watchwords of the new decade are innovation and speed, service and quality."

The text above sounds like it could be from a promoter of agile techniques. In fact it is from an article written nearly 30 years ago by Michael Hammer. This article was one of the first in the Business Process Reengineering (BPR) movement. In the early 90s, this movement called for companies to radically redesign their processes from scratch supported by the latest information technology to meet the needs of their customers. In practice this led to two outcomes. Firstly, BPR was a rationale for mass redundancies during the early 90s recession. Secondly, BPR was a rationale for massive ERP (SAP, Oracle, PeopleSoft, JD Edwards, Baan) implementations that sought to implement standardized best practices and processes across large organizations. Many of these implementations did not yield the promised benefits. By 1995, the three biggest names in BPR (Michael Hammer, Thomas Davenport, James Champy) had all disowned their baby.

Agile and BPR are obviously not the same thing. Nevertheless, they share some similarities. And I want to explore what those differences and similarities and what those mean for the future of agile, the persistence of bureaucracy, and the cycle of organizational change. I want to briefly ponder the relationship between automation and agile and what that means for nature of organizations.

So lets start with the obvious differences:

• In terms of where they place authority, agile focuses on team autonomy. The agile team works with their customer to identify what needs to be done next. BPR initially talked about empowering managers but also had a strong centralizing focus. BPR largely ended up putting process redesign in the hands of external consultants.

• In terms of how they envision work, BPR wanted standardized, repeatable best practices whereas agile seeks iterative increments. BPR would view agile as making too many mistakes.

• In terms of the technological paradigms that underpin them, agile is most at home with microservices and platforms (in fact, it doesn't work well without them) whereas BPR prefers holistic, end-to-end systems. In short, API vs ERP.

• These differences lead to the biggest challenge that both movements face - the problem of coordination - that they attempt to solve in opposite ways. BPR solves by centralization, leaving it vulnerable to being overwhelmed by problems at the edge. Agile solves by attempting to network its teams together which leaves it vulnerable to the quality of that network.

But on closer inspection, the two movements have a lot in common.

• Both use a revolutionary language of change. While agile talks about incremental increments for work products, its promoters talk about a need for root-and-branch transformation for the organization. The whole business must become agile, not isolated pockets like IT.

• Both have a heavy focus on training, retraining and ideology. The industry of scrum certifiers almost matches the six sigma machine. The Black Belt in every office has been replaced by the Scrum Master.

• Both have common origins in movements like quality management and lean. The kanban boards beloved of agile teams have their origins in the Toyota Production System that became lean. BPR techniques drew on both lean and six sigma for their process redesign and quality monitoring.

• Both have been picked up by the large consulting firms to yield many billable hours of transformation projects.

• Both have genuinely useful elements and both solve real problems (at least partially).

• Both claim to hate bureaucracy but may in fact promulgate it.

That last point deserves some attention. BPR targeted bureaucracy but its tendancies towards centralization and standardization made it easy to form new bureaucratic structures. Meanwhile agile risks falling into bureaucracy through ossified rituals. The continual revolution that some forms of agile demand are exhausting. That product backlog is one bad day from becoming a bottleneck.

I cannot prove this but I suspect that bureaucracy cannot be eradicated from large organizations. That is not say it cannot be minimized or contained. Rather that the parts of organizations that are risk-averse or require standardization will not only tend towards it but that it may be the most stable state for them.

Perhaps there is one thing that may not so much eradicate bureaucracy as remove it from human experience: Automation. The full impact of Robotic Process Automation (RPA) and Cognitive Automation (CA) have yet to be felt in the workforce. There are many ways this could play out. It could be implemented as a key part of the agile team architecture. In fact, if you look at software development, you need large dollops of automation (esp. in testing and devops) to make it work. Automation is used to simultaneously shorten the iterative cycle times and reduce the coordination issues of agile teams. That is just one possibility. Another is that automation is used to exert centralized control across the organization. A robot bureaucracy splicing together red tape and magnetic tape in a moebius loop of efficiency.

Perhaps even an agile bureaucracy.