The End of Process

If a knowledge worker has the organization's information in a social context at their finger tips, and the organization is sufficiently connected to tap experts and form groups instantly to resolve exceptions -- is there a role for business process as we know it?

My favorite Clay Shirky quote is "process is an embedded reaction to prior stupidity." That is, there was an exception to process and an expert designed a way for people to work together in one context that should fit all prior contexts. The problem is, the process becomes calcified and accepted as the rule. After all, it's a rule, and in corporations we follow them, even if it fails us or simply doesn't make sense. Because of constant change in our environment, processes are outdated the immediately after they are designed. The 90s business process re-engineering model intended to introduce change, but was driven by experts which simply delivered another set of frozen processes. Because participants in process are not considered experts in theory, they are empowered to make decisions on their own when something fails.

Organizations are trapped in a spiral of declining innovation led by the false promise of efficiency. Workers are given firm guidelines and are trained to only draw within them. Managers have the false belief engineered process and hoarding information is a substitute for good leadership. Processes fail and silos persist despite dysfunctional matrices. Executives are so far removed from exceptions and objections that all they get are carefully packaged reports of good news and numbers that reveal the bad when it's too late.

John Seely Brown and John Hagel point out that while 95% of IT investment goes to support business process (to drive down costs), most employee time isn't spent on process -- but exceptions to process. Further, competitive advantage comes from how we innovate in handling exceptions. When something fails, informed and empowered employees turn to their social network. The informal network, or heterarchy, where most business gets done.

Today, some staid corporations are abandoning process all together (I wish I could quote the source for this). Google is a more public example, albeit an exceptionally new large enterprise, where wikis and weblogs enable a culture of working openly in a flatter and decentralized organization. This is data point helps plot the trend of decentralized organizations that realize economies of scale, as described by Thomas Malone in his book, Decentralization.

Assume for a moment that the 25% of GDP that is search costs falls. Or the 50% of GDP that is transaction costs similarly declines. Coordination costs fall with rising connectivity. The cost of personal publishing and easy group forming are rapidly falling to zero. If a knowledge worker has relevant information at their finger tips, can form the right group to handle an exception, leverage the social context of information and contribute to memory as a natural by-product of getting work done -- what is the role of process?

A process is like a standard. It provides a common definition for others to build upon. This is generally a good thing. In technical systems it helps resolve complexity so higher order abstractions can keep things simple. But even in technical systems, efficiency comes at a cost of adaptation. In social systems, especially where not everyone helps design what they participate in, the constraints against adaptation are compounded.

At best, a process should serve as a reference model. Something that others can reference when completing a task. Something that can be leveraged for innovation, a boundary condition for experimentation at the margin.

As with many things, gaining greater participation and innovation requires sharing control. I do not believe we are near the End of Process, yet. I do believe the arguments for engineering organizations are being trumped by new practices and simple tools.  The first organizations bringing it to an end will have a decided competitive advantage.

UPDATE: Comments are starting to get interesting.  Euan Semple nails it: Process is the sort of word that grown ups in suits use to throw their weight around and to convince others that they know what is going on and that it makes sense.

Comments

[Jake Kaldenbaugh]

Is this an oversimplified assault on process? You argue that business value can be driven from how professionals are allowed to handle exceptions. Completely agree, but business value can also be driven by handling non-exceptions in an efficient manner (whether through human or IT resources). The structure of process doesn't necessarily need to be confining -- it can be guiding. I don't think that most organizations use process as confinement, but maybe I've just been lucky to work with progressive organizations. I would argue that good process includes freedom to innovate and allows for exception handling. I would also argue that there are cases where restrictive process could be a good thing. For instance in situations where the consequences for deviance are known and disastrous (say, running a nuclear power plant). I'd like to know that their processes are relatively restrictive and well-defined!
Just as I think Shirky was premature in destroying ontologies, I think you might be premature in destroying process. It's useful when done correctly and in the right situations.

[Jeffrey McManus]

You really think that Google doesn't have process, huh? Interesting.

[Ross Mayfield]

Let me respectfully disagree to emphasize my point.

It's easy to say there are cases where restrictive process are called for. But never presume static conditions or closed systems.

Stewart Brand pointed out that with 911 the only airplane that didn't get hijacked wasn't one following process and authority, but the one where people took authority into their own hands. They were the ones with cell phones.

Since cockpits were given the same information as control towers, we haven't had a catastrophic crash.

[Ross Mayfield]

Jeff, I'm sure Google has process, just a great deal less of it. I had a source tell me of other organizations with a strategic imperative that are abandoning process all together. I'll try to get permission to share about it, but it will take time (hint: it involves a pr process).

[Jon Husband]

Bravo .. which I think means i agree and that you have said ina comprehensive fashion what I ahve often tried to say.

The way I usually say it is that ERP systems, implemented after and during business process reengineering, are akin to pouring "electronic concrete" over a newly-designed process that almost certainly is fated to begin changing relatively quickly as customers and markets' needs keep on changing .. then, all too often, customers become the prisoners of a given business's process(es).

And the investments in these systems .. financial and in knowledge and work design resources and in the psychology of the executives and management teams that "own" said processes .. are usually too large to allow for adaptation and rolling re-design.

And yes, I do think that simple fundamental work design principles, brough to life by clear AND generic practices and simple tools, hold out a great deal of promise ... not to mention greatly reduced budgets and hence significant cost savings. But they also threaten the roles of CIO's and a significant range of managerial positions, at least in their current form.

[Rutger]

Although I understand your point, isn't your call for the end of process actually a call for more flexible processes? Or are processes inherently inflexible?

I was just wondering what the alternatives would be? Should we borrow some terminology from IT and adopt a human form of Service Oriented Architecture (SOA)? The vision of a dense network of service-oriented knowledge professionals organizing themselves in ad-hoc teams sounds quite nice actually. But eh, isn't that the description of every PR and creative department (or should it be)? How applicable is your idea for the majority of companies/departments?

[Dennis Howlett]

Ross - process isn't an either/or, good/bad thing (which is how I'm reading your argument).

I believe you're right when you talk about exceptions but in all the process driven projects I've seen it is precisely about automating processes that don't require manual intervention or which can be readiuly automated. These projects have always been about paving the way for exception management. That's nothing new. Neither is the fact that individuals will turn to their peers in unstructured ways to solve specific issues. It's how, as an accountant I became involved in IT issues some 20+ years ago.

To imply Google is largely process free is fundamentally wrong. They tell us time and again they use specific techniques (and processes) to drive things like product development. The method may appear unstructured and lacking in process but it isn't. They apply plenty of process rigor to project selection. There's a BIG difference.

[Al]

Great post, you made me think again..

bifurcation
____---
----____---
---

Process is a birfurcation
It gets more complicated the more descisions/junctions exist in the process

Enterprises have traditional applied harsh pruning to solve this issue (Lop off exceptions) when modelling in I.T. This results in a tree with very few brances and no leaves.

If however genetic or evolotuionary software could be applied to the processes it could eventually resolve the various complex sub processes (discover the bifurcation map/dimensions). Although this could take a long time and might be unpredictable in the normal business timescales.

If social software combined with this sort approach is applied one might actually get results in acceptable business time scales. The social dimension adds expertese and guidance and corrections, preventing obvious cul de sacs.

I guess similarly a wiki could be seen as a socially created bifurcation, if one could add the process programming/genetic logic to the wiki one might also begin to approach above solution.

I guess making it simple enough for anyone to use could be the hard bit though

[george]

So when one of your sites is hit by a tornado you want to have a disaster recovery process?

[Ethan]

"Stewart Brand pointed out that with 911 the only airplane that didn't get hijacked wasn't one following process and authority, but the one where people took authority into their own hands. They were the ones with cell phones."

Semantically, this has nothing to do with Process. It has to do with Policy, namely the policy of normally not allowing people to use their cell phones while a plane is in flight. The process for using cell phones is in fact static, however the policy to restrict their use is at issue in the above excerpt.

Reading through your post, a great deal of Policy runs through it, but the disagreement is with Process. As with any process system, you are indeed "defining the box", as is also the case when Process is completely forsaken.

[Euan]

Process is the sort of word that grown ups in suits use to throw their weight around and to convince others that they know what is going on and that it makes sense.

[Jon Husband]

yeah .. there do come to be good ways to do things, and get things done .. and people often know how to find or create these when they have a clear purpose, objective, and some simple principles and tools.

And when the context and/ or customers change, the good ways to do things often change, or need to change.

I think that if we want to consider using process as a backbone, it should be social process (exploration, interaction, cionstruction, testing, refinements, etc.) first, interacting with principles and the simplest technologies possible (big generalisation, I know).

This is not the kind of *process* suits prefer, and all too often don't understand well or don't want to understand.

[Bill Hart-Davidson]

This reminds me of Lucy Suchman's argument about plans (representations of process). She says that plans are insufficient to prescribe action in the future and to describe action in the past. They are, however, useful as resources in the present.

Process representations are a useful language game for folks to play who must work together. Helps them coordinate, test ideas, and anticipate outcomes. But when they become formalized and turned into algorithms, they hinder more than they help.

[Nelson Nones]

I am renovating a house in Thailand, where I live. Before signing the builder's contract, I requested a project schedule aligned with the progress payment dates. 13 weeks on, and about two-thirds complete, the project is currently 3 weeks ahead of schedule. This is utterly amazing when you consider that most construction projects here (like Bangkok's new airport, or the "highway improvements" near my new house) languish in perfect disarray many months after they were originally advertised to be completed.

One day I met the builder at the house and complimented him on his firm's professionalism. I told him that in all my years (too many, I'm afraid), I'd never seen a residential renovation project run so well.

He looked at me and said simply, "I have a process."

I'll leave it at that.

[Dan Kemp]

A very interesting and important discussion. However, relative to 9/11, I think the point is that we should have well-defined processes followed at every airport that prevent weapons from being carried onto airplanes. The last thing we want is a creative group of TSA screeners, with each innovating about who they will search and what weapons they will allow to be carried on board.

[Vinnie Mirchandani]

Ross, I have to agree with you. Process is like spell checking software. You can tell when it is missing, but no customer wants to pay much for it. Unfortunately, we have forgotten the customer POV as we lather our processes with technology, expensive labor, security and compliance ...you may enjoy my post titled Business Process Angioplasty below

http://dealarchitect.typepad.com/deal_architect/2005/11/business_proces.html

[Bob W.]

Don't forget...there is almost always a process in use when work is being accomplished. Sometimes the process is ad hoc, sometimes effective, sometimes restrictive, sometimes useless. My preference is for "just enough process."

[Art Gould]

Process based management is still useful where standard actions, regularly repeated, can create customer value. We know from experience that nothing ever goes away completely.

My experience, both as customer and provider, resonates with the John Hagel report that most employee time is spent inventing resolution to breakdown conditions. So, Ross, what comes after process? There is a provocative article in the Fall Sloan Management review about what the authors call Commitment Based Management, which seems to provide an approach to the situations Hagel has described; where almost everything is an exception.

[Charles Bess]

I liked your post. I had done a post on the value of the unique situation - something a process usually tries to drive out as an anomoly. Retuning our processes and tooling to identify and get people aware of unique situations instead of trying to brush them under the rug, or drive them out of existance will be a useful shift in focus going forward.

[charlie]

i think google has become the poster child of process-less business. that's not fair.

for one, they do have processes. things seem to progress there by step one, two, three, and so on. maybe the processes are not so apparent or specified as at other companies.

for two, google has a very fluid way of working in a flat hierarchy, similar to a tradtional academic research library. in such an environment, there are social structures (techs, undergrads, graduate students, doctors, post-docs) that establishes who can do what and with what authority (authority by skill).

which leads to, three, the key reason google works well in this loose structure is the high number (all? most?) of employess are self-motivated and _self-guided_. my experience is that process is for those who need to be motivated and guided as to what to do. it provides stability and comfort and an external reference. google's workers for the most part live in an environment with internalized references for how and when to do things.

we call that culture. isn't culture the process by which we all agree to live in society. process gone bad is culture codified and crusted up.

what's your company's culture that everyone integates into how they work? how much have you replaced corporate culture with corporate processes?

hmmm.

[Rudy Hoeboer]

I don’t buy it! Process is what provides us with a comfort zone, a path to follow, especially when times get hard. Processes change in time to adapt to changing contexts, they will never disappear; sorry Ross, not even formal business processes. The fact that some organizations are in a downward spiral when it comes to innovation has a lot to do with the shareholder induced efficiency and cost cutting dogma’s. This is not about process, this is about context! We just need to recognize the changing context and adapt. To adapt we need to work together in multi-disciplinary teams (Toyota) instead of squeezing every last drop of life blood out of our employees and suppliers (GM) to gain short term advantages at the cost of optimizing ourselves to our grave. The processes should be owned by the people who act them out, not forced to wear as straight jackets by the top brass.

A good process is like a good contract. It doesn’t deal with any and all we want to do together, it describes what we want to do together and provides handles for when we hit an exception to the rule. As soon as a process or contract doesn’t, they both become straight jackets instead of a means to provide direction and the necessary safeguards to get there. Flexibility built in! Innovation is for the most part recombining existing solutions and there are proven processes with which we solve our contradictions to innovate. Practically every contradiction has been encountered before and people have come up with processes to deal with them. What we need therefore is not a process to innovate, but a process to come up with the right way to innovate.

[Graham Hill]

Ross

I think you are in danger of throwing the baby out with the bathwater.

All individuals, groups and organisations make extensive use of processes. At their simplest, they are just common ways of doing things. It is generally only organisations that actually write them down in any formal way. It helps them define standardised ways of working. If you are running a credit card company, processes are absolutely necessary or internal chaos will rapidly drive you out of business. But credit card companies use more and robuster process in the card administration part of the company where efficiency & effectivness is paramount, than in the marketing part where creativity & experimentation are.

Business processes are not intended to cover all eventualities, just the most common ones. This drives 80% or more of most businesses. For the not so commmon ones, you need people who know what the organisation intends, understand how the business works in general and who can resolve the exception with these in mind. And they will probably use a personal problem solving process to do it.

We all need and make extensive use of processes to provide a standard for how we do things. But, like in so many things, we need to be intelligent in how we apply them.

Perhaps you should be looking at how individuals in Japanese companies like Toyota apply processes to great effect, rather than trying to reject their use in Anglo-Saxon companies.

Graham Hill
Independent Management Consultant

[Lucas Rodriguez Cervera]

Very interesting article. I agree with John Husband. It is true that processes tend to be too rigid, but the solution is to make them more flexible, not to avoid them. There are several mechanisms to achieve this, from isolating the parts of the process that are more likely to change in the form of business rules that allow fine tunning, to allowing workers to change, update and enhance the process descriptions.

On the other hand, it is true that process exceptions consume most of the time, but could this be due precisely to a lack of process?

By the way I believe that social networks are a great aid to handle exceptions.

[Rich Tauchar]

A good topic. We want people using processes (if sensible and necessary), but not processes using people.

[Jessica Margolin]

Interesting. To the extent explicitly described processes exist to manage risk, I'd think that the processes would evolve to manage risk within the context of readily available information. Validation ("groupthink of the experts" is still "groupthink") and the correct interpretation of received information (i.e. filtering) are two areas which I could easily imagine would persist.

[Neil Macehiter]

What you are highlighting here, IMHO, is the fact that business processes vary a great deal. In some cases, business processes are highly structured, with significant amounts of automated support for roles within those processes and extensive control of the interactions and collaborations between roles e.g. straight-through-processing or call centres. In others, the processes are largely ad-hoc, with limited automated support for individual roles and limited, if any, control of interactions e.g. strategy-setting processes in most organisations. The problem is that when we in IT talk about business processes, we tend to focus on the former.