Waterfall Should Have Never Existed: Part 1

I’d like to begin by sharing a brilliant quote that puts the latest project management fashion, Agile, into humbling perspective: “A Waterfall project is just an Agile (Scrum) project with one huge sprint! An Agile project is just a Waterfall project almost entirely implemented using Rolling Wave Planning! (from George Intzesiloglou, UK on 2020-Sep-11 at ProjectManagement.com)

In the following two-part article, I will argue that:

  1. Agile’ is not agile.
  2. Waterfall’ is not the same as Critical Path.
  3. Waterfall’ should never have existed.
  4. Hybrid approaches between ‘Agile’ and ‘Waterfall-that-should-not-be’ should not exist either.
  5. A true hybrid approach between ‘adaptive project management’ (Agile) and ‘predictive project management’ (like Critical Path) should exist, can exist, and does exist. This is most viable.

In other words, be prepared to have your world shaken up a little bit. To entice you to continue to read, I promise to follow my claims with a solution for doing both Agile and Critical Path concurrently. In fact, you do not have to choose between Agile and Critical Path. Both can be done in one schedule!

I will explain how you can marry these two seemingly‑opposite approaches to scheduling in one project in Part 2 of this article. I will even recommend using ‘Agile’ to manage-down (i.e. your project team). At the same time, I will recommend Critical Path (CPM):

  • To manage-up (i.e. your executives),
  • To manage-out (i.e. your clients), and
  • To manage-in (i.e. yourself as the project manager)

Let’s use CPM to manage the overview of a project … and your sanity, don’t forget. Before we get to that; however, let’s consider why Agile became so popular to begin with.

 

Why Did Agile Become So Popular?

For at least twenty years and counting, the world around us has become more and more software driven, and, as a result, more digital. Electric vehicles are about 50% software, in terms of value, whereas fossil fuel cars are mostly hardware. Banks have essentially been software developing organizations for a long time. Fintech is accelerating this trend. In the last twenty years, records management began moving away from paper records to electronic ones. We see this happening in the medical world nowadays. The wall of paper files in your doctor’s office will soon be gone.

Obviously, as digital transformation has occurred, and the number of software projects increased exponentially. This has led to a dramatic growth in software development projects and in the number of software project managers we’ve seen and will continue to see. It’s accompanied by a flourishing consulting industry eager to sell the latest management fad to anyone with a heartbeat and willingness to listen. At the turn of the millennium, software projects were mostly organized in a Waterfall way and were reported to have poor results (Standish Group reports). In 2001, a group of software thought-leaders got together and hammered out the Agile Manifesto. They called themselves the ‘Agile Alliance’ and the Agile movement was born. It is riding the wave of digitizing our world.

The Agile movement set up associations (‘Agile Alliance’ and ‘DSDM Consortium’). The latter, Dynamic Systems Development Method (DSDM) came from the Rapid Application Development movement. These associations and thought leaders, authors, and researchers developed several flavors of Agile: Scrum, Scrum XP (eXtreme Programming), ScrumBan, DSDM, DevOps, and Lean-Agile, and market them vigorously. There are even resellers for the DSDM Consortium if you want to sell DSDM.

In the process of this development, Agilists hijacked a few techniques and presented them as their own. For example, KanBan and incremental scheduling (a.k.a. ‘Rolling Wave’ scheduling which preceded Agile by many years). Many private companies jumped on the opportunity and have developed software applications to support the Agile way of delivering projects: JIRA, @Task, Monday, Trello, Wrike, VersionOne, etc. As you can see, a whole new cottage industry grew out of the ‘Agile’ movement.

Since about 2010, Agile was challenged to start providing solutions at the executive level within organizations. These organizations needed an approach that would start providing predictions. One response to this was Dean Leffingwell private company, Scaled Agile, Inc. He created the ‘Scaled Agile Framework’ methodology (SAFe) and SAFe-assessment tools for organizations, as well as ‘SAFe certification’ for professionals. At the time of writing, over 600,000 practitioners have been trained worldwide in SAFe by over 350 ‘Scaled Agile Partners’ Simultaneously, Scrum was put into the public domain and responded with its ‘Scrum of scrums,’ which seems to flounder in the shadow of Scaled Agile.

When executives started to hear about ‘Agile,’ they kind of liked the term and, hence, the term ‘the agile organization’ was born. In short, ‘Agile’ began shaping up for the big leagues (i.e. the executive level).

Please note I am using the uppercase ‘Agile’ (as in ‘Agile Project Management’) and the lowercase ‘agile’ (as in ‘more adaptive’). I will use these consistently going forward to separate the ‘Agile’ project delivery method from the common sense meaning of the word ‘agile’ executives felt attracted to.

 

The ‘agile’ Organization

Just about every executive wants to have an ‘agile organization.’ The word ‘agile’ is just like the word ‘Quality.’ You cannot say ‘No‘ to it; you cannot NOT want it. Executives cannot say ‘No’ to ‘agile,’ because it is a word that is inherently positive in nature and wanted in a dynamic, capitalist society. The idea of ‘agile’ is something most executives want for their organization. Who does not want their company to be flexible and responsive to our constantly changing world? There are enough changes right now: the COVID-19 pandemic, the Black Lives Matter (BLM) movement against systemic racism, the opposing forces of globalization versus protectionism, the arrival of new technologies like machine learning and artificial intelligence, and our changing climate.

In the context of a quickly changing world, executives are being sold on the concept of making their organizations more ‘agile’ (lowercase!) by adopting ‘Agile’ (uppercase!) project management practices. In retrospect, I think it is a widespread misunderstanding that if executives implement ‘Agile Project Management’ (Uppercase), they will make their organizations more ‘agile’ (lowercase). After all, Agile is ‘a highly structured way of delivering projects.’; in other words: ‘Agile’ is not ‘agile’! In my mind, ANY form of project management makes an organization more ‘agile’ as in ‘adaptive’. After all, project management, by its definition, is ‘managing a temporary endeavor that creates a unique product, service, or result’ (PMBOK® Guide, any edition). Hence, project management ALWAYS brings about change, and ‘bringing about change’ is perceived as being ‘agile.’

Over time, it amazed me how ‘Agile Project Management’ (uppercase!) ran away with the main prize of getting the ear of C-level executives: PMI had tried for many years to get their ear after all. Executives, in their eternal quest to making their organization more ‘agile’ grew interested in ‘Agile Project Management’. The good thing is that ‘Agile Project Management’ has brought the topic of ‘project management’ into executive discussions and into the boardroom of organizations, and so PMI followed along.

In my mind, implementing ‘predictive project management’ techniques (i.e. Critical Path, Resource Critical Path, Critical Chain, Earned Value, Earned Work) make an organization at least as ‘agile’ (in the sense of ‘more adaptive’) as the ‘adaptive project management’ technique of uppercase ‘Agile’ would.

 

Agile Will Not Conquer the World

Agile will never conquer the entire world of projects! There are many types of projects that will NEVER be planned in an Agile way: construction projects, plant shutdown and turnaround projects, software enhancements on core operations systems (as little as possible downtime allowed), and many others. Can you imagine that a cloud scraper would be built in an Agile way: “Oh, we built the next floor, let’s go back to the client now … see if they like this floor and then we will ask them for a budget to build the next floor! Not sure if it will be financed … oh well, we will just stop this project for a while…” This will never happen; Agile will not and should not be implemented in the construction industry—and many other industries for that matter that can only use, sell or lease finished products (as opposed to ‘minimum viable products’, a concept from Agile). Just today, I heard a project manager who builds nuclear power stations say: ‘Agile never made sense to me!’.

Currently, Agile is only applied in research, software development, IT, and marketing projects. As far as I have observed, broadly confirmed by the 14thAnnual State of Agile’ report from VersionOne: “The survey results indicate that agility is largely confined to [software] development, IT, and operations.” (page 4). In my estimation, Agile will be applied in at most 50% of our world’s projects and, in a few years, less than that as I see in my crystal ball. Why? I think that the market share of Agile will shrink in the future for two reasons:

  • Firstly, executives, sponsors, and investors will never stop asking for forecasts that Agile does not provide, cannot provide, and does not want to provide. Agile is in the school of ‘adaptive project management,’ after all. ‘Agile’ allows the continuous adding of scope to or removing it from the project that is delivered in several fixed periods with a team of fixed size. In this respect, Agile is not the opposite of predictive project management, because none of the predictive techniques prescribes that the scope should be fixed. They use change requests to manage it. Only ‘predictive project management’ (i.e. Critical Path, Resource Critical Path, Critical Chain, Earned Value, Earned Work, etc.) are designed to, and are able to and will always provide time forecasts by identifying the dependencies between the items in the WBS. The key difference between ‘adaptive’ and ‘predictive’ techniques is that all predictive techniques expect you to identify all dependencies between activities (create complete and correct network logic), whereas Agilists are not expected to do that.
  • The second reason that I think Agile will shrink in the future is that it is, in its essence, non-committal in terms of finishing the product or service, and in terms of time (deadline) or cost commitments (budget). Agilists will not commit to building a complete product, neither will they tell you in advance how much it will cost, or how long it will take. Agilists will get you to start investing and then keep you going in a way that reminds me of how consulting firms would string their clients along in the 1990’s using contracts of the ‘Time & Materials’ With a healthy dose of sarcasm, ‘Time & Materials’ can be summarized as: “I will work as slow as possible for as long as possible and expect to be paid for this!” Consulting firms knew how to maximize their profits in the 1990’s! Since, Agile is a new incarnation of Time & Materials contracting in my mind, I am afraid that Agile will be pushed back to a much smaller market share in the world within the foreseeable future. It was no surprise that Time & Materials contracting was relegated to the sidelines by the late 1990’s and replaced by ‘Firm Fixed’ price contracts. For ‘Firm Fixed’ price contracts, you need to establish the entire scope and produce accurate predictions. Predictive project management is much better at forecasting than Agile ever will be.

Many people think that ‘predictive project management’ (i.e. Critical Path) is the same as ‘waterfall.’ This is nonsense! ‘Waterfall’ means that, before the project starts, you plan an entire project out in (somewhat) chronological phases. The concept of ‘phases’ feels good, and essentially, ‘phases’ are an arbitrary way to cut up a large piece of bread into smaller slices that you can actually digest. Phases can even overlap in time, because there are no transition criteria that must be met before a project goes from one phase over into the next. This is where Dr. Robert Cooper has improved the phase-based thinking significantly by adding transition criteria to the phases. He captured this in his famous Stage-Gate methodology: Each gate has certain criteria that MUST be met before a project can move to the next stage (phase). Notice that he replaced the word ‘phase’ with the word ‘stage’; he could have called his invention ‘phase-gate’, but he didn’t.

Since ‘Agile’ was a reaction to ‘Waterfall that-should‑never‑have‑existed,’ what does this mean for ‘Agile’ itself? Is it a solution that is in a desperate search for a problem? I do not think so. When I state ‘Waterfall’ should have never existed, I am not saying ‘Agile’ project delivery should never have existed. Agile has its place and will hold its place. Agile fits well in a situation where people are working at the frontier of what people know in its broadest sense (knowledge-constrained projects). Obviously, R&D projects are in the field of knowledge-constrained projects. In software development projects, customers often find it difficult to verbalize all requirements, which imposes a knowledge-constraint to the project manager. If you are interested in the topic of picking the right scheduling approach, please retrieve my (free) White Paper ‘Choosing the Best Scheduling Approach for Your Project’ and look at its Project Ideals and Constraints (PIC) matrix: click here ( https://projectprocorp.com/collections/white-papers/products/choosing-the-best-scheduling-approach-for-your-project ) or buy my textbook ‘Forecasting Programs’ (same website, but under ‘Books’). Both discuss what place in this world all scheduling methods that I’ve mentioned in this article, have, including ‘Agile’.

 

‘Waterfall’ Should Never Have Existed!

Predictive project management has never used ‘phases’ for as long as I have been at it (soon 30 years!). Ever since the first PMBOK® Guide was published in 1983, PMI encouraged ‘deliverable-oriented’ Work Breakdown Structures (WBS) rather than ‘phase-oriented’ WBS’s. In contrast to ‘phases’, a ‘deliverable’ is simply defined as ‘a measurable thing of value that someone is willing to pay for’ (source: my textbook ‘Forecast Scheduling’). A ‘phase’ is not measurable, but a ‘deliverable’ is measurable by definition! So, given the fact that ‘Waterfall’ is based on a phase‑oriented breakdown of the work in a project AND ‘phases’ never were the PMI-recommended breakdown-orientation for developing WBSs, I would argue that ‘Waterfall’ never really existed (except in the minds of some software development project managers and their misguided world of Software Development Life‑Cycles (SDLC) of the 1990’s). Hence, I’d like to proclaim: Waterfall is dead! Waterfall should have never existed!

Why do we continue to refer to ‘Waterfall’ if it was a dead-end in the first place? Let’s simply stop talking about ‘Waterfall’; it is a dinosaur now; it is dead! Let’s get rid of it; it’s time to move on!

 

To Hybrid or Not to Hybrid?

In the past few years, more and more organizations are publicly admitting to the fact that Agile was not implemented as per the Agile textbook, but in a modified way, a mix between Agile and Waterfall-that-should-never-have-existed. Many people are leery of a ‘hybrid‘ approach that indeed raises an important question: Do we end up with the best of both worlds OR the worst of both?

Several project managers have written accounts in which they describe how they ended up with the worst of those two worlds of Agile and Waterfall-that-should-never-have-existed. For example, when there are many conflicting deadlines and the portfolio managers are not setting clear priorities, the autonomy of the agile teams will indeed get compromised: “Do this first instead of your sprint work!” Is this particular case, was it a real ‘hybrid’ approach? Not in my mind! At most, this is ‘Agile’ with a ‘portfolio manager override.

In my opinion, a true ‘hybrid’ form is a combination of an adaptive (Agile) and a predictive method (like Critical Path). When I realized that Critical Path is just a different approach and not the opposite of Agile, I started thinking about the following important question: Can Agile perhaps be combined with Critical Path in one project schedule? I’ve seen many situations where a project team was asking for Agile while executives asked for forecasts. There is a clear use-case for combining Agile with Critical Path in my experience.    

 

Can Agile be Combined with Critical Path in One Schedule?

I mentioned already that Agile and Critical Path are NOT each other’s opposite: Agile and Waterfall are opposites, and instead of coming up with some shade of gray that works for everyone, I would like to present a fresh and completely different approach. What if Agile and Critical Path happily existed side-by-side in a single project schedule without affecting each other (much)?

How would that work? Well, there are a few dilemma’s to resolve when you try to combine two approaches that look at first sight very different:

  1. The WBS of an Agile schedule is sprint-oriented (adaptive) with user stories within each sprint, whereas the WBS in a Critical Path schedule (predictive) is broken down in a deliverable-oriented way with activities within each deliverable.
  2. Agilists estimate in anything except hours, whereas predictive folks need the estimates expressed in hours to create their Gantt Charts and calculate Critical Path.
  3. Agilists don’t like identifying all dependencies between activities, but predictive people need all dependencies, the right types of dependencies and all entered correctly into their scheduling application. They need complete and correct network logic.

I have solved the first two dilemmas, and propose that Agilists need to compromise on the third to make combining BOTH approaches in ONE schedule work, which I will describe in Part 2 of this article series!

What do you think, is there a way to use Agile with your team and Critical Path with your executives? Please comment below and look for part 2 of this article, which will lay out solutions to each of these dilemmas we’ve discussed.

 

Read Part 2 of Waterfall Should Have Never Existed

 

Next Webinar

Waterfall Should Have Never Existed: Part 2

Written by Eric Uyttewaal
Eric is a thought leader on project, program, and portfolio management. He spends most of his time using software from Microsoft. He has authored seven well-known textbooks including ‘Forecasting Programs,’ 'Forecast Scheduling with Microsoft Project 2010/2013/Online,’ and ‘Dynamic Scheduling with Microsoft Project 2000/2002/2003.’ He founded ProjectPro, which specializes in Microsoft Project, Project Server and Project Online. Eric developed several Add-ins with his team that enhance the capabilities of Microsoft Project in creating better schedules (Forecast Scheduling App), managing cross-project dependencies (CrossLinksPro), identifying and documenting the Critical Path (PathsPro) and creating S-curve reports (CurvesPro). He was president of PMI-Ottawa in 1997. Eric has received awards from PMI in 2009, from MPUG in 2012, and from Microsoft from 2010 until 2017 (MVP).
Share This Post
Have your say!
00
15 Comments
  1. This is gold! While I haven’t got your experience yet, you are certainly putting some of my thoughts into words. The dead end of agile as fix-it-all, agility becoming a dangerously seducing word to powerful non-practitioners in a time when complexity seems to have got out of hand, the somewhat weak position of “traditional” project management linked with waterfall… There must be a better way. Putting the weight on “predictive” requires not only a better marketing, but also better processes & tools. See Scrum for example. What made this particular flavor of Agile extremely appealing is its simplicity & availability – the manual is concise, immediately available to all (in many languages), with an accessible 3-5-3 concept (roles, events, artifacts). If you compare this to what the “traditionalists” propose… (myself included). Even the concept of “Gantt charts” and the body of knowledge size related to one of its most popular tools (MS Project) make it discouraging, difficult to sell. Not to mention EVM or Monte Carlo Simulations.

    To sum up, we have work to do in that respect. I can’t wait to read the 2nd part of your article. Thank you for what you’re doing.

    • Hi Lech, You are right that we need to make predictive project management simpler and more accessible. I have noticed that young project managers who are so used to entirely discoverable software (without reading books or attending courses) that they are not willing to learn the principles of project management or the right way to use project scheduling software (as described in my ‘Forecast Scheduling’ textbooks).

  2. Looking forward to part 2. I totally agree with all u said… and before I retired last year, I was working with PMs that had hand-rolled the hybrid into schedules. I’ve tried having both models in the same schedule using MSP and Project 365, and well, I wasn’t impressed with the implementation… but let’s see what u come up with 🙂
    Cheers,
    Jigs

    • Hi Jigs, I never was impressed with any hybrid schedule I have seen. I have never even been impressed with any real-world Agile schedule I have seen: Inevitably, it had one or more items repeated (extended) over multiple sprints because it just couldn’t fit into one sprint, which defeats the purpose of sprints in my mind. Thanks!

  3. Hallelujah! I have been saying this for years and had to deal with being “anti-agile”. I wrote a blog post in 2017 about this very thing. https://contrarypm.blogspot.com/2017/08/agile-we-have-seen-enemy-and-it-is-us.html

    I am amused at the Agile/Scrum zealots who are every bit as purist, rigid, and strict as they accuse “waterfall” project managers of being. I attend an Agile Community of Practice and watch while members struggle to deal with situations for which project management has had answers for decades. There’s no need for them to reinvent project management.

    I find my scheduling solutions in the PMI Scheduling Standard. There are currently at least 19 different models to pull from. When I do enterprise level complex projects with many different project patterns involved, I have to apply what I know to each piece so the schedule model is appropriate. Thank you for this article!

    • Hi Sonya, I enjoyed your article as well! Thanks. I share your experience when I venture into Agile environments or events that they argue long and hard about what true Agile is. The PMI Scheduling Standard is indeed a very useful document still, unlike the PMBOK that has been too impacted by Agilists with its Agile Practice Standard attached to it in the current 6th edition. Thank you!

  4. This article is spot on! I’ve been dealing with execs that have been telling me we need to move to “Agile” for quite a while without understanding what they are asking. I will be sharing this article with several people! Looking forward to Part 2.

  5. Jeff, I have seen too many executives falling prey to Agile disciples as well and I am determined to start fighting back and push Agile back to the corner where it belongs. Agilists have been trying to take over the world of project management; they are even trying to make forays into the world of construction these days. They don’t belong there. They should even not be allowed to manage all software projects: New Software applications or disruptive software: yes. Enhancements to large, inhouse-developed, mission-critical software systems: No. Software that goes into existing physical products: no. If there is no missing knowledge (features, use cases) in projects, Agile should not be applied. I explain this in more detail in the White Paper I refer to.

  6. Great job, Eric! I have been an MPUG member for quite some time, and tend to see the regular newsletters as, “Wow, I should read this someday.” Your article DEMANDED I read this today! As several others have said – You have said what I have been thinking for quite some time! But, I further appreciate your knowledge of many methodologies, frameworks, etc., and your ability to fuse together several thoughts as well as you have. Thanks for framing this discussion as well as you have. I too look forward to Part 2! Congrats on what you do!

  7. Thanks Eric. Makes a lot of sense. Your articles always do. I really like the “Time and Materials” and Agile analogy. I have seen it for many years and never could put my finger on it and now call it Agile.. While working on many projects, there has been no incentive to finish a project or sprint for that matter. It keeps the money rolling in for consultants and personally, I like that. It’s may opinion, IT manager’s are probably ok with Agile to, because it keeps there staff busy and results are not required. And that is where the problem is. There isn’t as much accountability and visibility of failure. Budgets are not their own money. If it was, then the movement to predictive schedule and critical path would be the mainstream.

    There is a millennial mindset today. Let’s hide our accountability in a group and now we aren’t responsible. It’s somebody else’s money so who cares.

    I may sound pessimistic, but a pendulum swings back and forth and with each swing things change for the better. I see a day where critical path, agile and resource management will be part of the same project management fabric.

    • HI Michael, It took me long time as well to realize that Agile is Time & Materials in new clothes. I like the way you describe the consequences of Agile. I also hope for a better toolset now we learn how to integrate some Agile concepts into our thinking. Eric

  8. Fantastic article which drew together and articulated the thoughts I’ve been having about the Agile movement lately. It seems Agile is the ultimate buzz word nowadays. I cringe when I hear phrases like “business agility”. I hear those phrases and thing, “Well duh! Of course it’s good to adapt and evolve” – there doesn’t seem to be too much substance behind these types of phrases. Like you said, it’s been a clever way to get executives on board with project management. On a similar note, I fear with the new edition of the PMBOK Guide coming out will overcompensate and put way too much emphasis on Agile approaches (If you think the 6th Edition had too much emphasis on Agile, wait until you read the 7th). It’s being roughly halved in length which I fear will dumb it down to an extent (maybe that’s too harsh of a word?) by not mentioning proven Project Management techniques . But I’ll wait to make further judgement. On the other hand in an optimistic light, the narrowing of the PMBOK Guide to a more focused document that’s potentially more palatable and ultimately useful to those that were turned off by lengthier editions “could” be a positive in one regard.

  9. Hi Denver, I have also been worried about Agilists hijacking the PMBOK working group. I was glad to see that PMI opted for publishing it as a separate Practice Standard instead of re-writing the entire PMBOK Guide from an Agile perspective. PMI only incorporated snippets of Adaptive Project Management thinking in the 6th Edition of the PMBOK. We should indeed be concerned about the PMBOK and it is time for us to get involved again with the PMI Standards Groups to make sure that Predictive Project Management keeps its rightful place.

  10. Terrific article for a lay person. I teach a Project Management Fundamentals class and find so many are frustrated because they want to be “agile” using Agile. 🙂 I see the PMBOK 7 will be looking at Performance Domains vs Processes. Principles that are generally accepted. It seems like the combination of predictive and adaptive is very much stressed. Frankly, rocked my world a little, tbh. Thank you for clarification.

Leave a Reply