Monday

Tag Archives: Agile

A Brief History of Agile

Posted on February 3, 2024 by | 1 comment

The history of agile software development is not what most people think, and is nothing like the story pushed by most Agile Evangelists.

Our latest publication A Brief History of Agile shows that from the beginning of large system software development the people managing the software engineering understood the need for prototyping and iterative and incremental development. This approach has always been part of the way good software is developed.

The environment the authors of the early papers referenced and linked in the article were operating in, satellite software and ‘cold-war’ control systems, plus the limitations of the computers they were working on, did require a focus on testing and documentation – it’s too late for a bug-fix once WW3 has started…..  But this is no different to modern day control systems development where people’s lives are at stake. Otherwise, nothing much has changed, good software is built incrementally, and tested progressively,  

The side-track into ‘waterfall’ seems to have bee started by people with a focus on requirements management and configuration management, both approached from a document heavy bureaucratic perspective. Add the desire of middle-management for the illusion of control and you get waterfall imposed on software developers by people who knew little about the development of large software systems. As predicted in 1970, ‘doing waterfall’ doubles to cost of software development. The fact waterfall survives in some organisations through to the present time is a factor of culture and the desire for control, even if it is an illusion.

The message from history, echoed in the Agile Manifesto, is you need to tailor the documentation, discipline, and control processes, to meet the requirements of the project. Developing a simple website with easy access to fix issues is very different to developing the control systems for a satellite that is intended to work for years, millions of miles from earth.

To read the full article and access many of the referenced papers and third-party analysis see: https://mosaicprojects.com.au/PMKI-ZSY-010.php#Agile

1 Comment

Posted in Agile Ideas, General Project Management, Project Controls

Tagged , , ,

Agile’s Hidden Secret!

Posted on January 23, 2024 by | Leave a comment

The two fundamental questions standard agile metrics cannot answer consistently are:

1.  How far ahead or behind schedule are we currently?

2.  When are we expected to finish?

Most of the tools and techniques used to manage Agile projects are good at defining the work (done, in-progress, or not started) and can indicate if the work is ahead or behind a nominated planned rate of production, but there is no direct calculation of the time the work is currently ahead or behind the required production rate, or what this is likely to mean for the completion of the project. A full discussion of this topic is in Calculating Completion.  However, most project sponsors and clients need to know when the project they are funding will actually finish, they have other people that need to make use of the project’s outputs to achieve their objectives. At present all Agile can offer is an educated assessment based on the project teams understanding of the work.

Work Performance Management (WPM) has been designed to solve this challenge by providing answers to these questions based on consistent, repeatable, and defensible calculations.

WPM is a simple, practical tool that uses project metrics that are already being used for other purposes within the project, to assess progress and calculate a predicted completion date by comparing the amount of work achieved at a point in time with the amount of work needed to have been achieved. Based on this data WPM calculates the project status and the expected completion date assuming the rate of progress remains constant.

Our latest article, WPM for Agile Projects identifies the cause of this information gap in Agile project management, explains the inability of current tools to accurately predict completion and demonstrates how WPM will effectively close this critical information gap.
Download WPM for Agile Projects: https://mosaicprojects.com.au/Mag_Articles/AA040_-_WPM_for_Agile_Projects.pdf

For more on the practical use of WPM, free sample files, and access to the tool see: https://mosaicprojects.com.au/PMKI-SCH-041.php  

Leave a comment

Posted in Agile Ideas, Project Controls

Tagged , , , ,

The Problem with Waterfall

Posted on January 14, 2024 by | 9 comments

The term ‘waterfall’ is seen in lots of different posts without any clear definition of what the writers of those posts mean by the term.  The only constant seems to be in each of the writer’s view ‘waterfall’ is not Agile, and generally represents bad project management practice. In summary, the agile advocates view seems to be:

Agile: A well-defined flexible project delivery process, based on the Agile Manifesto, applicable to software development and a wide range of other “soft projects” such as business change. Agile = Good!

Waterfall: Any project delivery process that is not Agile. Waterfall = Bad!

There are many problems with this simplistic viewpoint starting with the fact the concept of ‘waterfall’ had a very short life and with the possible exception of a few, very traditional, software development organizations, no one uses waterfall for anything.
 

History of Waterfall.

To the best of my knowledge, the first publication to use the term Waterfall was in the 1976 paper Software Requirements: Are They Really a Problem, by T.E. Bell and T.A. Thayer. This paper misrepresented the 1970 paper Managing the development of large software systems, by Dr Winston Royce[1]. Royce proposed an iterative approach to the development of large systems, but Bell and Thayer falsely claimed he supported ‘waterfall’[2].  

Summary diagram from Royce 1970.

The real start of Waterfall was the publication in 1988 of DOD-STD-2167A by the US Department of Defense, which established uniform requirements for the development of software based on the Waterfall approach[3].   

Extract from DOD-STD-2167A

Problems with the Waterfall approach were quickly identified and in 1996 MIL-STD-498 was released by the US Department of Defense to correct the problems. Officially Waterfall was dead and buried but many companies had adopted waterfall and because waterfall projects were slow and subject to delay, hourly paid staff and contractors had a powerful incentive not to change despite many better software development processes being developed starting from the early 1980s.   

Other types of projects and project delivery.

Waterfall was a short-lived software development methodology. The vast majority of projects in the construction, engineering, oil & gas, defence, and aerospace industries use project delivery methods based on the approaches described in A Guide to the Project Management Body of Knowledge (PMBOK® Guide)—Sixth Edition, and a range of other standards. These other projects generally have three phases:

  1. A definition phase undertaken by the client organization to define the capabilities of the product being developed
  2. A procurement phase where the client selects a delivery agent for the development of the product
  3. A delivery phase where the delivery agent builds and delivers the product

The design of the product (ship, building, rocket, etc.) may be undertaken in full or in part during any one of the three phases. A minimum level of design is required to initiate procurement, but for simple buildings and civil engineering projects, it is not unusual for a complete design and specification to be provided by the client.

The procurement phase may be a simple pricing exercise, or a complex, and phased, design process (sometimes even involving the production of working prototypes), with selection being based on the capabilities of the design produced by the successful tenderer.

Then, in many projects, a significant amount of detailed design is still required during the delivery phase, including shop drawings produced by subcontractors and suppliers.

Similarly, the procurement arrangements vary widely. The client may choose to enter into some form of alliance or partnership with the preferred delivery agent based on shared risk and profits, or the client may choose a hard-dollar contract based on a fixed price to deliver a fixed scope, or some other form of contractual arrangement.

The only certainties are that the typical project approaches used for the vast majority of ‘other’ projects bear no resemblance to the waterfall approach, and this ‘other’ classification includes more than two-thirds of the world’s projects by value.

Conclusions

  1. I suggest it is time to follow the US DOD lead from 1994 and bury the concept of ‘waterfall’ – using the name 30 years after it was officially dropped is long enough.
  2. People involved in the ‘Agile’ industry need to wake up to the fact that software development is only one of many types of project. Most of the ‘other’ types of project do not use Agile, and they certainly don’t use waterfall.
  3. Agile and agility are not synonymous – all organisations benefit from a degree of agility, but this has nothing to do with selecting the best project delivery methodology (more on this later).
  4. In the 21st century, Waterfall is not synonymous with over documentation and/or bad project management. There is plenty of bad project management practice around. But bad management needs to be called out for what it is – 99.999% of the time the bad managers are not trying to use waterfall in their work.   

Ditching the concept of waterfall does create a couple of challenges – we all have an understanding what Agile means as a project delivery process, we need similar generally accepted classifications for other types of project delivery – more on this later. Similarly, the bad management practices branded as ‘waterfall’ need to be identified and understood, you cannot improve a bad process until the root cause of the problem is understood.

For more on Agile management see: https://mosaicprojects.com.au/PMKI-ITC-040.php#Process1

Note: THE MYTH OF THE ‘WATERFALL’ SDLC by expands on this post in far greater detail and is highly recommended as a reference: http://www.bawiki.com/wiki/Waterfall.html

[1] Download a copy of the 1970 Royce paper: https://mosaicprojects.com.au/PDF-Gen/Royce_-_Managing_the_development_of_large_software_systems.pdf  See Fig. 10.

[2] Download a copy of the 1976 Bell & Thayer paper: https://mosaicprojects.com.au/PDF-Gen/software_requirements_are_they_really_a_problem.pdf

[3] Download DOD-STD-2167A Defense System Software Development (1988): https://mosaicprojects.com.au/PDF-Gen/DOD-STD-2167A.pdf

9 Comments

Posted in Agile Ideas, General Project Management

Tagged , ,

Commercializing Agile

Posted on January 2, 2024 by | 1 comment

Agile in its various forms is becoming mainstream, and this means an increasing number of commercial contracts are being delivered by contractors who either choose, or are required, to use an agile methodology to create their contracted deliverables. While this is probably a good thing, this shift in approach can cause a number of problems. The major shift in approach is managing the legally imposed, contractual requirement to deliver 100% of the designated project deliverables on time.  The funds available to the contractor to do this work are defined by the contract price, and failure to deliver the contracted deliverables within the contracted timeframe can lead to significant cost penalties being applied[1].  

The requirement to deliver a project as promised in the agreed contract is business-as-usual for most construction and engineering project and is common across many other industries. While relatively rare software companies have also been successfully sued for breach of contract when their deliverables did not meet the contracted obligations, some early cases are discussed in Software sales hype and the law, and IT Business Sued for US$300 million+.  In short, choosing to use Agile as a project delivery methodology will not change the laws of contract, which means organizations using the agile methodology will need to become more commercial  and adapt their processes to include:

  1. Developing the realistic time and cost estimates needed to enter into a contract.
  2. Monitoring and controlling the project work to optimize the delivery of the contracted requirements within the contract timeframe.
  3. Enhancing their contract administration to deal with changes, variations, reporting, claims and other contractual requirements and issues.

This post is a start in looking for practical solutions to some of these challenges.

Contract Claim Administration

Two of the core tenets of Agile are welcoming change when it creates additional value for the client, and working with the client to discuss and resolve problems. While these are highly desirable attributes that should be welcomed in any contractual situation, what happens when the relationship breaks down, as it will on occasions?

The simple answer is that every contract is subject to law, and the ultimate solution to a dispute is a trial, after which a judge will decide the outcome based on applying the law to the evidence provided to the court. The process is impartial and focused on delivering justice, but justice is not synonymous with a fair and reasonable outcome.  To obtain a fair and reasonable outcome, evidence is needed that can prove, or disprove each of the propositions being put before the court.

The core elements in dispute in 90% of court cases relating to contract performance are about money and time. The contractor claims the client changed, or did, something (or things) that increased the time and cost of completing the work under the contract; the client denies this and counterclaims that the contractor was late in finishing because it failed to properly manage the work of the contract.    

The traditional approach to solving these areas of dispute was to obtain expert evidence as to the cost of each change and the time needed to implement each of the changes and its effect on the completion date. Determining the cost of a change is not particularly affected by the methodology used to deliver the work. The additional work involved in the change and its cost can be determined for a change in an ‘agile’ project in a similar way to most other projects. Where there are major issues is in assessing a reasonable delay.

For the last 50+ years, courts have been told by many hundreds of experts, the appropriate way to assess project delay is by using a critical path (CPM) schedule. Critical path theory is based on an assumption that to deliver a project successfully there is one best sequence of activities that have to be completed in a pre-defined way to achieve the best result. Consequently, this arrangement of the work can be modelled in a logic network and based on this model, the effect of any change can be assessed.

Agile approaches the work of a project from a completely different perspective. The approach assumes there is a ‘backlog’ of work to be accomplished, and the best people to decide what to do next are the project team when they are framing the next sprint or iteration. Ideally, the team making these decisions will have the active participation of a client representative, but this is not always the case. The best sequence of working emerges, it is not predetermined and therefore a CPM schedule cannot be developed before the work is started. 

Assessing and separating the delay caused by a change requested/approved by the client from delays and inefficiencies caused by the contractor is difficult at the best of times, this process becomes more difficult in projects using an agile approach to the work but is essential for assessing time related claims under a contract.

There are some control tools available in Agile, but diagrams such as a burndown (or burnup) chart are not able to show the effect of a client instructing the team to stop work on a particular feature for several weeks, or adding some new elements to the work. The instructions may have no effect, the team simply work on other things, or they may have a major effect. The problem is quantifying the effect to a standard that will be accepted as evidence in court proceedings.  CPM has major flaws, but it can be used to show a precise delay as a specific consequence of a change in the logic diagram. Nothing similar seems to have emerged in the Agile domain.

These challenges are discussed in WPM – Schedule Control in Agile and Distributed Projects (and are the focus of ongoing work).

The agile paradigm has a lot to offer, but to become a commercially effective option, the project controls and contractual frameworks will need a major overhaul.  For more on managing agile see: https://mosaicprojects.com.au/PMKI-ITC-040.php#Process1

[1] Developing and defending contractual claims is discussed in Forensic analysis and reporting (cost & time): https://mosaicprojects.com.au/PMKI-ITC-020.php#Process1

1 Comment

Posted in Agile Ideas, General Project Management, Project Controls

Tagged , ,

Predicting project outcomes is important!

Posted on July 25, 2023 by | Leave a comment

The recent cancellation of the 2026 Commonwealth Games by the Victorian Government is a dramatic example of using predicted project outcomes to minimize damage to an organisation. The escalation in the predicted cost of delivery from $2.6 billion to above $6 billion suggests the original bid was wildly optimistic and discovering how such an error occurred should be worthy of enquiry, but that is not the focus of this post.

Once the predicted costs moved to a point where there was no benefit in continuing with the project, it was terminated. While the cancellation could have been done earlier and far more elegantly the fact remains cancelling the project was by far the best decision.

However, to be able to make this type of call, management need information they can rely on. Even then the decision is not simple.

Cost considerations

A decision to cancel a project has to balance: sunk costs, the cost to complete, the expected benefits, and the cost of not completing the project.  The three elements that matter are the cost to complete vs the benefits (sunk costs are lost either way), and the costs of not completing the project.

For more on sunk costs see: https://mosaicprojects.com.au/Mag_Articles/P022_Sunk_Costs.pdf

Time considerations

Time is usually a secondary consideration but can be vital – the Commonwealth Games facilities would need to be open before the games start!  For more normal projects knowing the current projected completion date and the variance at completion are still important for two reasons.

First, the cost of time matters and needs to be included in the cost to complete estimate. Delayed completion may also impact benefits.

Second and more important, time issues tend to emerge as problem well before cost issues show up. A project that is losing time and is expected to finish late will almost inevitably show negative cost variances sooner or later.  Conversely, fixing the root cause of the time issues will often have a positive effect on the overall costs as well.

The problem is most projects do not run systems that are capable of producing a reliable prediction of the expected completion date. Our recent paper Calculating Completion looked at seven different methodologies for managing a project, only two gave reliable predictions of the completion date; these were Earned Schedule and Work Performance Management.

Earned Schedule (ES) was the best option, but implementing ES requires a significant investment in skills and systems. 

Work Performance Management (WPM) achieved similar results to ES without the overhead. All that is required to use the WPM spreadsheet is three bits of information.  To set up the WPM model the amount of work to be accomplished and the time allowed is needed, any metric can be used provided it is applied consistently. This baseline gives you the amount of work expected to be accomplished by a given date. The other bit of information is the actual amount of work achieved by the date.  From this data the predicted completion date is calculated.

The assumption built into WPM is that work will continue at the current rate. If the result is not acceptable, management needs to do something to change the rate of working. If this is not feasible, then the viability of the project needs to be considered and/or the baseline reset to what is achievable. For more on WPM see: https://mosaicprojects.com.au/PMKI-SCH-041.php#Overview

Leave a comment

Posted in General Project Management, Governance

Tagged , ,

Scheduling Challenges in Agile & Distributed Projects

Posted on February 10, 2023 by | Leave a comment

Our paper looking at the scheduling challenges in agile and distributed projects has been published in the February 2003 edition of PM World Journal: https://pmworldjournal.com/

Critical path theory is based on an assumption that to deliver a project successfully there is one best sequence of activities to be completed in a pre-defined way. Consequently, this arrangement of the work can be modelled in a logic network. However, while CPM has proved to be an effective controls tool for many types of projects, it is equally apparent the CPM paradigm does not apply to a wide range of other project types including soft projects and distributed projects.

The focus of this paper is to:

  • Briefly define the management assumptions that support the use of CPM scheduling, its origins, and limitations
  • Develop a classification framework of project characteristics to help define the potential usefulness of CPM scheduling
  • Briefly describe some of the management approaches currently used in non-CPM projects including agile and lean, their benefits and limitations
  • Consider the application of the framework discussed above applied to a typical wind farm project
  • Develop general recommendations for the management of non-CPM projects focused on optimizing the efficient use of resources.

Based on this foundation, two additional papers will look at:

  1. Implementing a robust system for reporting progress and predicting completion in agile and distributed projects that can be applied to any class of project.
  2. Assessing delay and disruption in agile and distributed projects where the use of a CPM schedule is not viable.

Download Scheduling Challenges in Agile & Distributed Projects: https://mosaicprojects.com.au/PDF_Papers/P208_Scheduling_Challenges_in_Agile_+_Distributed_Projects.pdf  

For more on this topic see: https://mosaicprojects.com.au/PMKI-SCH-010.php#Issues-A+D

Leave a comment

Posted in Agile Ideas, Project Controls, Project Typology, Scheduling

Tagged , , ,

Do project plans predict or create the future?

Posted on June 30, 2022 by | 2 comments

Our latest article, Is Planning Predictive or Persuasive suggests that project controls staff and management place too much emphasis on attempting to develop the ‘perfect plans’ that accurately predicts future outcomes (a passive process that is doomed to failure), and not enough on using the planning and scheduling processes to proactively influence the direction of the project’s future work.

Download Is Planning Predictive or Persuasive from: https://mosaicprojects.com.au/Mag_Articles/AA019_Is_Planning_Predictive_or_Persuasive.pdf

2 Comments

Posted in Agile Ideas, General Project Management, Project Controls

Tagged , , , ,

There’s Agile and there’s Agile, understand the difference!

Posted on January 1, 2020 by | Leave a comment

One size does not fit every situation even in an agile world!  The focus of this new article is to identify the differences in management approach needed to maximize value in three different situations where an ‘Agile’ approach can be used.

For any Agile approach to achieve its promise, the upper echelons of the organisation need to become agile aware and adapt the way projects are initiated, funded and governed so that the project team can optimize their use of Agile processes to maximize the creation of value. Download the article from: https://mosaicprojects.com.au/Mag_Articles/SA1060_Agile_Spaces.pdf

For more papers on Managing Agile see: https://mosaicprojects.com.au/PMKI-XTR-010.php#Process1

Leave a comment

Posted in Agile Ideas

Tagged , ,

2 New Presentations Uploaded

Posted on September 14, 2019 by | Leave a comment

Following on from presentations at Monash University and the PGCS Symposium, 2 new presentations have been uploaded to the Mosaic website.

Controlling Agile looks at the the various options for ‘agility’ in a project and then maps potential controls options to the different ‘agile approaches’.
– Download the presentation
– See more on managing agile.

Setting Your Project Up For Success is a simple paper outlining the minimum elements necessary to have a reasonable chance of delivering a project successfully.

Leave a comment

Posted in Agile Ideas, General Project Management, Project Controls

Tagged , ,

The maturing or ‘agile’

Posted on May 8, 2016 by | 1 comment

A deliberately provocative article on Linked-In asks the question is ‘Agile Dead?’; a discussion on how various aspects ‘agile’ invented by different individuals and groups are fading from prominence follows.  Agile is not my area of expertise but the article seems designed to generate attention without really saying anything new.

What the article did prompt in my thinking was the question ‘What is agile?’. Concepts vary from:

  • The Agile Manifesto (which is basically 101 common sense) created to overcome the failures of rigid IT development that required a 100% complete fully detailed plan before people really knew what the problem was (often referred to as ‘waterfall’ development but nothing like the original ideas in the waterfall concept).
  • Through to the agile anarchist community who’s mantra seems to be ‘trust us all of our teams are above average’ and we will make you really nice software without any discipline (a concept that ignores the mathematical fact that 50% of any group have to be below average….).
  • Then there are all of the various ‘agile’ methods from ‘Scrum’ to ‘XP’.

Ergo ‘Agile’ or ‘agile’ can mean virtually anything to anyone.  In contrast to all of these specific variants, I would suggest at its root ‘agile’ is a concept or philosophy rather than a methodology or process; useful philosophies rarely ‘die’.

What is emerging I believe is a gradual understanding that the false concepts of ‘command and control[1]’ and ‘certainty, based on a fully detailed plan[2]’ are slowly disappearing from management thinking (although there are still plenty of recalcitrant ‘fossils’ embedded in far too many management structures) – detailed planning months or years in advance of the work, done at a time where the work is imprecisely understood cannot control an uncertain future regardless of contract conditions and the exhortation of management. These ideas are slowly being replaced by an adaptive approach to projects that engages stakeholders and focuses on actually achieving the stakeholder’s objectives and realising benefits, ie, an ‘agile’ approach.

Every project and every project management system can benefit from some elements of ‘agile’ (which overlaps with many other concepts such as ‘light’, ‘lean’, and ‘last planner’. The key tenets seem to be:

  • involve your stakeholders,
  • trust your team,
  • don’t waste time planning in detail things you don’t have detailed knowledge of[3],
  • adapt to changing circumstances, and
  • wherever possible avoid a ‘big bang’ approach – iterative and incremental developments mitigate the risk of catastrophic failure.

The agile manifesto certainly highlighted these important concepts but it did not invent them. These elements of fundamental common sense are ignored in far too many situations. What the agile manifesto and the subsequent changes in attitude have done is refocus on the importance of people and relationships in any project.

On the ‘Agile front’, many of the ridiculous excesses promoted by consultants and experts are certainly fading into obscurity. Executives are learning that ‘agile’ is not a cure all ‘silver bullet’ it needs pragmatic management and proper planning the same as everything else, it just the way planning and managing is done that differs; for more on this see: http://www.mosaicprojects.com.au/PDF_Papers/P109_Thoughts_on_Agile.pdf

Certainly there has been a realisation that the agile anarchist’s concept of ‘trust us’ (and their abandonment of any pretence of strategic planning and documentation) really does not work. An appropriate degree of planning, coordination and documentation are essential to achieve success, particularly on larger projects and in the longer term when the inevitable updates and maintenance cut in.

In summary, if ‘agile’ is a philosophy that prioritises people over rigid process, and it will change and adapt over time; it’s not ‘dead’ but it is evolving into a pragmatic management process. Certainly some of the narrowly defined concepts and methodologies branded as ‘agile’ are failing and being abandoned as ‘passing fads’ and new adaptations are emerging, but that’s normal. The core underpinnings of the original Agile Manifesto are still alive and well.

___________________

[1] In the 1950’s Peter Drucker identified the need for a new way of managing ‘knowledge work’, see: http://www.mosaicprojects.com.au/PDF_Papers/P070_A_Simple_View_of_Complexity.pdf

[2] “All models are wrong, but some are useful” (Prof. George E.P. Box), and every estimate used in the plan is wrong to a greater or lesser degree, see: http://www.mosaicprojects.com.au/WhitePapers/WP1051_Cost_Estimating.pdf

[3] For more on ‘rolling wave’ planning see: http://www.mosaicprojects.com.au/WhitePapers/WP1060_Rolling_Wave.pdf

1 Comment

Posted in Agile Ideas

Tagged , , , ,