There seems to be a lot of misunderstanding around the intention and use of the DCMA 14 Point Schedule Assessment. Following on from several discussions over the last month or so we have updated our White Paper: DCMA 14-Point Assessment Metrics and uploaded the last published version of the Metrics: Earned Value Management System (EVMS) Program Analysis Pamphlet (PAP), DCMA-EA PAM 200.1, October 2012.
The EVMS-PAP is designed for use in performing an integrated baseline review of a major program using EVM, but EVM relies on a competent schedule and Section 4 of DCMA-EA PAM 200.1 defines the last published version of the DCMA 14 Point Schedule Metrics. As can be seen from the date of publication, the DCMA 14 Points are quite old, and they did change in the years before 2012 (for more on the evolution of the DCMA 14 Points see: The evolution of the DCMA 14 Point Schedule Assessment). This leads to two significant problems:
The first is many people misunderstand the objective of the assessment. The objective is stated explicitly in the document:
The DCMA 14 Point Schedule Metrics were developed to identify potential problem areas with a contractor’s IMS…… These metrics provide the analyst with a framework for asking educated questions and performing follow-up research. The identification of a “red” metric is not in and of itself synonymous with failure but rather an indicator or a catalyst to dig deeper in the analysis for understanding the reason for the situation. Consequently, correction of that metric is not necessarily required, but it should be understood.
Earlier versions talked about pass/fail, this concept has been dropped (and was never a good idea).
The second issue is implementation of the assessment. The implementation of the DCMA 14-Point Assessment in the various software tools is not certified by the DCMA or any other body and varies between the tools! The biggest issue is around counting of the number of tasks to be considered. The 2012 version stated that the Total Tasks should exclude: Completed tasks, LOE tasks, Subprojects (called Summary tasks in MS Project), and Milestones (Zero Duration Tasks). This differs from the 2009 update, and the 2009 update changed from earlier versions.
There is an established correlation between a competently prepared schedule and project success – successful projects tend to have an effective controls system and a ‘good’ schedule, but the key measure of a good schedule is it is useful and is used. The purpose of the DCMA checks is to identify issues that need to be understood.
For more on schedule quality see: https://mosaicprojects.com.au/PMKI-SCH-020.php#Overview
Pat Weaver
Lynda Bourne
Aavssitedev's Blog
Have you benchmarked the GAO’s “Best Practices in Scheduling” https://www.gao.gov/assets/gao-16-89g.pdf (Appendices II and VI) against the DCMA’a 14 points?
And where is your proof to back up your claim that “there is an established correlation between a competently prepared schedule and project success”?
The IPMA/KPMG research from 2019 and 2020 certainly doesn’t support any claims for correlation, much less causation.
Try reading the links instead of asking rhetorical questions.
I have read the links (many times!!), and based on published research by Wayne Abba https://www.dau.edu/library/defense-atl/blog/Reboot-Earned-Value-Management and Chad Reynolds https://www.dau.edu/library/defense-atl/blog/PlanningfortheFutureofEarnedValueManagement the DAU has finally admitted that Earned Value, as being advocated by the US Government (and PMI and AACE et al.) has not worked in the past, is not working now and they are finally admitting EVM has to be changed for the future.
For 50+ years, we (and our clients) have been successfully using EVM not as the DAU advocates it but as it evolved on the factory floors of the 18th Century Industrial Revolution and as a “payment for performance” or “incentive payment system” https://pmworldlibrary.net/wp-content/uploads/2022/09/pmwj121-Sep2022-Giammalvo-origins-and-history-of-evm-a-contractors-perspective.pdf and under that system, the only VALID project schedule technique is what we in construction call “Rolling Wave Planning,” and our IT colleagues call “SCRUMS” or “SPRINTS.”
The “fatal flaw” in creating IMS can be found in these quotes from Helmuth von Moltke-“no plan survives first contact with the enemy”; Gen Eisenhower, who told us, “Plans are useless, but planning is essential,” and Gen. Omar Bradley’s quote- “AMATEURS study STRATEGY while PROFESSIONALS study LOGISTICS.”
The problem is Paul, the DCMA 14 point checklist is for evaluating CPM schedules – IT HAS NOTHING TO DO WITH EVM. You continue to show you lack of comprehension.
Pat, thanks for the post. There is a common misunderstanding of what the DCMA 14 points assessment is for, and both points, you have raised are valid and important.
Many posts present DCMA-14 points as a GENERIC schedule QUALITY assessment, while 3 metrics (11, 13 & 14) are purely about project performance.
It is barely applicable outside the Defence Contract Management Agency portfolio. This portfolio has a huge number of contractual projects, and they need a SIMPLE set of metrics to identify which projects require deeper analysis.
Also, the recent ’14 points check’ addon to the desktop version of Primavera creates the illusion that this tool now supports the DCMA assessment, and it is not. Even well-recognised P6 gurus rushed to announce that Primavera supports the assessment only because the total number of assessments is the same and some checks have similar names.
Another point I want to make is that assessment can’t guarantee that the GREEN schedule (based on the assessment) is reliable and doesn’t have a critical quality issues. With the volume of projects, they can’t “catch” all problematic schedules, but the health assessment is sufficient to catch many of them. I would say, it is probabaly practical to apply the ‘80/20’ rule for that type of portfolio and maybe absolutely not applicable to other portfolios.
It would be great to have a white paper to review the gaps. Happy to participate.
Regards,
Alex Lyaschenko
Hi Alex, I would differentiate between technical issues (assessed by the DCMA and similar approaches) and quality as in ‘fit for purpose’. .
Pingback: The evolution of the DCMA 14 Point Schedule Assessment | Mosaicproject's Blog
Pat and Alex,
Keep in mind that under the requirements from the DAU and DCMA, their primary focus is on AUDITING- designed to “catch” contractors if they have or are in the process of cheating the US Government. (See Abba, Wayne (2023) “It’s Time to “Reboot” Earned Value Management” Defense Acquisition University Blog
https://www.dau.edu/library/defense-atl/blog/Reboot-Earned-Value-Management and Reynolds, Chad (2023)
https://www.dau.edu/library/defenseatl/blog/PlanningfortheFutureofEarnedValueManagement
When I was actively involved with the GAO as an “independent consultant” when compiling BOTH the Cost and Scheduling “best tested and PROVEN” practices, those of us contributing from the roles of CONTRACTORS are not looking at the schedule for AUDITING but as FORECASTING- as a RESOURCE ALLOCATION/MANAGEMENT TOOL and to serve as DEFENSIVE (or OFFENSIVE) purpose to protect us against claims and help to perfect or substantiate our claims against the OWNERS DELAYS and CHANGE ORDERS. VERY DIFFERENT from how the NDIA and private sector contractors need or use the CPM schedule. https://pmworldlibrary.net/wp-content/uploads/2023/09/pmwj133-Sep2023-Giammalvo-futility-of-master-plans-prepared-with-little-or-no-hands-on-experience.pdf
And IF the CPM schedule has NOTHING to do with EVM, then why do they bother with SV or SPI? Or the false and misleading claims by PMI and Walt Lipske with their claims about “Earned Schedule”? Wayne Abba and I have agreed since PMI and Lipske made that claim both Wayne and I agreed that “Earned Time” and “Earned Schedule” have always dated back to the ORIGINS of Earned Value, from the Industrial Revolution, which we still see in use today in shoes and clothing (and nearly all other factories, as well as “job shops” like Doctors, Dentists, Lawyers, Auto mechanics, Barbers, Hair Dressers, Engineers and anyone else who offers their services on a “walk-in” basis that is sold on a “Unit Completed” basis) in the form of what is known as “piecework.” https://pmworldlibrary.net/wp-content/uploads/2022/09/pmwj121-Sep2022-Giammalvo-origins-and-history-of-evm-a-contractors-perspective.pdf
PS, FWIW, in a lengthy debate with Vladimir Liberzon (VP of Spider Software) on Linked In a couple of years back, he finally admitted that he does the same thing we do, which is to use Rolling Wave Planning (Sprints or Scrums by our IT colleagues) and he does exactly what we’ve been advocating for well over 50 years, which we have been treating all projects as being RESOURCE CONSTRAINED and well over 80% of the activities in our CPM schedules utilize PREFERENTIAL or “Soft” logic. Why? Because no project EVER has exactly the optimum size or exactly the optimum number of equipment and as most contractors utilize a mix of their own people and subcontractors, very rarely can you achieve the OPTIMUM mix of trades (by ratios or by numbers) And as we were a Union Contractor, if we needed more tradespeople, all we had to do is call the hall and if they were short, they would call the adjacent locals. (We did a lot of offshore oil platforms and because platforms are considered ships and you can only put so many people on the ship at the same time, you cannot begin to imagine how we have to jump through the hoops to “balance” the composition of the different trades, often on a shift-by-shift adjustments.)
Bottom line. There are precious few planners/schedulers who have that depth of field experience, which explains the TRUTH about why so many projects run LATE and/or OVER BUDGET. https://pmworldlibrary.net/wp-content/uploads/2023/09/pmwj133-Sep2023-Giammalvo-futility-of-master-plans-prepared-with-little-or-no-hands-on-experience.pdf