Ask for information you can use

Folowing on from my earlier post ‘A though on PMOs and Project Controls’ , PMOs and projects regularly collect and report on data but you have to ask how often is the data converted into information the organisation can use?

I have just wasted 15 minutes trying to respond to a survey commissioned by our local city council. The structure of the survey was so bad there is no way the data collected would allow any useful information to be gathered. Two of the questions (from memory) were:

• ‘Overall how do you rate the Port Phillip City Council’s performance’ with a range of 5 options from bad to excellent. Sound like an insightful question but what does the answer tell anyone?? The Council delivers a wide range of totally separate services from refuse collection to maternal and child welfare. Any overall answer is meaningless; unless the Councillors are spending rate payer’s money on a ‘feel good’ ego trip and hope they will be rated ‘good’. It does not matter if the answer is bad, good or indifferent unless the data collected identifies what is good or bad so management action can be taken to lock in the good and improve the bad.
• “How do you rate the street cleaning service overall?”  More specific but still useless data. The council cleans shopping strips on a daily basis, other roads on an occasional basis and numerous lanes and footpaths very rarely. My answer was shopping strips great, major roads OK, side roads like the one I live in fairly poor and the laneways are a total disaster….. But how would I rate the service overall??? At this point I terminated the interview!

  Overall rating should be complied from specific measurements weighted appropriately. In this case a weighted assessment of the relative importance of clean shopping strips -v- clean local streets -v- clean laneways to the local citizens. How do you assess the weightings? Ask a sample group of people which they feel is more important on a comparative basis before running the survey. Then collect specific data and build some useful information that can help direct improvements in the overall service.

When designing this sort of data gathering, you also need to filter out influences like staff culture (the Port Phillip staff are really great and helpful) from systemic issues such as the contract conditions the street cleaning contractor operates under.

Then by compiling all of the various rating for the specific services, an overall rating for the council could be compiled and more importantly the high performing areas noted and lessons from these areas transferred to the less well performing areas.

There’s a lot to learn from this example of bad surveying. Designing surveys and collecting data is not a trivial exercise. There is nothing simpler than bogging a PMO down collecting masses of data that can never be converted into useful information. Do this and the PMO will be seen as a useless bureaucracy and sooner or later it will be reorganised out of existence.

Focus the data collection on a limited range of key factors that can provide useful management information and the PMO will be seen as a real value add to the business.

So how do you rate your PMO overall?     Only kidding…..

On a closing note –
The number of really bad surveys seem to be increasing exponentially – I think around 80% of the various project management surveys I look at, mostly from post graduate students, seem to be either designed to support a pre-determined answer or so badly designed the data could be interpreted to suit any answer the researcher chooses. There is a real discipline and skill in developing an effective survey; unfortunately it’s a skill that seems to be in very short supply.