Process Appraisal or Benchmarking?

Search:   

by: Charles Symons

Clients often ask us, if their choice is between CMM® based appraisal or external benchmarking, which do we recommend to improve their software development and maintenance performance?

The short answer is that “the two approaches are complementary”.

Grant Rule chaired a panel on this subject at the SEPG 2000 Conference in Amsterdam recently. Ken Dymond (SEIsm authorised Lead Assessor) gave the case for CMM and Bram Meyerson (QPeP Director of QuantiMetrics) gave the case for benchmarking.

CMM® based appraisals compare the maturity of software development organisations' processes against industry best practice. To ensure good buy-in, appraisals involve everyone from top management to the most junior team member. The results give clear guidance on where and how to make improvements. CMM® makes organisations value the transition from individual expertise (heroes) to team expertise and, ultimately, a learning organisation.

Worldwide experience with CMM® shows it is possible to significantly reduce costs, cycle times and staff turnover, while improving predictability and quality.

CMM® was started to address problems in the US defence industry.

Appraisal results often highlight shortcomings in the use of size measures to estimate and track work output.

In a way, you can think of the CMM® as a 'bottom up' approach.

QPeP benchmarks give quantitative comparisons of performance. A benchmark study enables an organisation to compare itself and its projects against other similar organisations and projects. It not only highlights strengths and weaknesses, but also how performance varies with factors such as application size, technology level etc. It shows how effort, time and quality can be traded.

Quantitative benchmarking, of which the PEP service is one of the longest established, started in business software. It is based on the availability in that domain of Function Point methods for measuring work output.

The results of a QPeP benchmark, however, are of great interest to top management. They give immediate data about the value for money that the software organisation is delivering, with pointers to quick actions that can be taken to improve performance by changing the way work is allocated and managed.

External benchmarking is a 'top down' approach to performance improvement.

 A good analogy for the two approaches is to look at how engineers improve the performance of a car. To build a high performance car, you need specialist design engineers. But, you would never expect a new car to be launched without first putting it through extensive road tests.

You would expect the results not only to improve the car but also the process for designing the next model. Similar reasoning underlies process appraisal and benchmarking in software development.

If management is already convinced, start with a CMM® appraisal. If you need to build management commitment, start with a QPeP benchmark.

The best way to drive and quantify performance improvement is to alternate external benchmarks and CMM® appraisals. The business environment is ever changing therefore continue alternating to understand both what needs improving next and the efficacy of the actions taken.

Related Information

back to top

Articles

Managing Supply Chains
If all stakeholders applied the rules of the CMMI® Acquisition PAs to all external interactions, just imagine how smoothly communications would run! Misunderstandings, blame, recriminations and buck-passing could become things of the past.

Does Your Selected Supplier Have the Capability to Deliver?
Now outsourcing is regarded as a preferred method of acquiring a business system or even an organisation's entire portfolio, the CMM® is increasingly applied for its original purpose, as a means of supplier capability evaluation and monitoring.

Maturity Brings a Different View
of the hierarchy in higher maturity organisations.

Services

Assessing Capability

Appraisal
Assessments of capability to aid internal process improvement.

Audit
Audits for organisations (internal teams, projects and external suppliers) based around the SEI Capability Maturity Models (CMM®), the SPICE Model and the ISO 9001/2000 standards.

Evaluation
Supplier capabillity evaluations and software process evaluations. Usefull for selecting internal or external suppliers and ensuring appropriate capabilities are maintained once selected.

Improving Processes

Process Consultancy
To develop, improve and support processes against industry standards and best practice.

Starting an Improvement Programme
Services to plan and run the irst improvement cycle(s) of an improvement programme.

Maintaining an Improvement Programme
Services to keep an improvement programme working once it has completed the first cycle.

Measuring Performance

Benchmarking
An accepted technique used to calculate and improve organisational performance with respect to appropriate benchmarks.

Performance Measurement and Analysis
A range of services to help organisations determine what measures, data collection and analysis techniques are appropriate.

Sourcing

Contract Management
A set of processes for management of the work subcontracted to those suppliers, to ensure compliance and ameliorate the issues and risks involved.

Planning and Supplier Selection
A reliable process for identfying a suitable supplier or suppliers for given packages of work. This also identifies issues or risks to the work that may be a consequence of using each supplier.

Training

Assessing Capability

Using and Implementing the CMMI® / Intro to the CMMI® SE/SW/IPPD/SS  Workshop Learning Break
How to use the model for appraising and improving an organisation's processes

Improving Processes

Overview of the CMMI®  Overview
Understand the relevance of the most popular model for process improvement and how it can benefit you and your organisation

Senior Managers Briefing in the CMMI®  Overview
Sponsorship is an essential factor for improving performance – understand the manager’s crucial role & the actions needed to generate & sustain commitment 

Sourcing

Supplier Capability Evaluation – an Overview  Overview
Understand the proven, CMMI-based approach used by world-class organisations to evaluate & select suppliers to minimise risk

Software Measurement Services Ltd.
124 High Street, 
Edenbridge, 
Kent, 
TN8 5AY 
United Kingdom  
  tel +44 (0) 1732 863 760
  fax +44 (0) 1732 864 996
 e-mail: sales@measuresw.com
  www.measuresw.com

© Copyright 2000-2004 Software Measurement Services Ltd. All rights reserved.

CMMI®, CMM® and Capability Maturity Model® are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University

                                               
Applying Software Metrics
Assessing Capability     
Estimating and Risk       
Improving Processes     
Measuring Performance
Sourcing                       
Tools and Techniques   
             
                
Services               
Training                
Events                  
Reference             
                
About SMS         
Opportunities
Copyright & Legal