benchmarking

Refine benchmarking to deliver useful results

15 Feb 2005 | (Survey)

Most companies are getting poor value from the tens of millions of pounds spent on benchmarking the performance of their IT, according to a new survey.

The survey, by research company OTR Group, questioned 300 users, outsourcing suppliers and benchmarking companies across Europe and found that benchmarking internal IT services was not an effective way to measure the value of IT investment.

Respondents also said that benchmarking internal IT did not give them practical advice on how to improve their service.

Benchmarking has become a standard way for UK organisations to measure the effectiveness of their IT department or outsourcing suppliers.

Benchmarking judges a particular service based on criteria including its costs, daily performance and a comparison with other IT departments or outsourcing suppliers.

"The survey found widespread dissatisfaction with benchmarking," said Andrew Pooley, group commercial director at research company OTR.

"Benchmarking can provide value for money if the right things are measured, but many users need to review what they are doing."

The survey found that benchmarking worked well when used on certain types of deal. Benchmarking of "open book" outsourcing agreements, in which users have access to their suppliers' accounts, was judged particularly effective.

Organisations benchmarking these types of outsourcing agreements saved an average of 30% on IT costs through negotiating more favourable terms and conditions with suppliers.

'Lack of insight'

Scottish & Newcastle used a benchmarking consultancy to measure its IT costs and look for best practices before beginning a large project. The brewing and leisure group had previously used its own staff to benchmark IT services.

Scottish & Newcastle said benchmarking helped identify which supplier would give the best service at the lowest cost.

But it was less happy with other parts of the benchmarking service. It said that the bench-marking service did not "deliver any insight" into whether their IT plans were along the right lines, or help it identify best practices to apply to its IT department.

Councils prove value for money        

In the public sector benchmarking has become a useful tool for councils which have to show inspectors their services are providing value for money.  According to Socitm, the association of local authority IT managers, more than 100 UK local authorities currently use its benchmarking service. 

The benchmarking service measures in-house and outsourced services on performance indicators, such as cost and reliability. Council performance is compared to similar organisations using information held on a Socitm database. 

Martin Greenwood, Socitm Insight manager, said, "Benchmarking can work well when used to judge the effectiveness of an IT department. The important thing is to use the right measures, for example, user satisfaction."

Related Content: