Procurement: What have you done for me lately?

1 Using indexing to measure performance.

One of the more interesting phenomenon in business is the day you realize your boss and everybody else’s boss majored in the same course at university – the course they all completed was ‘What the heck have you done for me lately’.

This question is often directed at supply chain professionals as a means of justifying why have them in the organization. It is mission critical for procurement to demonstrate annualized savings. Like it or not – it comes with the territory. In the bigger picture, it is difficult to significantly affect the bottom line contribution to profits as an aggregate value. Incremental changes in costs go largely unnoticed. Volumes change in accordance with demand. Radical changes to manufacturing methods or outsourcing have a more dramatic and measurable effect. In spite of this, it is important for procurement to track costs and savings and to report out in an objective manner.

Human behaviour says that being objective in reporting goes against our self-serving interests. Therefore, savings or cost reductions are often reported out of context to influence the perception of the reader to assess procurement’s performance in a more favourable light.

The following article examines an objective tracking and reporting tool on the ability to measure price or cost changes in a generic model. It can, and is being applied, to public sector and private organizations as another tool to assess performance and monitor trends. The model provides a means of evaluating strategic and tactical business practices by comparing in-house metrics with external cost drivers and indices.

The purpose is to develop an in-house price index and then benchmark against other known indices and published business metric references and within this context, objectively compare the results over similar time periods.

If we consider how the Consumer Price Index (CPI) provides information, we get a fundamental understanding of how an in-house weighted-price index works. Interestingly, the CPI value is widely accepted as a reasonable indicator of price changes in many industrial and commercial contracts. Sellers like to draw on the CPI to revise prices because it generally always goes up in value.

By developing a representative basket of goods for the frequently purchased items by an organization, we can create an index to track our ability to manage our costs; assess the effectiveness of our contract management; separate inflationary factors out of pricing factors; forecast budgeted costs more accurately; measure a component of negotiating skills; assess the influence on the profit-leverage effect; and identify cost avoidance benefits.

With a minimal application of statistical theory, we can capture many of these issues rather easily. With the assistance of readily available software such as Excel, this data can be massaged to report on many aspects of cost/price management.

The author’s research was based on originally developing a weighted-price index to compare buying strategies in health care. We will look at two cases which demonstrate how the price index model can be applied.

Case 1: Hospital medical/surgical supply agreement.

It was a year after a B.C.-based health region of four acute care facilities had joined a Canadian, national not-for-profit buying group, and the question was raised as to how cost effective this strategy had proven to be. Could they measure any incremental savings to support their strategic shift from participating in a local, geographical buying group of hospitals to a nationally managed consortium of one hundred twenty hospitals – and still continue to buy similar products from similar suppliers? To reiterate, one of the benefits of the shift to a national program was the ability to leverage the buying power which allowed the B.C. hospital group to minimize supplier and product changes which are a sensitive issue with medical professionals and practitioners.

Methodology: A price index uses a reasonably simple method of collecting and analyzing data to measure incremental changes to prices and then uses this as a basis to create an index.

To create the index for the B.C. hospital group, staff compiled a list of 100+ medical supplies used on a frequent basis. The supplies were characterized as being packaged medical/surgical supplies that comprise a significant portion of the annual operating costs. There is an expectation that the goods would be required on a continuous basis. Items which are purchased once or twice per year do not lend themselves to tracking via an index.

Using the actual costs from the year 2000 in the database for the items, they factored in the actual volume of usage for 2000 for the basket of items, following Pareto criteria. The actual volume from 2000 gives the index its “weight”. By multiplying the unit price by the volume, by each line item, the price of a basket of goods in 2000 was determined to be $1,102,655.00. This created the index value of 1.00 for 2000. The index value is an aggregate value.

It should be noted that items can be added or deleted over time. The index value relationships will still be relevant. The author cautions against changing the items too often. The initial items which go into the basket of goods should be selected with reasonable care. The data should be clear as to whether it is a tax in or out value or whether it is a landed cost or contract pricing – it matters that the data referenced be consistent on a year-over-year basis.

By referencing the same items in the second fiscal quarter (2Q) of 2001 and by using the volume from 2000, staff extended the prices for each item again for comparison. Depending upon the contract terms and other factors, some prices increased, some decreased, and some remained the same. The total cost for the same basket of goods for 2Q 2001 was $1,047,742.00. This was $55,000.00 (net) or ~4.9% less as an aggregate value and led to an index of .951 for 2Q 2001.

An in-depth review of the items was quite revealing. Many of the same products being purchased in 2001 had moved to a new supply contract through the national buying group. Of these items, they measured a 7.61% decrease off-invoice price where they participated on national contracts and only 1.77% on the other products. The latter were mainly localized agreements that the B.C. hospital group continued to purchase through their previous buying practices, as not all items were yet available on a national contract basis. The strategy to participate in the national not-for-profit buying group indicated a favourable decision when using price as the definitive factor.

The analysis and observation showed the new contracting strategy managed through the national buying group was effective; the contract management efforts were in line with expectations; the hospitals were not experiencing the amount of inflation they had anticipated; there was better information to predict 2002 costs; the hospital buying staff were better utilized by working on localized product, equipment, and service requirements; the costs to meet servicing needs were lower; and they have avoided many price increases during this period.

Later in 2001, staff compiled a 3Q index at .958 which was a slight increase over 2Q, but was still less than their base 2000 prices. Without developing the index, procurement would have been relying on anecdotal or an intuitive sense of their effectiveness to manage costs. When asked to prove their ability, due to the bias in human nature, they would likely have selected items where they knew there were savings and reported out on those findings in a less objective manner.

The weighted-price index is more meaningful over time as trend lines become stronger. The hospital management can also drill down into segmented product groupings or apply this model to other commodities such as MOR (maintenance, operating, and repairs) items. It is now relatively easy to track their ability to manage costs. In addition to the indexing data, they should capture the key drivers of costs such as energy costs, bank prime rates, exchange rates, fuel costs, base metal prices, or other metrics which influence the bottom line specific to health care.

Case 2: The City of Granston case 2. Written by Larry Berglund, SCMP, MBA and Collin Ashton, SCMP

In 2003, the author had the opportunity to apply the price index model to a large municipal operation and to add more objective criteria to the model.

Methodology: Similar to the basic data collection as in the hospital case. In this case, the basket goods were comprised mainly of industrial-based, manufactured products along with other packaged goods from a wide, cross-section of civic operations. However, external indices and cost driver information was gathered to benchmark the City index values to other external data for the purposes of being objective.

Learning outcomes and applications for the price index model:

  • Using the CPI as a reference on price reviews for contracts is questionable. The PPI may be more relevant or refer to the core CPI
  • Negotiation strategies should adjust for periods of inflation versus deflation
  • An index provides objective information to challenge prices in negotiations
  • Each business should develop their own weighted-price index to monitor trends and revise its strategies accordingly
  • Efficient suppliers are cost effective
  • Can be used to track maintenance, operating, and repair items Awareness helps staff to understand how their role affects costs
  • May lead to the use of target costing
  • Benchmarking with other organizations and indices will give a sense of effectiveness
  • Recalibrate the index in 5-year increments for relevance

Appendix 1

Sample of a Hospital’s Weighted-price Index




USAGE 00/01

2000 Price

2000 Total

Current Price

Sep 2001

Oct – Dec 2001 Pricing

Oct – Dec 2001 Totals


































































































2Q 2001


3Q 2001






Interpreting table 1:

In the table above, #3 indicates 21 units were used in 2000 at $27.25 per unit, for a total of $572.25. Those items would cost $399.00 in 2Q and $399.00 in 3Q. The index values are shown as .951 (1047742.35/1102655.470) and .958 (1056358.30/1102655.47), respectively. Weighting is the value realized by multiplying the annual usage by the unit price per fiscal period. The usage in the index model always remains the same as the first year, which allows the comparison on the affect of price changes over time.

1 Revised 2014. Originally published ~2005

2 The City of Granston case was first published in Leenders, Michiel R., P. Fraser Johnson, Anna E. Flynn, and Harold E. Fearon. Procurement and Supply Management, 13th edition. Homewood, Ill.: Richard D. Irwin Inc., 2005. Although the City of Granston is a fictitious name, it is based on actual data, as is the data in tables 1 and 2 and Exhibit 1.

Contract Management: Ensuring Effective Supplier Performance

3 Background

Frequent instances occur within contracts where disputes arise with respect to the quality of the materials or service performance. Invariably the relationship between the owner and the contractor deteriorates with the result being that the owner will not receive full value for money.

Execution of a contract does not give license to the owner to demand services that were not contracted, or for the supplier to supply inferior product or provide unsatisfactory service. Thus, it is essential that the RFP includes language that will facilitate management of the contract, the terms and performance metrics which are usually developed collaboratively between the owner and the successful proponent prior to execution of the contract.

To maintain a successful relationship, the key ingredient is for the two parties to set scheduled milestone meetings to review the performance to date with the purpose to identify any deficiencies and strive for mutual improvement. This implies that all parties can contribute to ongoing improvements.

Why are supplier evaluations important?

One of the weaknesses in contract management is where the owner does not have a structured process to identify the performance deficiencies in a timely manner. In the absence of a formal supplier evaluation process, the ability to measure value for money is forgone. If suppliers know their performance will be assessed, they tend to make a stronger commitment to meet expectations.

Supplier performance evaluations should be a shared responsibility between the end user and the procurement or designated contract staff. Verbal complaints or undocumented perceptions about poor performing suppliers won’t cut it and are not defensible when trying to avoid an award to a poor performing contractor. An evidence-based evaluation model indicates where poor performing suppliers can improve and good performing suppliers can be recognized as a matter of record which assists them in future contracting opportunities.

The lack of a formal evaluation process and a subsequent record on file, defaults in favour of the poor performer. The situation continues until such time as this service provider hits an unacceptable level or complete failure and remedial actions are necessary. Unfortunately this adds to the administration costs and does not address the core problems. Replicating successful contracting processes is an objective which leads to the adoption of best practices in contract management.

Supplier Evaluations

Supplier evaluations on contracts are related to the expected level of services as stated in the RFP or as set out in the contract, and the perceived level of services received by the organization during the performance of the contract. The perceived level of services equates to the actual assessment of the services after they have been performed. The expected level of services equates to the pre-contract evaluation and any subsequent meetings to clarify the services. The latter is analogous to the hiring of a prospective employee. The employee’s resume and references give the employer an expected level of competency and then the perceived value of that performance is assessed periodically through job performance evaluations.

The comparison between expected and perceived services can be summarized as follows 4:

  • Where Expected Services > Perceived services = Unsatisfactory evaluation
  • Where Expected Services = Perceived services = Satisfactory evaluation
  • Where Perceived Services > Expected services = High level of satisfaction

The reason for the discrepancies between the expected and the perceived level of services should be determined as a part of the continuous improvement process. Where a perceived service exceeds the expected level of service, this too is an important learning lesson resulting from a supplier evaluation. It is the relationship between the expected (desired) level of services and the perception of the actual services received which is what needs to be assessed in the supplier performance evaluation.

Not all evaluations should be conducted in the same manner; one size does not fit all. Based on the type of contract, the performance evaluations should address the nature of the goods or service and the inherent risks.

Types of Contracts

  • One-time services i.e. need to have a document translated;
  • Multi-year delivery of services i.e. need to have painting contractors available to meet ongoing service requests;
  • Multi-year outsourcing of services i.e. need to have IT support available to manage technology upgrades
  • Short-term project services i.e. need to have facilitation of training programs
  • One-time goods acquisition i.e. need to buy an electron microscope
  • Multi-year goods delivery i.e. need for office supplies and related equipment on an ongoing basis
  • Capital projects i.e. construction, infrastructure investments and upgrades i.e. often complex technologies or critical time lines for completion

Each type of contract implies different sets of criteria and outcome measurements and therefore different contract management tools or areas of focus. All contracts share the common component of the perceived level of satisfaction to be assessed at the end of the contract.

Although there are differences in the types of contracts, there are common best practices for managing contracts. Nevertheless, this does not presuppose every evaluation should encompass the same degree of detail. Rather, evaluations will differ given the variety of factors impacting procurement such as the circumstances under which the goods or services were acquired. This could include time frame, emergent situation, compatibility risks, project cost, or availability of alternatives in the market.

Service contracts may result from a successful competitive process or where justified, a direct award. One-time, short term contracts tend to be less problematic from a managerial perspective than multi-year agreements. However, all contracts require management of the terms and conditions specific to the nature of the goods or services to be provided, or unique to departmental requirements which the parties must comply with during the term of the contract.

Key responsibilities in successful contract management:

  • Compliance with the contract terms and conditions by all parties
  • Satisfactory performance by the contractor during the term of the contract
  • Value for money was received by the owner
  • A post-contract evaluation summary was conducted and stays in the file

The performance of the contractor during the term of the contract should be monitored for:

  • Identification of any problems or deficiencies as per the Statement Of Work
  • Remedial actions taken where necessary
  • Interpersonal skills and relationships of contractor resources
  • Failure to meet terms and conditions

Contractors should be advised during the contract as to how their performance is meeting expectations and not only after the fact. After the completion of the contract, an evaluation should be conducted as to how the contractor performed in meeting the expected outcomes and their relations with organization staff during the performance of their services.

Post-contract evaluations and reviews are conducted to:

  • Ensure the goods or services were provided as per the contract terms and conditions
  • Ensure the end user needs were met
  • Identify areas of improvement in procurement
  • Assess whether the contractor would be considered again for similar services in the future

The contract manager should follow up directly with end users and/or the RFP team lead to discuss the contract/project file, and engage with the supplier to identify areas for improvement. The post-contract evaluations are a commitment to continuous improvement which is a best practice. The performance evaluation forms a part of the contract file for future reference.

The learning outcomes derived from the evaluations provide an iterative process which contributes to the benefits of the owner and the supplier.

Does a supplier performance evaluation have a Return on Investment?

As advocates of supplier performance evaluations we share the following advice, comments, and recommendations:

  • A formal and comprehensive evaluation requires a wide variety of perspectives by affected stakeholders to be objective. The cost to conduct supplier evaluations during and post-contract is relatively low in comparison to the benefits being realized. (Many software programs enable online evaluations with automated results capturing quantifiable and subjective information, such as Survey Monkey, for less complex projects or contracts).
  • Suppliers should have a clear understanding at the outset of the contract as to how their performance will be evaluated. By having established mutually agreed to performance metrics, it allows the parties to focus on what is important and how well they are meeting expectations.
  • The results should be shared objectively with the specific supplier being evaluated. Where areas for improvement are noted, this can complement continuous quality strategies; and where a strong performance is assessed this too contributes to replicating this ongoing level of performance. Supplier improvements not only benefit one customer but have the added benefit of the multiplier effect where the resolutions can be applied to many customers. This is a great motivator for sales organizations.

Early detection and remediation of issues reduces the overall costs to all parties. This leads to improved performance and long-term competitiveness in the market. The parties have a stronger understanding of the expectations and commitments to the spirit and intent of the contract. The opportunity for leading suppliers to reference successful contracts with potential clients garners better evaluations in the future. The owner receives a quantifiable value for money with higher end user satisfaction. See ADDENDUM I

Supplier Reward Programs

Suppliers want to know their ranking determined through the performance evaluation metrics. The scores provide a percentage in relation to the overall points set out. Where a supplier’s perceived level of performance exceeds the expected level of performance, rewarding them with a “Preferred” or “Exceptional Supplier Status” certificate presented at an organizational event will provide a stronger motivation to maintain that ranking. This contributes to increased competition in the market to vie for this level of recognition.

The big payback to the owner is to be the ‘Customer of Choice”. In a recent provincial RFP the successful bidder immediately asked for a debriefing (debriefings are usually associated with unsuccessful bidders) since they knew that their proposal’s solutions were likely not ranked the highest in every category in the bid document. Therefore, prior to implementation, the supplier wanted to know where they could focus on improvement at the outset of the contract. This led to a strong working relationship between the parties where the eventual perceived levels of services were greater than the expected level of services.


Vendor/Contractor Performance Rating Guidelines

Rating Score Definition



Exceeds many of the objectives as per the statement of work to the department’s benefit; work completed early or on time; where necessary, any corrective actions taken were effective; deliverables including technical performance exceeded expectations

Very Good


Meets the objectives as per the statement of work and exceeds some to the department’s benefit; work was completed on time; corrective actions taken were effective; deliverables including technical performance were within expectations



Meets the minimum requirements as per the statement of work; most of the work was completed on time; corrective actions taken required departmental input



Failed to meet some of the requirements as per the statement of work; the current scope of work partially completed; and deliverables were partially met but full completion unlikely



Does not meet most contractual obligations; full completion unlikely; corrective actions were ineffective; the current scope of work incomplete; corrective actions failed to resolve related issues or were not implemented; deliverables were not met

Not Applicable


Based on specific conditions or criteria

The above guidelines provide a subjective means of assessing the performance. A final score should be reached by consensus and or averaged by the number of staff completing an evaluation to mitigate bias. The post-contract evaluation process should include end users, procurement, legal, finance or accounting staff or other affected stakeholders to assess their perception at the completion of a project. This forms a part of the contract record and can be used when evaluating future proposals.

3 Article written by: Larry Berglund, SCMP, MBA; Eric Lotz, SCMP

4 Parasuraman, Zeithami and Berry (1985)