Your supplier scorecard says 94 percent and the line stopped three times
Nov 19, 2026, 9 min read

A supplier scorecard answers a question that has four legitimate on time definitions, and it prints one of them without recording which. Average days late is the wrong statistic, because a supplier averaging a third of a day early can be the one that stopped your line three times. Split the receipts by whether the order was placed inside the quoted lead time before taking any score to a supplier, since part of the lateness is yours. Set the review cadence by event count rather than by the calendar, so that a two point move carries information rather than sampling noise.
The quarterly supplier review has two suppliers on the same page. One is at 94 percent on time and caused three line stops. The other is at 88 percent and has never caused one. The scorecard ranks the first above the second, the buyer knows the ranking is wrong, and there is nothing in the document that lets them say so without sounding like they are defending a favourite.
The scorecard is answering a question that has four legitimate answers, and it printed one of them without saying which.
One quarter of receipts, four on-time numbers
Take 240 receipts from one supplier across a quarter, with the deviation from the original promised date recorded for each. Sixty arrived three days early, 55 one day early, 62 on the day, 30 one day late, 18 two days late, 9 four days late and 6 nine days late.
Score that on time.
On or before the original promise gives 177 of 240, which is 73.8 percent.
Within a three day window either side of the promise gives 225 of 240, which is 93.8 percent.
On or before the latest confirmed date, where the supplier re-confirmed 35 of the late receipts, gives 212 of 240, which is 88.3 percent.
By value against the original promise, the arithmetic goes differently, because the late receipts were the big ones. The six nine-day receipts averaged 110,000 for 660,000. The nine four-day receipts averaged 55,000 for 495,000. The eighteen two-day receipts averaged 28,000 for 504,000, and the thirty one-day receipts averaged 21,000 for 630,000. That is 2.289 million late out of 4.8 million received, so 52.3 percent of the value arrived on time.
Same supplier, same quarter, same 240 rows in the receiving system. Numbers of 73.8, 93.8, 88.3 and 52.3, and every one of them defensible. The 93.8 is the one that appears on the scorecard, because a three day tolerance is what the template shipped with and nobody ever reopened it.
The mean is the wrong statistic
Average the lateness of those 240 receipts and you get minus 0.33 days. On average this supplier delivers a third of a day early. Any scorecard built on average days late will rate them excellent, and the rating will be arithmetically correct and operationally useless, because nothing in a supply chain is sized on an average lead time.
The standard deviation of that distribution is 2.33 days. The 95th percentile of lateness is 4 days and the 99th is 9. Those three numbers together describe the supplier; the mean on its own describes nothing.
The consequence is a buffer you are already carrying. With 400 units a day of demand, a lead time standard deviation of 2.33 days contributes 400 times 2.33, or 932 units, to the standard deviation of demand over lead time. With a daily demand standard deviation of 90 over a 21 day lead time, the demand side contributes 90 times the square root of 21, or 412 units. Combine them and you get the square root of 932 squared plus 412 squared, which is 1,019 units, and a 95 percent buffer of 1.65 times that, or 1,681 units.
Now suppose the supplier's delivery standard deviation were half a day instead of 2.33. The lead time term becomes 200, the combined figure becomes 458, and the buffer becomes 756 units. The difference, 925 units at 18 apiece, is 16,650 of stock you hold because of a tail that the 93.8 percent score does not mention. The full treatment of lead time as a distribution belongs to I3; the point for the scorecard is that the number you report has to carry the variance, or the buffer conversation never connects to the supplier conversation.
Weights, and the ranking they decide
Assume you have solved the definitional problem and have clean sub-scores. Supplier X scores 74 on delivery, 96 on quality, 88 on cost and 70 on responsiveness. Supplier Y scores 92, 82, 79 and 90.
Weight delivery 40, quality 30, cost 20 and responsiveness 10, which is roughly what most templates carry. X gets 29.6 plus 28.8 plus 17.6 plus 7.0, or 83.0. Y gets 36.8 plus 24.6 plus 15.8 plus 9.0, or 86.2. Y wins.
Weight delivery 20, quality 45, cost 25 and responsiveness 10. X gets 14.8 plus 43.2 plus 22.0 plus 7.0, or 87.0. Y gets 18.4 plus 36.9 plus 19.75 plus 9.0, or 84.05. X wins.
Same eight raw scores, opposite conclusions, and the weights came from a template.
There is a way to derive them. Price the failures. Over the quarter, late deliveries from this category triggered 22 expedites at around 1,800 each, which is 39,600, plus the carrying cost of the buffer computed above, call it 40,100 in total. Four quality escapes ran through containment, rework and customer notification at an average 46,000, which is 184,000. Price variance against the benchmark cost 61,000. Slow responses to engineering changes delayed two launches at an estimated 18,000. The total is 303,100, and the shares are delivery 13 percent, quality 61, cost 20 and responsiveness 6.
Apply those. X scores 90.0 and Y scores 83.2, and this time you can say why. The exercise also tells you something the template never would: in this category, quality failure costs about four and a half times what delivery failure costs, and the scorecard has been weighting delivery highest for six years.
Quality and responsiveness have the same problem
Delivery gets the attention because it is the easiest to compute, and the other three dimensions carry identical definitional ambiguity that nobody interrogates.
Take 14 rejects across the quarter from the same supplier. As a share of receipts, 14 of 240 is 5.8 percent. As parts per million against 480,000 units received, it is 29. As a share of distinct part numbers affected, 14 rejects landing on 14 of the 96 parts you buy from them is 14.6 percent. Three quality figures from one set of events, and a supplier shipping a few very large lots will look excellent on the parts per million view and poor on the receipt view, while a supplier shipping many small lots will look the opposite.
Then there is what counts as a reject. Material rejected at goods-in, material rejected at the line, material that reached a customer, and material accepted under concession all get treated differently by different systems, and a concession is a defect the business chose to absorb rather than a defect that did not happen. Count concessions separately and report them; a rising concession count with a flat reject rate is a supplier whose quality is deteriorating inside a metric that cannot see it.
Responsiveness is worse, because the clock start is almost never defined. Time from your email to their first reply, time to a substantive answer, and time to resolution are three different measures, and only the third correlates with anything you care about. Pick one, write down the two events that bound it, and accept that a supplier whose first reply is instant and whose resolution takes three weeks should score badly.
The part of the score you caused
Before you take any of this to a supplier, split the receipts by whether the order was placed inside the quoted lead time.
Of the 240 purchase orders, 58 went out with less than the quoted 21 days. Their on time rate against original promise was 51 percent, or 30 receipts. The other 182 ran at 81 percent, or 147 receipts. Together that is the 177 from earlier.
So the supplier's controllable performance is 81 percent, and roughly seven points of the reported 73.8 belong to your own planning. Taking a 73.8 to a supplier review and asking them to explain it is a good way to spend an hour being told about your own release behaviour, and a good way to lose the credibility you need for the conversation about the nine day tail.
Report the split every time: performance on orders placed with full lead time, performance on orders placed short, and the count of short orders. The third number is a measure of your planning, and it belongs on your own scorecard rather than on theirs.
What the score invites a supplier to do
Every definition above has a cheap response, and suppliers are at least as good at reading a scorecard as you are.
- Scoring against the latest confirmed date rewards revising the date. Track the revision count alongside the score. This supplier revised the promise on 47 of 240 lines, and 35 of those revisions came after the original date had already passed, which is a rescheduling notice dressed as a commitment.
- A three day tolerance window moves the target to the edge of the window. Watch the mode of the distribution drift from day zero to day plus three over a few quarters, and note that the buffer you carry is set by the edge rather than by the centre.
- Counting by line rewards shipping the small easy lines on time and batching the awkward ones. The value weighted view above is what catches it.
- Averaging makes the tail free. Report the standard deviation and the 95th percentile next to the mean, always.
The pairing that closes most of the remaining gaps is a consequence measure: the count of line stops, production reschedules or customer backorders (BB18) attributable to each supplier. It is harder to collect and it is the only number in the pack that says whether any of the rest mattered.
Review cadence and the arithmetic of small samples
Cadence should follow event count rather than the calendar. With 240 receipts a quarter, a monthly review has 80 events behind it and one late receipt moves the score by 1.25 points, so a two point move means something.
With 8 receipts a quarter, one late receipt moves the score by 12.5 points. A supplier scored quarterly on eight receipts will oscillate between 100 and 75 percent with no change whatever in their behaviour, and the review will spend its time explaining noise. For those suppliers, review annually and run an exception trigger in between: any receipt more than a set number of days late, any quality escape, any unrequested date revision. The trigger produces a conversation when there is something to discuss.
The other constraint on cadence is your own ability to act. Reviewing a supplier monthly when qualifying an alternative takes nine months gives you eight reviews with no lever in them. Match the formal review to the decision horizon and use the exception trigger for everything faster.
Where this stops
A scorecard measures the receipts you got, which is your tier one supplier's behaviour and nothing beyond it. A supplier scoring 96 across the board can be resting on a single sub-tier source that also supplies two of your other suppliers, and the score will stay at 96 right up until that source has a fire; that exposure is R1's subject and no amount of scorecard refinement finds it.
Scores also compare badly across suppliers. A supplier shipping 240 lines of commodity packaging and one shipping 8 lines of custom tooling get scored on the same template, and the resulting ranking mostly measures product complexity. Segment first, compare within segment, and accept that some suppliers need a written assessment rather than a number.
There is a data problem underneath all of it. Where the same supplier exists three times in the vendor master under slightly different names, you are scoring three ledgers and none of them describes the relationship; T3 covers the resolution work, and it is worth doing before the weighting argument rather than after.
Pull one supplier's last 200 receipts, plot the days late as a histogram, and see whether the mean you have been reporting describes the shape you get.
More on this in metrics on the DemandLabs platform.




