The situation
The manufacturer was losing product and value across its production process, and knew it — because loss showed up as a line on a monthly report. What the report could not say was where. An aggregate loss figure tells you that you have a problem; it gives nobody a decision to make on Monday morning.
How it works
Attribute loss to the unit
Which part of the process gave it up — not one plant-level number.
Rank where to intervene
A ranked list of specific places to act.
Watch two drift signals
Data drift and model drift tracked separately; they fail differently.
Retrain on evidence
Retraining fires on a threshold, not on the calendar.
Why this was harder than it looksThe technical detail
There were two difficulties, and the second one is where most manufacturing ML projects actually die.
The first is attribution. Loss at plant level is an accounting fact you can simply read off. Loss at unit level is an inference problem — you have to work out which part of the process gave it up, from data that was never collected with that question in mind.
The second is that a factory is not a stationary environment. Recipes get reformulated, lines get retooled, suppliers change, volumes swing seasonally. A model trained on this year's process degrades against next year's, and it does so silently — the predictions keep arriving, they just stop being true.
Most manufacturing ML doesn't fail at deployment. It fails quietly, six months later, and nobody notices until the plant has stopped believing it.
What we built
Loss, broken down to the unit
Instead of reporting loss in aggregate, the solution attributes it granularly — producing not a number but a ranked list of specific places to intervene. That is the difference between a dashboard the plant looks at and a model the plant acts on.
Drift monitoring on two separate axes
We monitor data drift and model drift as distinct signals, because they fail differently and demand different responses. Data drift says the inputs no longer resemble what the model was trained on — the world changed, and a retrain may not be enough on its own. Model drift says predictive performance is degrading against outcomes — the mapping is stale even though the inputs look familiar. Collapsing the two into a single "model health" indicator is a common shortcut and it destroys your ability to diagnose which one you are facing.
Retraining that fires on evidence, not on a calendar
Retraining is triggered when monitoring crosses defined thresholds — not quarterly because the calendar said so, and not reactively because a user complained. This is the part that converts a successful project into a system the client can keep.
The outcome
- 20% reduction in operational loss.
- Unit-level loss visibility in place of aggregate reporting — decisions became specific enough to assign to someone.
- A self-monitoring system, live in production — drift detection and automated retraining running against the real process, rather than a model that needed us to stay in the room.
If you're facing something similar
Ask any vendor one question: what happens to this model in month seven? If the answer is "we would retrain it," with no monitoring in place to tell them when or why, that is not a plan — it is a hope, and you will be the one who discovers the problem.
And on loss models specifically: the valuable output is not the prediction, it is the granularity. A highly accurate forecast of total loss changes nothing. A roughly accurate attribution of loss to a specific unit changes what someone does tomorrow.