What it takes to calibrate a PLS model for a real process stream

An inline spectrometer does not measure concentration. It measures absorbance at a few hundred wavelengths. Turning that into “3.4 g/L of product” is a regression problem, and partial least squares is the standard tool for it. The instrument is bought in a week; the model is where the project succeeds or quietly fails.

Why not just use one peak

Univariate calibration — pick the peak that belongs to your analyte, measure its height, fit a straight line — works in a clean binary system and fails in a real process stream. Peaks overlap. The baseline drifts with temperature and with particulates. Other components in the stream vary independently and absorb in the same region. A single wavelength cannot distinguish a rise in your analyte from a rise in something that overlaps it.

PLS uses the whole spectral region. It finds the directions in the spectral data that covary with the reference concentrations, which lets it use the pattern across many wavelengths to separate contributions that no single wavelength can.

Designing the calibration set

This is where most models are decided, before any software is opened.

Span the range. The model can interpolate; it cannot extrapolate. If production will see 0–40 g/L, the calibration set has to cover 0–40 g/L, including the ends, which are exactly the samples nobody wants to make.

Break the correlations. In a normal batch, substrate falls while product rises, so the two are almost perfectly anti-correlated. A model trained only on normal batches cannot tell which of the two it is actually seeing — it will predict product beautifully until a batch behaves abnormally, and then fail in exactly the situation you built it for. The fix is deliberately designed samples where the analytes vary independently: spiked samples, mixtures, off-normal runs.

Cover the nuisance variation. Temperature, particulate load, pH, raw-material lot, different vessels. Anything that will vary in production and affects the spectrum must vary in the calibration set, or the model will treat it as signal.

Get good references. Predicted values cannot be more accurate than the reference method they were trained against. If HPLC reproducibility is ±3%, the model’s floor is ±3%, and no amount of preprocessing will go below it.

Preprocessing, briefly

Raw spectra carry physical effects that have nothing to do with chemistry: scatter from particles, baseline offsets, path-length variation. Standard normal variate and multiplicative scatter correction remove most scatter effects. First and second derivatives remove baseline offset and slope, at the cost of amplifying noise. Choose deliberately and document the choice — preprocessing is part of the model, and a model transferred without its preprocessing is not the same model.

How many components, and how you know

The one tuning parameter that matters in PLS is the number of latent variables. Too few and the model misses real structure. Too many and it starts fitting noise, which shows up as excellent performance on the calibration set and poor performance on anything new.

Cross-validation gives you the honest estimate. Critically, it must be blocked by batch, not by individual sample: samples from the same batch are not independent, and random sample-wise cross-validation will report an optimistic error and let you overfit. Plot the cross-validated error against the number of components and take the point where it stops improving materially, not the absolute minimum.

Then validate on batches held out entirely from model building. That number — root mean square error of prediction on independent batches — is the one to quote.

Life after deployment

Models drift, because processes change. New raw-material suppliers, a new strain, a cleaned probe with a slightly different path length, a seasonal change in feedstock. Two habits keep a model alive:

• Monitor the residuals. A rising spectral residual means the instrument is seeing something the model was never trained on. That is your early warning, and it fires before the predictions visibly go wrong.
• Keep a reference cadence. Pull an occasional offline sample and compare. Not the pre-inline sampling frequency — enough to detect drift.

Plan for model maintenance from the start. An inline measurement project that budgets for the instrument and not for the model is budgeting for half the system.

Paeonia develops chemometric models against your own process samples as a service. See how that works.