The same compound, a different minute in every batch
Fish oil extracts from five acquisition batches (Aug 2015 to Jul 2016). Within a batch the runs agree; between batches whole chromatograms are shifted by up to several minutes, and not by a constant. Hover to read every batch at the same retention time; switch to see them in the shared frame.
Each row is a batch; each line one sample's normalised total ion chromatogram (TIC). Shaded on the left: the first ≈ minutes, before every run had started acquiring, which is excluded from every score on this page.
Pairwise warps disagree with each other
Align batches two at a time and you get ten separate warps that need not agree: going A → B → C can land somewhere different from going A → C directly. Pick a route through the batches below.
Route
Retention-time shift along the route. Pairwise: the composed path (dashed) vs the direct warp (solid). Joint: both are the same curve.
Choose a route
Click three batches in order. Line width is the number of direct anchors between two batches.
All 60 routes
Mean disagreement over 5–40 min, sorted. Hover a bar; click to show that route.
One warp per batch, into one shared clock
Every batch k gets a single monotone map fk(t) = t + dk(t) into a latent retention-time axis. A warp between any two batches is fl−1 ∘ fk, so all the pairwise warps are consistent by construction, and a pair with few anchors borrows strength from the others.
Harvest anchors
Peaks on the raw TIC are matched by mass spectrum between every cross-batch sample pair, with sub-bin refinement: anchors.
One warp per batch
Drift dk is piecewise linear on 41 knots; slopes are bounded to 0.5–2, so every warp is strictly monotone.
Fit them jointly
One bounded least-squares problem over all anchors, with Huber then Tukey reweighting so mismatched compounds are rejected.
Choose smoothness honestly
λ is picked by held-out anchor error on whole retention-time windows. TIC correlation is never used to tune anything.
Drift of every batch into the shared clock
dk(t) = fk(t) − t for the selected smoothness λ. The reference batch () is fixed at zero.
Picking λ without peeking
Median held-out anchor error (RT-blocked cross-validation). The minimum is the selected λ; the marker follows the slider.
TIC r shown for information only: it played no part in choosing λ.
Thousands of anchors, a few of them wrong
Each point is one compound matched between a sample of batch A and a sample of batch B: where it eluted in A, and how far it moved. The joint fit follows the dense consensus and gives zero weight to mismatches. Choose a batch pair.
Better alignment of drifted batches on average, little change where they already agree
Mean Pearson correlation between batch-A and batch-B chromatograms after warping A into B's frame (common acquisition window), for every batch pair and method. Drifted pairs are those with unaligned r < 0.5.
TIC r by batch pair and method
Darker = higher correlation. Outlined: best on that pair. "CF" = cross-fitted: each A sample warped by a model that never saw it.
Sample by sample
Every batch-A sample's mean r against batch B, for two methods. Lines join the same sample.
Delete a pair's anchors. The warp mostly survives.
Because every batch lives on the shared clock, the warp between A and B is pinned down by the paths through the other batches. Remove every direct A–B anchor, refit, and compare: for most pairs the correlation hardly changes; the right-hand panel shows where it does.
What changed
Joint fit with all anchors vs refitted without this pair's anchors.
TIC r on all ten pairs: with (●) and without (○) the pair's own anchors.
Narrow bands that under-cover
Resampling samples within each batch and refitting 200 times gives a band around every warp. Those bands are narrow, and the predictive intervals built from them cover fewer held-out anchors than they should, so they need calibration before anyone relies on them.
Band width along the run
Width of the 90 % bootstrap band for each batch pair (minutes). Hover for values.
Held-out coverage
Share of held-out anchors inside their 90 % interval.
Every method, the same yardstick
Published numbers mix protocols: some methods warp only A, some resample both batches, some fit a separate warp for every sample pair. Here each method is reduced to the map it estimates (RT in B → RT in A) and scored two ways: in B's frame, and in the frame halfway between the batches.
What these results do and don't show
Five batches, one study
Batch is confounded with species and tissue, and aug2015 has two samples. A dataset with more batches and repeated QC samples is needed.
TIC r is a proxy
Correlation of total ion chromatograms rewards overlapping peaks, not correct compound identities. Held-out anchor error is reported alongside it.
Under-covering intervals
Bootstrap bands of a regularised fit miss its bias. Predictive coverage is against a 90 % target.
End-of-run anchors
Late, bleed-like peaks with similar spectra produce conflicting anchors; the robust fit rejects them, so the last minutes rest on extrapolation.
Run it on synthetic data with a known answer
The fish oil chromatograms are private, so the repository ships a generator for a public stand-in: five batches with the same names and design, drifts shaped like the real ones (two batches eluting 2.5–5 min early, growing along the run), staggered acquisition starts and column bleed. The true drift of every batch is saved next to the data, so you can check that the joint model recovers it.
# joint-align, plus its pinned shared core (alignbench) from GitHub git clone https://github.com/woodRock/joint-align && cd joint-align pip install -e ".[neural]" # 31 chromatograms, 5 batches, true drifts in truth.json python scripts/make_synthetic_data.py # joint model on it, ~1.5 min python scripts/01_multibatch_align.py --data data/synthetic \ --results results/synthetic --figs figs/synthetic # every method, same protocols, ~1 min python scripts/03_fair_benchmark.py --data data/synthetic \ --results results/synthetic --neural-steps 300 # or benchmark the joint model against the baselines # with alignbench (same world, generated in memory, ~30 s) alignbench run --methods joint,pooled-pchip,cow-tic \ --dataset synthetic:fishlike --seeds 0 \ --protocols tic_bframe,tic_midpoint,map_error \ --out results/bench
What to expect
On the synthetic batches the joint warps land within about 0.1 min of the true drift wherever compounds exist in both batches (checked in tests/test_pipeline.py). Beyond the last compound of a batch the warp is an extrapolation.
alignbench run writes a report, a LaTeX table and every score per sample pair to results/bench, with the code revisions at the top; the map error against the true drift is only defined for methods with one map per batch pair, so COW-TIC's column stays empty. The WarpTransformer baseline needs warpalign and its pretrained checkpoint (pip install -e ".[wt]"), so the demo benchmark leaves that column empty too. make check runs the linters and the test suite, which includes this demo.
Figures on this page: fish oil data, , exported .