Data Normalization

Commander Mirza

Before the deep scan runs, normalize every sensor feed. One scale. No exceptions.

Cut to: Briefing

Ensign Demir

The thermal channel reads in the thirties, signal strength lives under one. If we feed both in raw, the model's going to think temperature is the whole story.

Cut to: Briefing

Dr. Farah

Not think. Compute. The bias isn't an opinion the model forms; it's arithmetic it can't avoid. Fix the scales and the bias never exists.

Cut to: Briefing

Why Scales Must Match

You've computed min-max normalization before, probably this month: scroll progress. scrollY / (docHeight - viewportHeight) takes a value in pixels and maps it to 0-1 so a progress bar can use it. That's the entire min-max formula: subtract the minimum, divide by the range.

Here's why a model needs the same treatment. The temperature channel runs 19 to 35. Signal strength runs 0.1 to 0.9. Feed both in raw and temperature's numbers land roughly forty times larger, so the louder feature dominates what the model learns while the quiet one barely registers, even if the quiet one carries the signal you actually care about.

One practical wrinkle that bites in production: whatever mean and std you compute here, you must save. Inference data has to be normalized with the training statistics, not its own; otherwise the same reading maps to different numbers on different days, and your model quietly answers a different question each time.

When to Use Which

Z-score (mean=0, std=1): the default for neural network inputs. Distorted far less by outliers than min-max, and assumes nothing about the data's bounds.

Min-max (range 0-1): for data with true, known bounds: pixel values, percentages, probabilities. One extreme outlier crushes everything else into a corner, so don't use it on open-ended streams.

Challenge

Normalize the Archimedes sensor data using z-score normalization.