Practical ML Is About Iteration Not Architecture

November 16, 2022

The gap between ML research papers and ML in production is not about choosing the right model architecture. It is about building tight feedback loops between data, training, evaluation, and deployment.

"Overfitting is the single most important and challenging issue when training for all machine learning practitioners, and all algorithms. It is easy to create a model that does a great job at making predictions on the exact data it has been trained on, but it is much harder to make accurate predictions on data the model has never seen before." Jeremy Howard, Deep Learning for Coders

The academic presentation of machine learning suggests a clean pipeline: pick an architecture, train it, evaluate it, deploy it. In practice, the architecture choice is often the least important decision. What matters far more is how quickly you can iterate on your data, your features, your validation set, and your understanding of what the model is actually learning versus memorizing. Jeremy Howard's fastai philosophy exemplifies this: start with a pretrained model, get a baseline fast, then iterate on data quality and augmentation before touching the model itself.

The Hands-on Machine Learning checklist reinforces this. The recommendation to "try out many other models from various categories without spending too much time tweaking the hyperparameters" before shortlisting two to five candidates is the opposite of how most beginners approach ML. They pick one model and spend weeks tuning it. Similarly, Data Science for Business emphasizes that "a critical skill in data science is the ability to decompose a data-analytics problem into pieces such that each piece matches a known task for which tools are available" the real work is problem formulation, not model selection.

The startup world has learned this the hard way. Yi Tay at Reka described how training great LLMs required abandoning "the systematicity of Bigtech" and relying on intuition built from many prior iterations, using "Yolo runs" when compute was limited. The lesson is universal: models rot as data evolves, validation sets need constant scrutiny, and monitoring live performance matters more than initial accuracy.

Takeaway: Spend 80% of your time on data quality, validation design, and iteration speed the model architecture is rarely the bottleneck.

Linked from

Data Quality Is the Real Moat in Applied ML