You need Python to run ML code. The key things: loops (repeat tasks over a list of documents), dictionaries (store mappings like label → index), and functions (reusable blocks of logic). You don't need to be an expert — just comfortable reading and writing scripts. Almost every ML library is Python-first, so this is the entry point for everything else.
Statistics is the language of measuring model quality. You need: mean (average score across a dataset), probability (likelihood that something belongs to a class), and distributions (how values spread). Nothing beyond high school maths is needed. The confusion comes later when people throw terms like PSI and KL divergence around — but those just describe how distributions shift over time.
A model is a function. You put something in — a document, an image, a sentence — it processes it, and gives you an output: a label, a number, a prediction. The "learning" part means the function adjusts its internal settings based on thousands of examples you show it, until it gets consistently good. That is literally all training is. The math behind how the settings adjust is calculus, but you don't need to know that to understand what's happening.
A model learns from examples. Each example has an input (a document) and a label (the correct answer). The more high-quality examples, the better the model. Bad or missing data is why most real-world ML projects fail — not the model architecture. The smartest model trained on bad data loses to a simple model trained on good data. Every time.
Two things can go wrong: you miss real things (low recall) or you flag things that aren't real (low precision). Precision = of everything I flagged, how much was correct? Recall = of everything that should have been flagged, how much did I catch? F1 = one number balancing both. Score of 1.0 is perfect, 0.0 is useless. I encountered this trying to understand why a model could look "good" overall but still fail badly in specific areas.
During training the model makes a guess. The loss function measures how wrong it was — a single number. High loss = bad guess, low loss = good guess. Training is: make a guess → measure loss → adjust settings slightly → repeat millions of times until loss is small. Understanding loss is the key to understanding why models train the way they do and why certain design choices (like separate losses for hierarchy levels) matter.
Always hold back some data the model never trains on — the test set. Use it to check if the model learned general patterns, or just memorised the training data (overfitting). Golden rule: never let training data touch the test set. This sounds obvious but in real systems with large datasets it's surprisingly easy to violate, and when you do, every metric you report is a lie.
Given an input, pick the right label from a fixed list. Email spam/not-spam is binary (2 options). Document typing with 50 types is multi-class. The model outputs a score for each label and picks the highest. The interesting challenge is when two labels look nearly identical to the model — this is where hierarchy, better training data, and post-processing rules come in.
Natural Language Processing is the field of making computers understand human text. It covers finding names in documents (NER), translating languages, summarising text, and generating responses. If your ML task involves text as input — classifying it, extracting things from it — you're doing NLP. Almost all document AI falls under this umbrella.
Models can't read raw text — they only understand numbers. Tokenization breaks text into chunks (tokens) and converts each to a number ID. "The quick brown fox" becomes something like [464, 2068, 7586, 21831]. The model processes these numbers, not the original words. One practical consequence: models have a maximum token limit, so long documents get truncated — understanding this matters when designing any text-based classifier.
An embedding converts a word — or sentence, or entire document — into a list of numbers where similar things end up with similar numbers. "Bank statement" and "financial document" would have close embeddings because they're related concepts. This is how models capture meaning. The CLS embedding from a transformer encodes the whole document in one vector, and that's what a classification head reads to make its prediction.
The dominant model architecture for text. A transformer reads the whole sentence at once and figures out how every word relates to every other word — this is the "attention" mechanism. BERT is a popular pre-trained transformer that produces a rich summary of any text you feed it. Models for document understanding typically start from a pre-trained transformer and fine-tune it on domain-specific data.
Smart highlighting. Given a document, the model finds and labels specific spans of text — dates, names, amounts, ID numbers. The output is a list of (start position, end position, label) tuples. It's one of the most common NLP tasks in document processing because most real business documents contain structured facts buried in unstructured text, and extracting those facts automatically is extremely valuable.
Document comes in, pick the best label from N options. Easy when N is small and classes are visually distinct. Hard when N is large and some classes look nearly identical. This is the problem that motivates hierarchical classification: instead of one hard N-way decision, make two easier decisions in sequence — first pick the family, then pick within the family.
The final step in most classifiers. The model produces raw scores for each label (A: 4.2, B: 1.1, C: 0.3). Softmax converts these into probabilities that add up to 100% (A: 92%, B: 6%, C: 2%). The highest probability wins. Understanding softmax helps you understand confidence scores — a 92% vs 52% prediction feel completely different even if both technically chose the right label.
When a model outputs 99% for a label, it's very confident. When it outputs 52% vs 45%, it's basically guessing. These scores drive multiple mechanisms: early exit (skip computation if already confident), active learning (send uncertain predictions to humans), and calibration (making sure 90% confidence actually means correct 90% of the time — models are often miscalibrated out of the box).
Before a model can learn, someone has to create the training data — reading documents and tagging things manually. This is the most expensive and time-consuming part of real ML projects, and it scales terribly. With a small annotation team and many models to maintain, you quickly hit a wall. This bottleneck is what drives techniques like weak supervision and active learning.
Serving means making the model available to process real requests in real-time. You wrap it in an API — a document comes in, a prediction goes out. The challenges: speed (users expect fast responses), reliability (a crash is a business outage), and scale (hundreds of thousands of requests per day require efficient batching, caching, and model compression to stay affordable).
Latency = how long a prediction takes. p50 = median (half of requests are faster than this). p99 = 99th percentile (only 1% of requests take longer). You care about p99 because that's your worst-case user. In systems processing millions of documents, 1% is still a lot of unhappy moments. Early exit architectures bring down p50 for easy inputs while keeping p99 acceptable for the hard ones.
Regular expressions are patterns that match text. ID numbers, dates, currency amounts, and postal codes all follow fixed formats — one regex catches all of them with near-perfect accuracy. In production ML systems, rules often outperform learned models on structured patterns and are much cheaper to write and debug. The smart approach is to use rules where they're reliable and models where they're needed.
Instead of one overall F1 score, you break it down by subgroups — entity type, language, document source, confidence band. A model might score 85% overall but only 62% on a specific combination that matters a lot to your users. Slicing makes invisible failure visible. Aggregate metrics are almost always hiding something, and slicing is the only way to find it.
Instead of randomly picking documents to label, pick the ones the model is most uncertain about. If the model is 51% vs 49% between two labels, that example is right on the decision boundary — labelling it teaches the model more than ten confident examples would. Combined with slice targeting, you can fix a specific failing subgroup with a fraction of the annotation effort random sampling would need.
Write rules that create labels automatically instead of humans doing it one by one. An ID number regex that fires on valid IDs gives you free labels — called "silver labels." The trick is combining multiple imperfect rules whose errors don't correlate, so you end up with reliable labels even when each individual rule makes mistakes. This is the core insight behind the Snorkel framework.
Models trained on old data degrade silently as the world changes. Drift detection catches this automatically before users notice. Three complementary signals: input drift (are incoming documents different from training data?), prediction drift (is the model behaving differently day-to-day?), and label-function agreement (do your deterministic rules still match the model's outputs?). Each signal catches different failure modes.