Learning Machines: What learning actually means

For years, I wrote code by telling the computer exactly what to do. If X, then Y. Else Z. It felt like playing god inside a very small, completely predictable sandbox. You write the rules, the machine follows them, and as long as your syntax is valid and your edge cases are covered, the output is guaranteed.

But machine learning flips that sandbox upside down.

As software engineers, we are trained to maintain complete control over our execution paths. We want predictability. Shifting to a model where the machine generates its own logic based on data feels like stepping off a cliff.

But it is a necessary step if we want to build systems that can handle the complexity of the real world.

To understand this transition, we have to go back to the beginning to figure out what “learning” actually means in a practical, engineering sense.

The Rule-Writing Sandbox (and Why It Breaks)

In traditional programming, we combine rules (our code) and data (our inputs) to produce answers. If you want to build a spam filter, you write a long list of regex rules matching terms like “wire transfer” or “free gift.”

It works—until it doesn’t.

Spammers change their words, new edge cases appear, and suddenly your simple rule-engine has grown into a 2,000-line spaghetti monster that nobody wants to touch. Every time the environment shifts, you have to write more rules, re-compile, and redeploy.

Machine learning changes the flow. Instead of rules and data producing answers, we feed the machine data and answers, and it produces the rules.

When the system works, the model uncovers patterns we might have missed entirely. Even better, when the environment changes, we don’t have to rewrite our code. We retrain the model on new data.

Why Make the Shift?

We choose to build machine learning systems over traditional, rule-based software for four main reasons:

1. The Rule-Maintenance Trap

Some problems require an astronomical list of rules just to achieve basic accuracy. A traditional spam filter or document parser might start simple, but as spammers or document formats evolve, you find yourself constantly fine-tuning and adding exceptions. A machine learning model can simplify this code entirely, performing far better while requiring a fraction of the maintenance.

2. Intractable Problems

Speech recognition is a classic example. Writing explicit rules to process acoustic frequencies for every human accent, tone, and inflection is practically impossible. The best machine learning techniques analyze vast amounts of audio data and discover the mathematical relationships themselves, finding solutions where traditional code fails.

3. Fluctuating Environments

A system that works today might break tomorrow because user behavior shifts. In a traditional system, you must manually audit the failure, write new rules, test the logic, and redeploy. A machine learning system handles this seamlessly: you simply retrain the model on the latest data, keeping the system up to date without manual logic rewrites.

4. Data Mining

When we train models on massive datasets, we often discover hidden patterns and relationships that were completely invisible to us (a process known as data mining). This helps us understand our own systems and users better, providing actionable insights that we can feed back into our designs.

Tom Mitchell’s Equation (and the Spec Sheet for Learning)

To understand this clearly, we have to drop the hand-wavy marketing terms and look at how computer scientists actually define this stuff. In 1997, Tom Mitchell wrote a formal definition that makes perfect sense to a developer:

A computer program is said to learn from experience $E$ with respect to some class of tasks $T$ and performance measure $P$, if its performance at tasks in $T$, as measured by $P$, improves with experience $E$.

If you strip away the math, it is just a spec sheet. Let’s look at spam filtering through this lens:

• Task ($T$): Classify incoming emails as spam or ham.
• Performance Measure ($P$): The accuracy of the classification (the ratio of correctly flagged emails).
• Experience ($E$): The historical dataset of labeled emails the model studies during training.

If the accuracy ($P$) goes up after the model runs through the training data ($E$), the system is learning. No magic—just optimization.

Classifying Machine Learning Systems

We can classify machine learning systems across three main axes:

1. How they are supervised during training: Whether they learn from labeled guide data, unlabeled structures, or trial-and-error feedback loops.
2. Whether they learn incrementally on the fly: Online versus batch learning.
3. How they generalize: Instance-based learning (comparing new points to known points) versus model-based learning (building a predictive model from training patterns).

Axis 1: Training Supervision

How we expose the machine to experience ($E$) defines the type of machine learning system we are building. This supervision falls into five distinct categories.

1. Supervised Learning: Learning with a Guide

In supervised learning, each instance in the training set contains the desired solution, known as labels or targets. The model studies the features of the input data and maps them to these targets.

We use supervised learning for two main tasks:

• Classification: Suppose you want to predict which patients likely have cancer and which do not. You gather medical records from patients whose definitive health status is already known, extract their features (age, blood work, or genetic markers), and feed them to an algorithm that learns to draw a boundary between the two classes.
• Regression: Predicting a target numeric value. If you run a second-hand car sales business (and want to avoid being swindled when buying stock), you can gather features from previously sold cars (mileage, age, and make) along with their actual sale prices. The regression model learns exactly how these features impact the final price, allowing it to predict a continuous monetary value for new cars.

2. Unsupervised Learning: Finding Hidden Patterns

In unsupervised learning, we feed the system completely unlabeled data. We want the algorithm to analyze the structure of the data and recognize patterns that we might have missed entirely.

Common tasks here include:

• Clustering: Grouping similar instances. A typical example is analyzing blog visitors (a scenario that hits a little too close to home). We feed visitor metrics (time of visit, duration, and category selection) into a clustering model. The model discovers that our visitors naturally fit into three categories: working-class people who read after work, students who read during the day (probably between classes), and elderly people trying to learn more about tech. This allows us to target content specifically for each group.
• Anomaly Detection: Spotting outliers. Here, we care less about the data that follows the patterns, and more about the data that does not. A model can study credit card transaction features (time, location, and whether the cardholder suddenly purchased twelve designer handbags at 3 AM) to learn normal behavior. It can then instantly flag transactions that do not fit the normal behavior, spotting potential fraud quicker and more accurately than a human could.
• Dimensionality Reduction: Simplifying datasets. We can often get most of the information from a dataset without relying on every single attribute. Dimensionality reduction reduces the size of our dataset, making it easier to work with, without losing a lot of the underlying signal. By combining highly correlated features or using techniques like PCA (Principal Component Analysis), we can speed up training and reduce noise.
• Association Rule Learning: Sifting through transactions to find interesting relationships between attributes. In a supermarket, running an association rule algorithm on sales logs might reveal that customers who buy barbecue sauce and potato chips also tend to buy steak. You can then place these items close to one another to drive sales.

3. Semi-Supervised Learning: The Best of Both Worlds

In the real world, labeling data is expensive. Your average store most likely doesn’t have all its data labeled, and your everyday blog doesn’t know which viewers are students or working class. Such cases are where semi-supervised learning shines, using a dataset with some labeled and mostly unlabeled instances. Most of the time, it is a smart combination of unsupervised and supervised learning.

Suppose we want a model that can classify geometric shapes, but only some shapes in the training set have labels. We can’t use supervised learning straight away because of the missing labels. Instead, we can start by grouping similar items together using a clustering algorithm.

Now we have clusters of similar shapes. Chances are that there are some labeled shapes within those clusters. If a specific label appears the most in a cluster, we can safely assume the other shapes in that cluster ought to share the same label. Once we propagate the labels this way, we have a fully labeled training set. From here, we can train a standard supervised classification model on the complete, labeled data.

4. Self-Supervised Learning: The Masked Predictor

The main idea of self-supervised learning is that the data itself is the label. It is a form of learning where there are no labels to begin with. Instead, we obscure or hide some part of the data and train a model to fill in the blanks.

A common example is image repair models. You start with an unlabeled dataset of dog images, obscure some parts of the image, and train the model to predict the missing pixels. The original, unmasked image becomes the target label. By learning to reconstruct the missing details, the model develops a deep mathematical understanding of the features in the image. This is the exact mechanism used to train large language models to predict the next word in a sentence.

5. Reinforcement Learning: Trial, Error, and Policy

Reinforcement learning is a different beast entirely. Here, a software agent observes an environment and takes certain actions within that environment. Each action results in a reward or a penalty (a negative reward). As the agent explores the environment, it must learn what the best strategy (known as a policy) is to get the maximum rewards over time.

The policy defines exactly what the agent should do in any given situation.

For example, many robots implement reinforcement learning to learn how to walk, constantly adjusting motor controls to avoid falling (a penalty) and move forward (a reward). DeepMind’s AlphaGo program is also a good example: it made headlines in May 2017 when it beat Ke Jie, the number one Go player in the world at the time. It learned its winning policy by analyzing millions of games and playing many games against itself.

What’s Next?

I had to pause there—a production alert called, and it was time to get back to my day job.

If we strip away the hype, machine learning is just curve-fitting on a massive scale. It is still just math and code. We are just writing the code that finds the curve, instead of writing the curve ourselves.

In the next post, we will look at the other fundamental ways of classifying models: online versus batch learning, and instance-based versus model-based learning. Until then, the gap between using these tools and understanding them is a little smaller.