Machine Learning for Trading

What problem is ML solving?

The question that drives every ML model in trading is simple: can historical data predict future returns? If past prices, volumes and volatility carry any repeatable structure, then a model can identify useful patterns and turn them into trading signals.

We already build signals by hand. Momentum, mean reversion and volume rules are explicit, human-written if/then statements. ML asks a different question: instead of writing the rule, can we let an algorithm learn the rule from the data?

What is machine learning?

Quant firms lean on ML because the modern data environment plays to its strengths:

ML does not replace the research workflow — it slots into one stage of it:

flowchart LR
  A([Data]) --> B([Features])
  B --> C([Model])
  C --> D([Prediction])
  D --> E([Backtest])
  E -.->|Refine features / model| B
  classDef s fill:#eef2ff,stroke:#4f46e5,stroke-width:1px,color:#3730a3;
  classDef d fill:#ecfdf5,stroke:#0d9488,color:#0d9488;
  class A,B,C,D s
  class E d
      

Prediction vs explanation — and why ML is not magic

There is a genuine trade-off between the two approaches. A hand-written rule is easy to explain: "buy when the 20-day return is positive." An ML model is more flexible but harder to interpret — it may use fifty features in combinations no one can narrate.

Features: the inputs to a model

Typical features are derived from raw OHLCV data:

A concrete setup might use return, volatility and volume ratio as features, with future return as the target.

Good features vs bad features

Validation: training data vs test data

Validation exists to answer one question: will this model work on data it has never seen? That property is called generalization, and it is the only thing that matters for live trading.

Overfitting: memorizing noise

The core distinction is signal vs noise: a signal is a persistent pattern that recurs out of sample; noise is random variation that happens not to repeat. An overfit model has confused the second for the first.

Out-of-sample testing & walk-forward validation

Out-of-sample testing evaluates the model on unseen, independent data, giving a realistic proxy for future performance. It is more honest than in-sample scores, validates better, and exposes overfitting early.

Walk-forward validation makes this repeatable: train on a window, test on the next slice, then roll the window forward and repeat. The model is always tested on data that came after its training window — exactly how it would face the real world.

flowchart LR
  subgraph R1[Round 1]
    A1([Train])-->B1([Test])
  end
  subgraph R2[Round 2]
    A2([Train])-->B2([Test])
  end
  subgraph R3[Round 3]
    A3([Train])-->B3([Test])
  end
  B1 -.->|move window| A2
  B2 -.->|move window| A3
  classDef s fill:#eef2ff,stroke:#4f46e5,stroke-width:1px,color:#3730a3;
  classDef d fill:#ecfdf5,stroke:#0d9488,color:#0d9488;
  class A1,A2,A3 s
  class B1,B2,B3 d
      

Evaluating predictions: R², MAE, MSE

We measure prediction quality to compare models and detect overfitting. The common metrics are R², MAE (mean absolute error) and MSE (mean squared error). The course focuses on R².

where the residual and total sums of squares are

Models: regression and decision trees

Linear regression intuition

A linear regression takes feature inputs — momentum, volatility, volume — and combines them with learned weights to output a predicted return. For example: positive momentum, low volatility and high volume might combine into a positive predicted future return.

Decision trees

A decision tree is a series of yes/no decisions arranged like a flowchart. It is easy to interpret — you can read the path that led to any decision.

flowchart TD
  A{Volume > average?} -->|Yes| B{Momentum positive?}
  A -->|No| H([Hold])
  B -->|Yes| C([Buy])
  B -->|No| H
  classDef s fill:#eef2ff,stroke:#4f46e5,stroke-width:1px,color:#3730a3;
  classDef d fill:#ecfdf5,stroke:#0d9488,color:#0d9488;
  class A,B s
  class C,H d
      

Why ML often fails in finance

Key takeaways