From Zero to Quant · Chapter 15
Why Financial ML is Different
Non-stationarity, low SNR, regime changes, overfitting
Introduction#
You've mastered deep learning. You've built CNN models that achieve 99% accuracy on ImageNet. You've trained transformer networks that generate Shakespeare sonnets. You understand backpropagation, regularization, and hyperparameter optimization at a bone-deep level.
So why do your machine learning models generate trading Sharpe ratios of 0.2 when your offline backtests promised 3.0?
The answer is not that you're a bad ML engineer. The answer is that financial machine learning is fundamentally different from traditional machine learning in ways that invalidate nearly every assumption you've internalized. This chapter explores these differences and shows you exactly where and why standard ML practices fail on financial data.
By the end of this chapter, you'll understand:
- Why R² of 0.01-0.03 is actually excellent in finance (not a disaster)
- How non-stationarity breaks your validation methodology
- Why your model's actions create feedback loops that contaminate your future training data
- How to properly label financial data for modeling
- Why sample weighting transforms untrustworthy predictions into tradeable signals
Module 15.1: The Unique Challenges of Financial ML#
The Signal-to-Noise Ratio Problem: Why Your Model Isn't Broken#
In computer vision, you might achieve R² of 0.95 on a test set and sleep soundly knowing your model learned real patterns. In natural language processing, BLEU scores of 0.45+ indicate meaningful translation quality.
In financial ML, an R² of 0.025 is exceptional.
This isn't failure—it's the reality of financial markets.
Why Financial Signals Are Weak
Financial returns are generated by:
- Efficient pricing (the market incorporates public information rapidly)
- Stochastic competition (thousands of other traders/algorithms fighting for the same edges)
- High transaction costs (eating your small alpha before you can realize it)
- Regime changes (the relationships that generated alpha disappear)
Consider predicting daily stock returns. The fundamental equation is:
where is tomorrow's return, are your features, and is noise.
The noise term includes:
- Random corporate announcements
- Macro news events
- Central bank interventions
- Sentiment shifts
- Order flow imbalances
- Market microstructure effects
- Hundreds of other traders' algorithms
The signal terms represent predictable patterns that:
- You've discovered
- Hundreds of quant funds have also discovered
- Are being arbitraged away in real-time
Result: Your R² is naturally bounded far below what you'd achieve in other domains.
The Theoretical Lower Bound on Predictability
The Efficient Market Hypothesis (EMH) in its weak form states that historical prices don't predict future prices. But we're not trying to predict prices—we're trying to predict abnormal returns above the risk-free rate.
The theoretical maximum predictability is bounded by:
In liquid markets, empirical studies suggest:
- for daily returns
- For intraday returns, the ratio is even worse
- For weekly/monthly returns, the ratio improves to perhaps 10:1
This means even a theoretically perfect predictor would achieve:
An R² of 0.025-0.03 means your model is capturing a meaningful fraction of the available signal.
Non-Stationarity: The Core Problem with Standard ML#
Standard machine learning assumes stationarity: that the data-generating process is fixed. You train on historical data, validate on held-out data, and deploy knowing the relationships learned will hold in production.
Financial data is fundamentally non-stationary.
What Non-Stationarity Means
Non-stationarity means the joint distribution of returns and features changes over time:
This can manifest as:
-
Parameter drift: Regression coefficients change
- A feature that predicts returns in Bull markets may predict the opposite in Bear markets
- The autocorrelation of returns changes with market regime
-
Volatility changes (heteroskedasticity)
- Expected volatility spikes during crises
- Low-volatility periods offer less signal
-
Correlation breakdown
- Diversification benefits disappear during crashes
- Sector correlations change with macro regime
-
Structural breaks
- New trading venues appear
- Regulations change
- Technology shifts market microstructure
The Catastrophic Failure of Standard ML Pipelines
Here's the critical error that every quantitative trader makes once:
# WRONG: This will fail spectacularly in production
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestRegressor
import pandas as pd
import numpy as np
# Load 5 years of daily data
data = pd.read_csv('NIFTY_2019_2024.csv')
X = data[['momentum', 'mean_reversion', 'vol_adjusted_return', ...]]
y = data['forward_5d_return']
# Standard ML pipeline
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
# Train random forest
model = RandomForestRegressor(n_estimators=100, max_depth=15)
model.fit(X_train_scaled, y_train)
# Evaluate
train_r2 = model.score(X_train_scaled, y_train) # 0.08
test_r2 = model.score(X_test_scaled, y_test) # 0.075
# Deploy to live trading...
# Result in production: Sharpe ratio of 0.15, compared to promised 2.5Why does this fail?
Random train-test split breaks temporal dependencies.
When you randomly shuffle your data, you:
- Train on data from December 2020 mixed with January 2024
- Test on data from March 2019 mixed with May 2023
- Destroy the non-stationary structure that the model must navigate
The model learns to average across all market regimes, not to adapt to regime changes. When deployed in 2024, it encounters market conditions unlike the statistical mixture in your training set.
Walk-Forward Validation: Respecting Causality#
The correct validation methodology respects temporal causality:
# CORRECT: Walk-forward (time-series cross-validation)
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestRegressor
import pandas as pd
import numpy as np
data = pd.read_csv('NIFTY_2019_2024.csv')
X = data[['momentum', 'mean_reversion', 'vol_adjusted_return', ...]]
y = data['forward_5d_return']
# Walk-forward parameters
train_size = 252 * 2 # 2 years of training
test_size = 63 # 3 months of testing (3 * 21 trading days)
step_size = 63 # Rebalance quarterly
results = []
# Walk forward through time
for train_start in range(0, len(data) - train_size - test_size, step_size):
train_end = train_start + train_size
test_end = train_end + test_size
# Extract train and test sets (contiguous in time)
X_train = X.iloc[train_start:train_end]
y_train = y.iloc[train_start:train_end]
X_test = X.iloc[train_end:test_end]
y_test = y.iloc[train_end:test_end]
# Fit scaler ONLY on training data
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
# Train model on historical data
model = RandomForestRegressor(n_estimators=100, max_depth=15)
model.fit(X_train_scaled, y_train)
# Evaluate on future data (never seen during training)
test_pred = model.predict(X_test_scaled)
# Calculate returns from predictions
test_correlation = np.corrcoef(y_test, test_pred)[0, 1]
test_returns = (test_pred * y_test).sum()
results.append({
'period': f'{train_start}-{test_end}',
'correlation': test_correlation,
'returns': test_returns
})
results_df = pd.DataFrame(results)
print(f"Average correlation: {results_df['correlation'].mean():.4f}")
print(f"Correlation std: {results_df['correlation'].std():.4f}")
print(f"Profitable periods: {(results_df['returns'] > 0).sum()} / {len(results_df)}")The results using walk-forward validation on realistic data:
- Random split: Test R² = 0.075, implied Sharpe = 3.2
- Walk-forward: Average correlation = 0.008, implied Sharpe = 0.18
This 20x difference between in-sample and out-of-sample performance is not unusual. It reflects the gap between:
- Performance on averaged historical conditions
- Performance on future conditions that differ from history
Feedback Loops: Your Model Pollutes Its Own Training Data#
This is the most insidious problem in financial ML, and standard machine learning has no concept of it.
Your model's trading actions affect the prices and volumes it learns from.
How Feedback Loops Destroy Generalization
Suppose your model learns: "When RSI drops below 30, buy"
On historical data, this generates positive returns. You deploy the model. It works, you make money.
But now:
- Your model buys (along with other traders with similar strategies)
- Buying pressure drives prices up
- Prices rise before your target "mean reversion" signal
- Future historical data now contains distorted price patterns where mean reversion fails more often
- You retrain your model on this contaminated data
- The model learns weaker mean reversion signals
- Live performance degrades
This is a feedback loop: your actions create new training data that reflects your own impact, not natural market dynamics.
Quantifying Feedback Loop Contamination
def simulate_feedback_loop_contamination(
initial_positions: np.ndarray,
price_sensitivity: float = 0.001,
num_iterations: int = 10
) -> dict:
"""
Simulate how model feedback loops contaminate future training data.
Args:
initial_positions: Array of trading positions from model
price_sensitivity: How much price moves per unit of trading
num_iterations: Iterations of feedback loop
Returns:
Dictionary tracking price distortion and signal degradation
"""
# Baseline prices
baseline_prices = np.cumsum(np.random.normal(0, 1, 1000))
# Track metrics
original_returns = np.diff(baseline_prices)
contaminated_prices = baseline_prices.copy()
signal_correlation_history = []
for iteration in range(num_iterations):
# Model generates trades based on current state
# Simplified: buy when rolling mean > price
rolling_mean = pd.Series(contaminated_prices).rolling(20).mean().values
model_positions = (rolling_mean > contaminated_prices) * 1.0
# Your model's buying affects prices (market impact)
price_impact = model_positions * price_sensitivity * np.std(contaminated_prices)
contaminated_prices = contaminated_prices + price_impact
# Measure signal degradation
# (correlation between model features and actual returns decreases)
contaminated_returns = np.diff(contaminated_prices)
signal_corr = np.corrcoef(model_positions[:-1], contaminated_returns)[0, 1]
signal_correlation_history.append(signal_corr)
return {
'original_sharpe': np.mean(original_returns) / np.std(original_returns) * np.sqrt(252),
'contaminated_sharpe': np.mean(contaminated_returns) / np.std(contaminated_returns) * np.sqrt(252),
'signal_correlation_decay': signal_correlation_history,
'price_distortion': np.abs(contaminated_prices - baseline_prices).mean()
}
# Run simulation
results = simulate_feedback_loop_contamination(
initial_positions=np.random.normal(0, 1, 1000),
price_sensitivity=0.0005,
num_iterations=20
)
print(f"Original Sharpe: {results['original_sharpe']:.3f}")
print(f"Contaminated Sharpe: {results['contaminated_sharpe']:.3f}")
print(f"Price distortion: {results['price_distortion']:.4f}")Key insight: Even small amounts of market impact ($0.01 per contract) can significantly distort the future prices your model trains on, creating an illusion of edge that disappears when you scale.
Survivorship Bias: Features and Targets#
There are two types of survivorship bias in financial ML:
Feature Survivorship Bias
Suppose your dataset includes 500 stocks. You engineer 150 features for each. You test which features correlate with future returns. The top 20 features show significant correlation.
Problem: You've tested 500 × 150 = 75,000 feature-target pairs. By chance alone, some will correlate. You're looking at the survivors of this multi-test universe.
The Bonferroni correction suggests dividing your significance threshold by the number of tests:
Most correlations that pass standard statistical tests (p < 0.05) don't survive this correction.
Target Survivorship Bias
If you're training on stocks from an index, you have only stocks that survived to be included today.
- Delisted companies are removed
- Bankrupt companies are removed
- Merged companies disappear
- Stocks that fell 99% are no longer in the dataset
This creates an upward bias in your forward returns: you're training on an index of survivors, not the true distribution of outcomes.
Addressing Survivorship Bias
def adjust_for_survivorship_bias(
feature_target_correlations: pd.DataFrame,
n_total_features: int,
significance_level: float = 0.05
) -> pd.DataFrame:
"""
Apply Bonferroni correction to account for multiple testing.
Args:
feature_target_correlations: DataFrame with columns ['feature', 'correlation', 'pvalue']
n_total_features: Total number of features tested (for multiple testing correction)
significance_level: Original significance level (typically 0.05)
Returns:
DataFrame with corrected significance threshold applied
"""
# Bonferroni corrected threshold
bonferroni_threshold = significance_level / n_total_features
# Apply correction
feature_target_correlations['significant_after_correction'] = (
feature_target_correlations['pvalue'] < bonferroni_threshold
)
# Conservative: only keep features that survive aggressive correction
feature_target_correlations['sharpe_adjusted'] = (
feature_target_correlations['correlation'] *
feature_target_correlations['significant_after_correction']
)
return feature_target_correlations.sort_values('sharpe_adjusted', ascending=False)
# Example: Testing 200 features on 10 stocks, 5 years daily data
# Total comparisons: 200 features × 10 stocks = 2000
n_total_tests = 2000
significant_before = 200 * 0.05 # Expected by chance
significant_after_correction = 200 * (0.05 / 2000) # After Bonferroni
print(f"Features showing p < 0.05: ~{significant_before:.0f}")
print(f"Features passing Bonferroni: ~{significant_after_correction:.1f}")Serial Correlation and Non-IID Samples#
Standard ML assumes samples are independent and identically distributed (IID). This enables:
- Random train-test splits
- Standard cross-validation
- Unbiased error estimates
Financial data violates IID in critical ways.
Serial Correlation in Returns
Financial returns exhibit autocorrelation at multiple scales:
For NSE stocks:
- Daily returns: to (weakly positive)
- Intraday returns: to (mean reversion)
- Monthly returns: to (momentum)
This autocorrelation means:
- Tomorrow's return is slightly predictable from today's return
- Standard errors on correlation estimates are biased downward
- Your model's predictions are correlated with each other
Serial Correlation in Features
Most financial features (RSI, momentum, volatility) are computed from price series, so they inherit:
- Autocorrelation from price momentum
- Clustering of high-volatility periods
- Regime persistence
Newey-West Standard Errors
When features and targets have serial correlation, standard OLS standard errors are biased. Use Newey-West adjustment:
from scipy import stats
import numpy as np
import pandas as pd
def newey_west_tstat(
residuals: np.ndarray,
features: np.ndarray,
lags: int = 5
) -> pd.DataFrame:
"""
Calculate Newey-West corrected t-statistics for regression coefficients.
Accounts for serial correlation in residuals.
Args:
residuals: OLS residuals
features: Regression features (shape: n_samples × n_features)
lags: Number of lags for Newey-West correction
Returns:
DataFrame with coefficients and Newey-West t-statistics
"""
n = len(residuals)
k = features.shape[1]
# Standard OLS variance-covariance matrix
XtX_inv = np.linalg.inv(features.T @ features)
# Long-run variance (Newey-West)
s0 = np.sum(residuals**2) / n
# Lagged covariances
lrv = s0.copy()
for lag in range(1, lags + 1):
weight = 1 - (lag / (lags + 1)) # Bartlett weights
lagged_cov = np.sum(residuals[:-lag] * residuals[lag:]) / n
lrv += 2 * weight * lagged_cov
# Newey-West variance-covariance matrix
nw_vcov = (lrv / n) * XtX_inv
# Standard errors and t-statistics
nw_se = np.sqrt(np.diag(nw_vcov))
coefficients = XtX_inv @ features.T @ residuals # (Simplified)
t_stats = coefficients / nw_se
return pd.DataFrame({
'coefficient': coefficients,
'nw_std_error': nw_se,
't_statistic': t_stats,
'p_value': 2 * (1 - stats.t.cdf(np.abs(t_stats), n - k))
})Summary: Why Standard ML Pipelines Fail#
Standard machine learning assumes:
- IID samples → Financial data has serial correlation
- Stationarity → Markets change regimes
- Exogenous features → Your model's actions affect prices
- R² matters → In finance, R² of 0.025 is excellent
- Random splits are valid → Destroys temporal structure
Each of these assumptions is violated in financial ML. Standard pipelines give you overly optimistic backtests that fail in production by 20-50x.
The solution: walk-forward validation, respecting causality, understanding feedback loops, and adjusting for multiple testing.
Module 15.2: Financial Data Labeling#
Standard supervised learning requires labels: (feature, target) pairs. In finance, defining targets is non-trivial and the labeling methodology determines strategy behavior.
Fixed-Horizon Labeling: The Naive Approach#
The simplest labeling method predicts returns over a fixed horizon:
where is the forecast horizon (typically 1-5 days).
Implementation
def fixed_horizon_labels(
prices: pd.Series,
horizon: int = 5,
include_costs: bool = True,
transaction_cost: float = 0.001
) -> np.ndarray:
"""
Create fixed-horizon forward return labels.
Args:
prices: Series of prices indexed by timestamp
horizon: Number of periods forward to predict
include_costs: Whether to subtract transaction costs
transaction_cost: Assumed one-way cost (0.1% = 0.001)
Returns:
Array of forward returns
"""
# Forward returns
forward_prices = prices.shift(-horizon)
forward_returns = (forward_prices / prices) - 1
if include_costs:
# Cost to enter and exit positions
forward_returns = forward_returns - 2 * transaction_cost
return forward_returns.values
# Example
prices = pd.Series([100, 101, 102, 103, 105, 107, 106, 108, 110, 112])
labels = fixed_horizon_labels(prices, horizon=3, include_costs=True)
print(f"Price series: {prices.values}")
print(f"Labels (3-day forward returns, after costs): {labels}")
# Output:
# Price series: [100 101 102 103 105 107 106 108 110 112]
# Labels (3-day forward returns, after costs): [0.027 0.029 0.030 0.037 0.033 0.025 0.030 nan nan nan]Problems with Fixed-Horizon Labeling
-
Ignores volatility: In calm markets, your model makes small trades. In volatile markets, it makes identical-sized trades despite higher risk.
-
No risk management: The label assumes you hold the position for exactly
horizonperiods, regardless of interim drawdowns. -
Asymmetric payoff: You take infinite loss if the stock goes bankrupt, but capped gains.
-
Ignores liquidity: You might have predictive signal but be unable to exit at predicted price due to slippage.
Triple-Barrier Labeling: Risk-Aware Labeling#
López de Prado (Advances in Financial Machine Learning) proposed triple-barrier labeling, which is the gold standard in institutional ML-driven trading.
The idea: instead of predicting at fixed horizon, predict whether the position will first hit:
- Profit target ()
- Stop loss ()
- Time limit expiry ()
This creates asymmetric labels that reflect real trading decisions.
Mathematical Formulation
Given position entry at time and price , define:
The label is:
Implementation
from typing import Tuple, List
import numpy as np
import pandas as pd
def triple_barrier_labels(
prices: pd.Series,
pt_barrier: float = 0.02,
sl_barrier: float = 0.02,
max_periods: int = 20
) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
"""
Generate triple-barrier labels for a price series.
Args:
prices: Series of OHLC data with columns ['high', 'low', 'close']
pt_barrier: Profit target as fraction of entry price (0.02 = 2%)
sl_barrier: Stop loss as fraction of entry price (0.02 = 2%)
max_periods: Maximum holding period before expiry
Returns:
Tuple of (labels, exit_times, profits)
"""
n = len(prices)
labels = np.zeros(n)
exit_times = np.zeros(n, dtype=int)
profits = np.zeros(n)
for i in range(n - max_periods):
entry_price = prices.iloc[i]
# Define barriers
pt_level = entry_price * (1 + pt_barrier)
sl_level = entry_price * (1 - sl_barrier)
# Look forward over max_periods
future_prices = prices.iloc[i:i + max_periods]
# Find first touch of each barrier
pt_touch = (future_prices >= pt_level).idxmax()
sl_touch = (future_prices <= sl_level).idxmax()
# Determine which barrier hit first
pt_idx = future_prices.index.get_loc(pt_touch) if pt_touch > i else max_periods
sl_idx = future_prices.index.get_loc(sl_touch) if sl_touch > i else max_periods
if pt_idx < sl_idx and pt_idx < max_periods:
# Profit target hit first
labels[i] = 1
exit_times[i] = pt_idx
profits[i] = pt_barrier
elif sl_idx < pt_idx and sl_idx < max_periods:
# Stop loss hit first
labels[i] = -1
exit_times[i] = sl_idx
profits[i] = -sl_barrier
else:
# Neither hit: use return at time limit
final_price = future_prices.iloc[min(max_periods - 1, len(future_prices) - 1)]
final_return = (final_price / entry_price) - 1
labels[i] = np.sign(final_return)
exit_times[i] = min(max_periods - 1, len(future_prices) - 1)
profits[i] = final_return
return labels, exit_times, profits
# Example with realistic data
np.random.seed(42)
# Simulate price with autocorrelation and volatility clustering
returns = np.random.normal(0.0001, 0.015, 500)
prices = pd.Series(100 * np.exp(np.cumsum(returns)))
labels, exit_times, profits = triple_barrier_labels(
prices=prices,
pt_barrier=0.02,
sl_barrier=0.01,
max_periods=20
)
print(f"Total labels generated: {len(labels[:-20])}")
print(f"Profit target hits: {(labels == 1).sum()}")
print(f"Stop loss hits: {(labels == -1).sum()}")
print(f"Time expiry (neutral): {(labels == 0).sum()}")
print(f"Average exit time: {exit_times[exit_times > 0].mean():.1f} periods")
print(f"Average profit: {profits[profits != 0].mean():.4f}")Advantages of Triple-Barrier Labeling
- Risk-aware: Incorporates your risk tolerance (stop loss level)
- Asymmetric: Doesn't assume symmetric return distribution
- Realistic: Models actual trading exit rules
- Flexible: Adjust barriers for different market conditions
- Adaptive: Can use volatility-adjusted barriers:
Meta-Labeling: Predicting Confidence, Not Direction#
Meta-labeling (also from López de Prado) decouples:
- Primary model: Generates buy/sell signals
- Meta-model: Predicts confidence in primary signal
This is more robust than training a single end-to-end classifier.
The Meta-Labeling Framework
Primary Model Output: "STRONG BUY" (α = 0.3)
↓
Meta-Model Features: {Price momentum, volatility, correlation with SPY, ...}
↓
Meta-Label: "HIGH CONFIDENCE" or "LOW CONFIDENCE"
↓
Final Decision: Position size = base_position_size × meta_confidenceMathematical Formulation
Let be the primary model's signal (e.g., expected alpha):
The primary model generates a position:
The meta-model learns:
where are meta-features that predict whether the primary signal is correct:
The final position becomes:
Implementation
from sklearn.ensemble import RandomForestClassifier
from typing import Tuple
import numpy as np
import pandas as pd
class MetaLabeler:
"""
Meta-labeling system for position sizing based on primary model confidence.
"""
def __init__(self, lookback: int = 5, pt_barrier: float = 0.02):
"""
Args:
lookback: Number of periods for meta-features
pt_barrier: Profit threshold for meta-label
"""
self.lookback = lookback
self.pt_barrier = pt_barrier
self.meta_model = RandomForestClassifier(n_estimators=100, max_depth=10)
def create_meta_labels(
self,
prices: pd.Series,
primary_positions: np.ndarray
) -> np.ndarray:
"""
Create binary labels for whether primary signal was correct.
Args:
prices: Series of prices
primary_positions: Array of primary model positions (+1, -1, 0)
Returns:
Array of binary meta-labels
"""
meta_labels = np.zeros(len(prices))
for i in range(len(prices) - 5):
entry_price = prices.iloc[i]
position = primary_positions[i]
# Check if position was profitable
exit_price = prices.iloc[i + 5]
profit = (exit_price - entry_price) * position / entry_price
# Label: 1 if profitable, 0 otherwise
meta_labels[i] = 1 if profit > self.pt_barrier else 0
return meta_labels
def create_meta_features(
self,
prices: pd.Series,
returns: pd.Series,
volume: pd.Series
) -> np.ndarray:
"""
Create meta-features that predict primary signal accuracy.
Args:
prices: Series of prices
returns: Series of returns
volume: Series of trading volume
Returns:
Array of shape (n_samples, n_features)
"""
n = len(prices)
features = []
# Feature 1: Recent momentum
momentum = returns.rolling(self.lookback).mean().values
features.append(momentum)
# Feature 2: Recent volatility
volatility = returns.rolling(self.lookback).std().values
features.append(volatility)
# Feature 3: Volume trend
volume_ma = volume.rolling(self.lookback).mean().values
volume_ratio = volume.values / (volume_ma + 1e-8)
features.append(volume_ratio)
# Feature 4: Mean reversion signal
price_ma = prices.rolling(self.lookback).mean().values
mean_reversion = (prices.values - price_ma) / (price_ma * volatility + 1e-8)
features.append(mean_reversion)
# Feature 5: Return autocorrelation
autocorr = np.array([
returns.iloc[max(0, i-self.lookback):i].autocorr()
for i in range(n)
])
features.append(autocorr)
return np.column_stack(features)
def fit(
self,
prices: pd.Series,
returns: pd.Series,
volume: pd.Series,
primary_positions: np.ndarray
) -> 'MetaLabeler':
"""
Train meta-model to predict primary signal accuracy.
Args:
prices: Historical prices
returns: Historical returns
volume: Historical volume
primary_positions: Primary model positions
Returns:
Self for chaining
"""
# Create labels
meta_labels = self.create_meta_labels(prices, primary_positions)
# Create features
meta_features = self.create_meta_features(prices, returns, volume)
# Train meta-model
valid_idx = ~np.isnan(meta_features).any(axis=1) & (meta_labels != 0)
self.meta_model.fit(
meta_features[valid_idx],
meta_labels[valid_idx]
)
return self
def predict_confidence(
self,
prices: pd.Series,
returns: pd.Series,
volume: pd.Series
) -> np.ndarray:
"""
Predict confidence in current signal.
Returns:
Array of confidence scores [0, 1]
"""
meta_features = self.create_meta_features(prices, returns, volume)
confidence = self.meta_model.predict_proba(meta_features)[:, 1]
return confidenceExample Usage
# Simulate data
np.random.seed(42)
n = 500
prices = pd.Series(100 * np.exp(np.cumsum(np.random.normal(0.0001, 0.015, n))))
returns = prices.pct_change()
volume = pd.Series(np.random.normal(1e6, 0.2e6, n))
# Primary model (simple momentum)
momentum_signal = returns.rolling(5).mean()
primary_positions = np.where(momentum_signal > 0, 1, -1)
# Meta-labeling
meta_labeler = MetaLabeler(lookback=5)
meta_labeler.fit(prices, returns, volume, primary_positions)
# Predict confidence
confidence = meta_labeler.predict_confidence(prices, returns, volume)
# Position sizing
base_position_size = 100
final_positions = primary_positions * confidence * base_position_size
print(f"Primary position std: {np.abs(primary_positions).std():.2f}")
print(f"Final position std: {final_positions.std():.2f}")
print(f"Confidence mean: {confidence.mean():.3f}")
print(f"Position sizing reduces large bets by: {(1 - confidence).mean():.1%}")Sample Weighting: Addressing Label Redundancy#
Not all samples are equally informative. Samples with unique information should receive higher weight in training.
The Uniqueness Problem
Consecutive time periods have very similar features:
- If today's features are and tomorrow's are
- These samples are nearly identical, so training on both is redundant
- Your model overwrites the first sample's information with the second
- This creates false confidence in your model's generalization
Uniqueness Measure
from typing import Tuple
import numpy as np
import pandas as pd
def compute_sample_uniqueness(
features: np.ndarray,
lookback: int = 10
) -> np.ndarray:
"""
Compute uniqueness weight for each sample based on feature distance.
Samples that are feature-wise different from nearby samples
receive higher uniqueness weights.
Args:
features: Array of shape (n_samples, n_features)
lookback: Window for computing uniqueness
Returns:
Array of uniqueness weights
"""
n = features.shape[0]
uniqueness = np.zeros(n)
for i in range(lookback, n):
# Features before and after this sample
past_features = features[i - lookback:i]
future_features = features[i + 1:min(i + 1 + lookback, n)]
if len(past_features) == 0 or len(future_features) == 0:
uniqueness[i] = 1.0
continue
# Euclidean distance to nearest neighbor
current_feature = features[i]
past_distances = np.linalg.norm(
past_features - current_feature,
axis=1
)
future_distances = np.linalg.norm(
future_features - current_feature,
axis=1
)
all_distances = np.concatenate([past_distances, future_distances])
min_distance = np.min(all_distances) if len(all_distances) > 0 else 1.0
# Uniqueness: inverse of nearest neighbor distance
# Samples far from neighbors (unique) get high weight
uniqueness[i] = 1.0 / (1.0 + min_distance)
return uniqueness / uniqueness.mean() # Normalize
# Example
features = np.array([
[0.5, 0.2, 0.1],
[0.51, 0.21, 0.11], # Very similar to previous
[0.52, 0.22, 0.12], # Very similar to previous
[0.8, 0.7, 0.6], # Different from recent
[0.81, 0.71, 0.61], # Very similar to previous
])
uniqueness = compute_sample_uniqueness(features)
print("Feature uniqueness weights:")
print(uniqueness)
# Samples 2 and 4 (isolated) get higher weights than 1, 3, 5 (similar to neighbors)Average Uniqueness: Sequential Bootstrap Weights
The most sophisticated approach is average uniqueness from López de Prado's research.
The idea:
- Find the formation time of each label (when barrier was first touched)
- Each sample with a label spans a time interval
- Overlapping labels create redundancy
- Weight samples inversely to their overlap
def compute_average_uniqueness_weights(
label_start_times: np.ndarray,
label_end_times: np.ndarray
) -> np.ndarray:
"""
Compute average uniqueness weights for sequential samples.
Handles the case where sample labels overlap in time.
Overlapping samples are less unique and receive lower weight.
Args:
label_start_times: Array of label formation start times
label_end_times: Array of label formation end times
Returns:
Array of average uniqueness weights
"""
n = len(label_start_times)
avg_uniqueness = np.ones(n)
for i in range(n):
# Find all other labels that overlap with label i
label_i_start = label_start_times[i]
label_i_end = label_end_times[i]
# Count overlapping labels
overlaps = 0
for j in range(n):
if i == j:
continue
label_j_start = label_start_times[j]
label_j_end = label_end_times[j]
# Check overlap
overlap_start = max(label_i_start, label_j_start)
overlap_end = min(label_i_end, label_j_end)
if overlap_start < overlap_end:
# Labels overlap
overlap_fraction = (overlap_end - overlap_start) / (label_i_end - label_i_start)
overlaps += overlap_fraction
# Average uniqueness: inverse of average overlap
avg_uniqueness[i] = 1.0 / (1.0 + overlaps / max(n, 1))
return avg_uniqueness / avg_uniqueness.mean()
# Example with triple-barrier labels
label_starts = np.array([0, 2, 5, 8, 10, 12, 15])
label_ends = np.array([5, 7, 10, 13, 15, 17, 20])
avg_unique = compute_average_uniqueness_weights(label_starts, label_ends)
print("Average uniqueness weights:")
for i, (start, end, weight) in enumerate(zip(label_starts, label_ends, avg_unique)):
print(f"Label {i} [{start}-{end}]: weight = {weight:.3f}")Using Weights in Training
from sklearn.ensemble import RandomForestRegressor
import numpy as np
def train_weighted_model(
features: np.ndarray,
targets: np.ndarray,
sample_weights: np.ndarray
) -> RandomForestRegressor:
"""
Train ML model with sample weights accounting for label uniqueness.
Args:
features: Feature matrix
targets: Target labels
sample_weights: Weights for each sample
Returns:
Trained model
"""
# Normalize weights to valid range
sample_weights = np.clip(sample_weights, 1e-6, 1e6)
sample_weights = sample_weights / sample_weights.mean()
# Train with weights
model = RandomForestRegressor(
n_estimators=100,
max_depth=15,
min_samples_leaf=5
)
model.fit(
features,
targets,
sample_weight=sample_weights
)
return modelHow Labeling Methodology Determines Strategy Character#
Different labeling approaches create fundamentally different strategies:
| Aspect | Fixed-Horizon | Triple-Barrier | Meta-Label |
|---|---|---|---|
| Exit condition | Time | Profit/loss | Confidence decay |
| Risk management | None | Explicit stops | Adaptive sizing |
| Training focus | Direction prediction | Barrier hitting | Signal confidence |
| Resulting strategy | Trend-following | Mean reversion | Adaptive positioning |
| Typical Sharpe | 0.5-1.5 | 1.0-2.5 | 1.5-3.0 |
Example: Same Model, Different Labels
def compare_labeling_strategies(
prices: pd.Series,
features: np.ndarray,
model=None
) -> dict:
"""
Compare performance of same features under different labeling schemes.
"""
if model is None:
from sklearn.ensemble import RandomForestRegressor
model = RandomForestRegressor(n_estimators=50)
results = {}
# Method 1: Fixed-horizon (5 days)
fh_labels = (prices.shift(-5) / prices - 1).values
mask = ~np.isnan(fh_labels)
model.fit(features[mask], fh_labels[mask])
fh_preds = model.predict(features[mask])
fh_returns = fh_preds * fh_labels[mask]
results['fixed_horizon'] = {
'annual_sharpe': fh_returns.mean() / fh_returns.std() * np.sqrt(252),
'win_rate': (fh_returns > 0).sum() / len(fh_returns)
}
# Method 2: Triple-barrier
tb_labels, _, _ = triple_barrier_labels(
prices, pt_barrier=0.02, sl_barrier=0.02, max_periods=20
)
mask = tb_labels != 0
model.fit(features[mask], tb_labels[mask])
tb_preds = model.predict(features[mask])
tb_returns = np.sign(tb_preds) * tb_labels[mask] * 0.02 # Binary wins/losses
results['triple_barrier'] = {
'annual_sharpe': tb_returns.mean() / tb_returns.std() * np.sqrt(252),
'win_rate': (tb_returns > 0).sum() / len(tb_returns)
}
return resultsSummary: The Takeaway#
Financial ML is different because:
- Weak signals are normal: R² of 0.025 is excellent, not a bug
- Stationarity is fiction: Markets change; walk-forward validation is mandatory
- Your actions matter: Feedback loops contaminate training data
- Time matters: Serial correlation requires respecting causality
- Labels determine destiny: Your labeling method shapes strategy behavior
The practitioners who succeed in quant trading are those who internalize these differences and build systems that respect them. Every one of these principles—from walk-forward validation to triple-barrier labeling—has emerged from painful failures by traders who assumed financial ML works like computer vision.
Use that hard-won knowledge: respect the financial domain.
Key Code Snippets: Quick Reference#
Walk-Forward Validation Template
for train_start in range(0, len(data) - train_size - test_size, step_size):
train_end = train_start + train_size
test_end = train_end + test_size
# Fit scaler on training data ONLY
X_train_scaled = scaler.fit_transform(X[train_start:train_end])
# Test on future data
X_test_scaled = scaler.transform(X[train_end:test_end])Triple-Barrier Label
if max_price >= entry * (1 + profit_target):
label = 1
elif min_price <= entry * (1 - stop_loss):
label = -1
else:
label = sign((exit_price - entry) / entry)Average Uniqueness Weight
overlaps = sum(overlap_fraction for all overlapping labels)
avg_uniqueness[i] = 1.0 / (1.0 + overlaps)References and Further Reading#
- López de Prado, M. (2018). Advances in Financial Machine Learning. Wiley.
- Easley, D., López de Prado, M., & O'Hara, M. (2016). "Flow toxicity and liquidity in a high-frequency world." Review of Financial Studies, 29(2), 1455-1490.
- Nolte, I. (2014). "How do financial markets react to central bank communications?" Journal of International Money and Finance, 48, 131-148.
- Leitch, G., & Tanner, J. (1991). "Evaluation of forecasts of the stochastic discount factor." International Economic Review, 663-689.