From Zero to Quant · Chapter 28
Continuous Improvement
Alpha decay, strategy rotation, research pipeline, scaling up
Introduction#
You've built a trading system. It works. You're profitable. But here's the uncomfortable truth: in quantitative finance, standing still is moving backward. Markets evolve, competition intensifies, and any edge you've discovered is being hunted by thousands of other quants simultaneously.
This chapter is about the unglamorous, deliberate work of continuous improvement—the research pipeline that keeps your system relevant, the systematic evolution of your infrastructure, and the long-term perspective required to build something that lasts. This is where successful quant traders differ from failed ones: not in one brilliant insight, but in the disciplined compounding of small improvements.
We'll cover three critical areas:
- Research Pipeline — How to systematically discover, test, and validate new alpha sources
- System Evolution — How to scale, integrate new data, retrain models, and expand strategies
- The Long Game — How to build something enduring, and what quant firms actually look for
By the end of this chapter, you'll have frameworks, code, and mindsets for sustainable competitive advantage.
Module 28.1: Research Pipeline
The Science of Alpha Discovery#
Alpha research is not random experimentation. It's a disciplined scientific process: hypothesis generation → testing → validation → deployment → monitoring → retirement.
Hypothesis Generation Framework#
The most common mistake in quant research is fishing for significance. You generate 1,000 ideas, test them all, and the 50 that work by random chance you declare as "alpha." This is statistical nonsense.
Good alpha research starts with economically motivated hypotheses derived from:
- Market Microstructure — How do information, order flow, and inventory dynamics create temporary mispricings?
- Behavioral Finance — How do systematic human biases create exploitable patterns?
- Fundamental Relationships — What economic relationships should hold, and where do they break down?
- Data Anomalies — What patterns appear in data that shouldn't exist under efficient markets?
For your NSE trading system, concrete hypotheses might look like:
- H1 (Microstructure): "High-frequency institutional transactions create temporary volume clusters that decay within 15 minutes. We can predict post-cluster price movement from order flow imbalance."
- H2 (Behavioral): "Retail investors overpay for stocks mentioned on financial news. We can short these overvalued positions for 2-3 trading days."
- H3 (Fundamental): "NSE mid-cap stocks with improving operating leverage outperform their sector. The signal decays as analyst coverage increases."
Each hypothesis must be:
- Economically sensible — Why should this exist? What prevents immediate arbitrage?
- Testable — Can we measure it? Do we have the required data?
- Time-bound — How long does the effect last?
Research Log Framework#
Maintaining a research log is non-negotiable. This is your institutional memory. Without it, you'll:
- Re-test ideas you've already proven worthless
- Forget how you built previous successful signals
- Have no systematic way to evaluate your research productivity
- Lack documentation when considering moving to a fund (they'll ask for this)
Here's a production-quality research log system:
from dataclasses import dataclass
from datetime import datetime
from typing import Dict, List, Optional, Any
from enum import Enum
import json
from pathlib import Path
import pandas as pd
class HypothesisStatus(Enum):
"""Status of a research hypothesis."""
IDEATION = "ideation"
BACKTESTING = "backtesting"
OUT_OF_SAMPLE = "out_of_sample"
PRODUCTION = "production"
DEPRECATED = "deprecated"
ABANDONED = "abandoned"
@dataclass
class ResearchEntry:
"""Single entry in the research log."""
hypothesis_id: str
hypothesis_statement: str
status: HypothesisStatus
created_date: datetime
# Hypothesis details
economic_rationale: str
data_sources: List[str]
testing_period_start: datetime
testing_period_end: datetime
# Results
sharpe_ratio: Optional[float] = None
hit_rate: Optional[float] = None
max_drawdown: Optional[float] = None
out_of_sample_sharpe: Optional[float] = None
# Implementation
model_filename: Optional[str] = None
signal_formula: Optional[str] = None
latency_ms: Optional[float] = None
# Lifecycle
deployed_date: Optional[datetime] = None
deprecated_date: Optional[datetime] = None
deprecation_reason: Optional[str] = None
# Notes
notes: str = ""
tags: List[str] = None
def __post_init__(self):
if self.tags is None:
self.tags = []
class ResearchLog:
"""Production research log management system."""
def __init__(self, log_path: str = "research_log.jsonl"):
"""
Initialize research log.
Args:
log_path: Path to research log file (JSONL format)
"""
self.log_path = Path(log_path)
self.entries: Dict[str, ResearchEntry] = {}
self._load_log()
def _load_log(self) -> None:
"""Load existing log from disk."""
if self.log_path.exists():
with open(self.log_path, 'r') as f:
for line in f:
entry_dict = json.loads(line)
# Convert datetime strings back to datetime objects
for date_field in ['created_date', 'testing_period_start',
'testing_period_end', 'deployed_date',
'deprecated_date']:
if entry_dict.get(date_field):
entry_dict[date_field] = datetime.fromisoformat(
entry_dict[date_field]
)
# Convert status string back to enum
entry_dict['status'] = HypothesisStatus(entry_dict['status'])
entry = ResearchEntry(**entry_dict)
self.entries[entry.hypothesis_id] = entry
def add_hypothesis(
self,
hypothesis_id: str,
hypothesis_statement: str,
economic_rationale: str,
data_sources: List[str],
testing_period_start: datetime,
testing_period_end: datetime,
tags: List[str] = None
) -> ResearchEntry:
"""
Add new hypothesis to research log.
Args:
hypothesis_id: Unique identifier (e.g., 'NSE_MICROSTRUCTURE_001')
hypothesis_statement: Clear statement of hypothesis
economic_rationale: Why this effect should exist
data_sources: List of required data sources
testing_period_start: Start of test period
testing_period_end: End of test period
tags: Optional tags for categorization
Returns:
Created ResearchEntry
"""
entry = ResearchEntry(
hypothesis_id=hypothesis_id,
hypothesis_statement=hypothesis_statement,
status=HypothesisStatus.IDEATION,
created_date=datetime.now(),
economic_rationale=economic_rationale,
data_sources=data_sources,
testing_period_start=testing_period_start,
testing_period_end=testing_period_end,
tags=tags or []
)
self.entries[hypothesis_id] = entry
self._save_entry(entry)
return entry
def update_backtest_results(
self,
hypothesis_id: str,
sharpe_ratio: float,
hit_rate: float,
max_drawdown: float,
model_filename: str,
signal_formula: str,
notes: str = ""
) -> None:
"""
Update hypothesis with backtest results.
Args:
hypothesis_id: ID of hypothesis being tested
sharpe_ratio: Sharpe ratio from backtest
hit_rate: Win rate / hit rate
max_drawdown: Maximum drawdown
model_filename: Path to trained model file
signal_formula: Mathematical formula or code expression
notes: Additional observations
"""
if hypothesis_id not in self.entries:
raise ValueError(f"Hypothesis {hypothesis_id} not found")
entry = self.entries[hypothesis_id]
entry.status = HypothesisStatus.BACKTESTING
entry.sharpe_ratio = sharpe_ratio
entry.hit_rate = hit_rate
entry.max_drawdown = max_drawdown
entry.model_filename = model_filename
entry.signal_formula = signal_formula
entry.notes = notes
self._save_entry(entry)
def update_oos_results(
self,
hypothesis_id: str,
oos_sharpe: float,
notes: str = ""
) -> None:
"""
Update hypothesis with out-of-sample results.
Args:
hypothesis_id: ID of hypothesis
oos_sharpe: Sharpe ratio on unseen data
notes: Observations about OOS performance
"""
if hypothesis_id not in self.entries:
raise ValueError(f"Hypothesis {hypothesis_id} not found")
entry = self.entries[hypothesis_id]
entry.status = HypothesisStatus.OUT_OF_SAMPLE
entry.out_of_sample_sharpe = oos_sharpe
entry.notes = notes
self._save_entry(entry)
def deploy_to_production(
self,
hypothesis_id: str,
latency_ms: float = None,
notes: str = ""
) -> None:
"""
Mark hypothesis as deployed to production.
Args:
hypothesis_id: ID of hypothesis
latency_ms: Latency to calculate signal in milliseconds
notes: Deployment notes
"""
if hypothesis_id not in self.entries:
raise ValueError(f"Hypothesis {hypothesis_id} not found")
entry = self.entries[hypothesis_id]
entry.status = HypothesisStatus.PRODUCTION
entry.deployed_date = datetime.now()
entry.latency_ms = latency_ms
entry.notes = notes
self._save_entry(entry)
def retire_signal(
self,
hypothesis_id: str,
reason: str,
notes: str = ""
) -> None:
"""
Mark hypothesis as deprecated (retired from production).
Args:
hypothesis_id: ID of hypothesis
reason: Reason for retirement
notes: Additional context
"""
if hypothesis_id not in self.entries:
raise ValueError(f"Hypothesis {hypothesis_id} not found")
entry = self.entries[hypothesis_id]
entry.status = HypothesisStatus.DEPRECATED
entry.deprecated_date = datetime.now()
entry.deprecation_reason = reason
entry.notes = notes
self._save_entry(entry)
def _save_entry(self, entry: ResearchEntry) -> None:
"""Save individual entry to log file."""
# Prepare entry for JSON serialization
entry_dict = {
'hypothesis_id': entry.hypothesis_id,
'hypothesis_statement': entry.hypothesis_statement,
'status': entry.status.value,
'created_date': entry.created_date.isoformat(),
'economic_rationale': entry.economic_rationale,
'data_sources': entry.data_sources,
'testing_period_start': entry.testing_period_start.isoformat(),
'testing_period_end': entry.testing_period_end.isoformat(),
'sharpe_ratio': entry.sharpe_ratio,
'hit_rate': entry.hit_rate,
'max_drawdown': entry.max_drawdown,
'out_of_sample_sharpe': entry.out_of_sample_sharpe,
'model_filename': entry.model_filename,
'signal_formula': entry.signal_formula,
'latency_ms': entry.latency_ms,
'deployed_date': entry.deployed_date.isoformat() if entry.deployed_date else None,
'deprecated_date': entry.deprecated_date.isoformat() if entry.deprecated_date else None,
'deprecation_reason': entry.deprecation_reason,
'notes': entry.notes,
'tags': entry.tags
}
# Append to JSONL file
with open(self.log_path, 'a') as f:
f.write(json.dumps(entry_dict) + '\n')
def get_summary_df(self) -> pd.DataFrame:
"""
Get research log as DataFrame for analysis.
Returns:
DataFrame with one row per hypothesis
"""
rows = []
for entry in self.entries.values():
rows.append({
'hypothesis_id': entry.hypothesis_id,
'status': entry.status.value,
'sharpe_ratio': entry.sharpe_ratio,
'oos_sharpe': entry.out_of_sample_sharpe,
'hit_rate': entry.hit_rate,
'max_drawdown': entry.max_drawdown,
'deployed_date': entry.deployed_date,
'deprecated_date': entry.deprecated_date,
'tags': ', '.join(entry.tags)
})
return pd.DataFrame(rows)
def get_active_hypotheses(self) -> List[ResearchEntry]:
"""Get all currently deployed hypotheses."""
return [e for e in self.entries.values()
if e.status == HypothesisStatus.PRODUCTION]
def get_by_status(self, status: HypothesisStatus) -> List[ResearchEntry]:
"""Get all hypotheses with given status."""
return [e for e in self.entries.values() if e.status == status]A/B Testing in Production#
The most powerful and dangerous thing you can do is test new signals against live trading. Dangerous because you're risking real money. Powerful because real market data beats all backtests.
A/B testing methodology for NSE trading:
from dataclasses import dataclass
from typing import Tuple, List
import numpy as np
from scipy import stats
@dataclass
class ABTestResults:
"""Results from A/B test comparing two signals."""
treatment_returns: np.ndarray # Returns from new signal
control_returns: np.ndarray # Returns from existing signal
# Statistical tests
t_statistic: float
p_value: float
mean_difference: float
confidence_interval_95: Tuple[float, float]
# Economic significance
sharpe_treatment: float
sharpe_control: float
sharpe_difference: float
# Practical significance
win_rate_treatment: float
win_rate_control: float
# Sample sizes
treatment_trades: int
control_trades: int
# Recommendation
is_significant: bool
recommendation: str
class ProductionABTester:
"""A/B testing framework for production signals."""
def __init__(self, risk_free_rate: float = 0.06):
"""
Initialize A/B tester.
Args:
risk_free_rate: Annual risk-free rate for Sharpe calculation
"""
self.risk_free_rate = risk_free_rate
def run_ab_test(
self,
treatment_returns: List[float],
control_returns: List[float],
confidence_level: float = 0.95,
min_trades_for_significance: int = 50
) -> ABTestResults:
"""
Run statistical A/B test comparing two signals.
Mathematical Framework:
We conduct a two-sample t-test with null hypothesis:
H0: μ_treatment = μ_control
Test statistic:
t = (μ_treatment - μ_control) / sqrt(σ_treatment²/n_treatment + σ_control²/n_control)
Under H0, t ~ t_distribution(df) where df depends on sample sizes.
Reject H0 if |t| > t_critical (typically α = 0.05, so |t| > 1.96 for large n)
Args:
treatment_returns: Daily/trade returns from new signal
control_returns: Daily/trade returns from existing signal
confidence_level: Confidence level for significance (0.95 = 95%)
min_trades_for_significance: Minimum trades to consider result significant
Returns:
ABTestResults with statistical and economic analysis
"""
treatment_arr = np.array(treatment_returns)
control_arr = np.array(control_returns)
# Statistical test
t_stat, p_val = stats.ttest_ind(treatment_arr, control_arr, equal_var=False)
# Confidence interval for difference in means
mean_diff = treatment_arr.mean() - control_arr.mean()
se_diff = np.sqrt(
(treatment_arr.std() ** 2 / len(treatment_arr)) +
(control_arr.std() ** 2 / len(control_arr))
)
alpha = 1 - confidence_level
t_critical = stats.t.ppf(1 - alpha/2, df=len(treatment_arr) + len(control_arr) - 2)
ci_lower = mean_diff - t_critical * se_diff
ci_upper = mean_diff + t_critical * se_diff
# Sharpe ratios (assuming daily returns, annualize)
sharpe_treatment = self._calculate_sharpe(treatment_arr)
sharpe_control = self._calculate_sharpe(control_arr)
sharpe_diff = sharpe_treatment - sharpe_control
# Win rates
win_rate_treatment = (treatment_arr > 0).sum() / len(treatment_arr)
win_rate_control = (control_arr > 0).sum() / len(control_arr)
# Significance determination
is_significant = (
p_val < (1 - confidence_level) and
len(treatment_arr) >= min_trades_for_significance and
len(control_arr) >= min_trades_for_significance
)
# Generate recommendation
if not is_significant:
recommendation = (
f"NOT SIGNIFICANT: Need more trades. "
f"p-value={p_val:.4f} (threshold={1-confidence_level:.4f})"
)
elif mean_diff > 0 and sharpe_treatment > sharpe_control:
recommendation = (
f"ADOPT NEW SIGNAL: "
f"Sharpe improvement of {sharpe_diff:.3f}, "
f"p-value={p_val:.4f}"
)
elif mean_diff < 0 or sharpe_treatment < sharpe_control:
recommendation = (
f"KEEP EXISTING: "
f"New signal underperformed by Sharpe {-sharpe_diff:.3f}"
)
else:
recommendation = "MARGINAL: Signals perform similarly"
return ABTestResults(
treatment_returns=treatment_arr,
control_returns=control_arr,
t_statistic=t_stat,
p_value=p_val,
mean_difference=mean_diff,
confidence_interval_95=(ci_lower, ci_upper),
sharpe_treatment=sharpe_treatment,
sharpe_control=sharpe_control,
sharpe_difference=sharpe_diff,
win_rate_treatment=win_rate_treatment,
win_rate_control=win_rate_control,
treatment_trades=len(treatment_arr),
control_trades=len(control_arr),
is_significant=is_significant,
recommendation=recommendation
)
def _calculate_sharpe(self, returns: np.ndarray) -> float:
"""
Calculate annualized Sharpe ratio.
Sharpe = (mean_return - rf) / std_return * sqrt(252)
Args:
returns: Array of daily returns
Returns:
Annualized Sharpe ratio
"""
excess_returns = returns - (self.risk_free_rate / 252)
return np.mean(excess_returns) / np.std(excess_returns) * np.sqrt(252)
def calculate_sample_size(
self,
expected_effect_size: float,
power: float = 0.80,
alpha: float = 0.05
) -> int:
"""
Calculate required sample size for detecting effect.
Using power analysis (Cohen's approach):
n = 2 * ((z_alpha + z_beta) / effect_size)²
Where:
- z_alpha is critical value for type I error (typically 1.96 for α=0.05)
- z_beta is critical value for type II error (typically 0.84 for power=0.80)
- effect_size is (μ_treatment - μ_control) / σ (Cohen's d)
Args:
expected_effect_size: Expected difference / std (Cohen's d)
power: Power of test (1 - Type II error probability)
alpha: Significance level
Returns:
Minimum number of samples per group
"""
z_alpha = stats.norm.ppf(1 - alpha/2) # Two-tailed
z_beta = stats.norm.ppf(power)
n = 2 * ((z_alpha + z_beta) / expected_effect_size) ** 2
return int(np.ceil(n))When to Retire a Signal#
This is the hardest part for traders: admitting defeat. Your signal is decaying. It needs to be retired.
Criteria for signal retirement:
-
Fundamental Decay — Does the out-of-sample performance materially differ from backtested performance?
Define: If OOS Sharpe < (Backtest Sharpe - 1.0), investigate retirement.
-
Market Evolution — Has the market structure changed, eliminating the edge?
Common causes:
- Retail trading volumes (GSM margins) becoming negligible
- AI trading becoming dominant in your signal's timeframe
- Regulatory changes (circuit breakers, trading halts)
- Elimination of a market microstructure you exploited
-
Capacity Constraint — As you scale, does the signal's profitability deteriorate?
Signal capacity: If trade size increases from 100 to 500 shares and hit rate drops 50%, you've hit capacity.
-
Opportunity Cost — Are capital and infrastructure better deployed elsewhere?
If a new signal has 2.5x Sharpe with same capital, retire the old one.
class SignalRetirementAnalyzer:
"""Analyze whether a signal should be retired."""
def __init__(self, threshold_oos_decay: float = 1.0):
"""
Initialize retirement analyzer.
Args:
threshold_oos_decay: If OOS Sharpe drops more than this
from backtest, mark for review
"""
self.threshold_oos_decay = threshold_oos_decay
def should_retire(
self,
backtest_sharpe: float,
oos_sharpe: float,
oos_months: int,
hit_rate_decline: float = 0.0,
capacity_constraint: float = 1.0
) -> Tuple[bool, str, Dict[str, float]]:
"""
Determine if signal should be retired.
Args:
backtest_sharpe: Sharpe ratio on backtest period
oos_sharpe: Sharpe ratio on out-of-sample period
oos_months: Months of out-of-sample data
hit_rate_decline: Change in hit rate (0.1 = 10% decline)
capacity_constraint: Capacity multiplier (< 1.0 means degradation)
Returns:
Tuple of (should_retire, reason, metrics)
"""
metrics = {
'sharpe_decay': backtest_sharpe - oos_sharpe,
'hit_rate_decline_pct': hit_rate_decline * 100,
'capacity_multiplier': capacity_constraint
}
reasons = []
severity = 0 # Score from 0-10
# Check 1: Fundamental decay
if oos_sharpe < (backtest_sharpe - self.threshold_oos_decay):
decay_pct = ((backtest_sharpe - oos_sharpe) / backtest_sharpe) * 100
reasons.append(
f"Fundamental decay: {decay_pct:.1f}% Sharpe degradation "
f"({backtest_sharpe:.3f} → {oos_sharpe:.3f})"
)
severity += 5 if decay_pct > 50 else 3
# Check 2: Hit rate decline
if hit_rate_decline > 0.15: # 15% decline
reasons.append(
f"Hit rate declining: {hit_rate_decline*100:.1f}% drop"
)
severity += 4
# Check 3: Capacity constraint
if capacity_constraint < 0.7: # 30% capacity degradation
reasons.append(
f"Capacity constraint: Profitability drops {(1-capacity_constraint)*100:.1f}% "
f"when scaling position size"
)
severity += 5
# Check 4: Insufficient OOS period
if oos_months < 6 and oos_sharpe < backtest_sharpe:
reasons.append(
f"Insufficient OOS period ({oos_months} months) to confirm decay"
)
severity += 1
# Decision rule
should_retire = severity >= 7
if not reasons:
reason = f"Signal healthy: {oos_months}m OOS, Sharpe {oos_sharpe:.3f}"
else:
reason = " | ".join(reasons)
return should_retire, reason, metricsModule 28.2: System Evolution
Adding New Data Sources#
Your first system uses OHLCV data from Zerodha. To compound alpha, you need to expand the information universe.
Data source hierarchy for NSE:
- Tier 1 (Essential) — OHLCV from multiple timeframes
- Tier 2 (Microstructure) — Order flow, bid-ask spread, trade size distribution
- Tier 3 (Fundamental) — Q earnings, cash flow, sector rotation
- Tier 4 (Alternative) — News sentiment, options flow, institutional activity
Framework for integrating new data sources:
from abc import ABC, abstractmethod
from typing import Dict, Any, Optional
import pandas as pd
from datetime import datetime
class DataSource(ABC):
"""Abstract base class for all data sources."""
@abstractmethod
def fetch(
self,
symbol: str,
start_date: datetime,
end_date: datetime,
**kwargs
) -> pd.DataFrame:
"""
Fetch data from source.
Args:
symbol: NSE symbol (e.g., 'INFY' or 'INFY-EQ')
start_date: Start date
end_date: End date
**kwargs: Source-specific parameters
Returns:
DataFrame with data
"""
pass
@abstractmethod
def validate(self, df: pd.DataFrame) -> Tuple[bool, str]:
"""
Validate data integrity.
Returns:
(is_valid, message)
"""
pass
class ZerodhaOHLCVSource(DataSource):
"""OHLCV data from Zerodha via kite API."""
def __init__(self, kite_instance):
"""
Initialize Zerodha source.
Args:
kite_instance: Authenticated kite.Kite instance
"""
self.kite = kite_instance
def fetch(
self,
symbol: str,
start_date: datetime,
end_date: datetime,
interval: str = "5minute"
) -> pd.DataFrame:
"""
Fetch OHLCV data from Zerodha.
Args:
symbol: NSE symbol
start_date: Start date
end_date: End date
interval: '5minute', '15minute', 'hourly', 'daily'
Returns:
DataFrame with OHLCV data
"""
try:
# Zerodha returns data in descending order, reverse it
data = self.kite.historical_data(
instrument_token=self._get_instrument_token(symbol),
from_date=start_date,
to_date=end_date,
interval=interval
)
df = pd.DataFrame(data)
df['date'] = pd.to_datetime(df['date'])
df = df.sort_values('date').reset_index(drop=True)
return df
except Exception as e:
raise DataFetchError(f"Zerodha fetch failed for {symbol}: {e}")
def validate(self, df: pd.DataFrame) -> Tuple[bool, str]:
"""Validate OHLCV data."""
required_cols = ['open', 'high', 'low', 'close', 'volume']
if not all(col in df.columns for col in required_cols):
return False, f"Missing columns. Required: {required_cols}"
if (df['high'] < df['low']).any():
return False, "Found bars with high < low"
if (df['close'].isna()).any():
return False, f"Found {df['close'].isna().sum()} NaN close prices"
if (df['volume'] <= 0).any():
return False, "Found zero or negative volumes"
return True, "Valid"
class NewsSourceAlternative(DataSource):
"""Mock implementation: sentiment data from news sources."""
def __init__(self, api_key: str = None):
"""
Initialize news sentiment source.
Args:
api_key: API key for news service
"""
self.api_key = api_key
def fetch(
self,
symbol: str,
start_date: datetime,
end_date: datetime
) -> pd.DataFrame:
"""
Fetch news sentiment for symbol.
Returns:
DataFrame with date, sentiment_score, headline_count
"""
# In production, integrate with News API, RSS feeds, etc.
# For this example, we show the integration pattern
raise NotImplementedError(
"Integrate with news provider (NewsAPI, ReutersConnect, etc)"
)
def validate(self, df: pd.DataFrame) -> Tuple[bool, str]:
"""Validate news data."""
required_cols = ['sentiment_score', 'headline_count']
if not all(col in df.columns for col in required_cols):
return False, f"Missing columns: {required_cols}"
if not (-1.0 <= df['sentiment_score']).all():
return False, "Sentiment scores must be between -1 and 1"
return True, "Valid"
class DataSourceRegistry:
"""Registry for managing multiple data sources."""
def __init__(self):
"""Initialize registry."""
self.sources: Dict[str, DataSource] = {}
def register(self, name: str, source: DataSource) -> None:
"""
Register a data source.
Args:
name: Unique name for source (e.g., 'zerodha_ohlcv')
source: DataSource instance
"""
self.sources[name] = source
def fetch_combined(
self,
symbol: str,
start_date: datetime,
end_date: datetime,
source_names: List[str] = None
) -> pd.DataFrame:
"""
Fetch and combine data from multiple sources.
Args:
symbol: Symbol to fetch
start_date: Start date
end_date: End date
source_names: Which sources to fetch (all if None)
Returns:
Combined DataFrame, aligned by date
"""
if source_names is None:
source_names = list(self.sources.keys())
dfs = {}
for source_name in source_names:
if source_name not in self.sources:
raise ValueError(f"Unknown source: {source_name}")
try:
df = self.sources[source_name].fetch(
symbol, start_date, end_date
)
is_valid, msg = self.sources[source_name].validate(df)
if not is_valid:
raise DataValidationError(f"{source_name}: {msg}")
dfs[source_name] = df
except Exception as e:
# Log error but continue with other sources
print(f"Warning: Failed to fetch from {source_name}: {e}")
# Merge all DataFrames on date
if not dfs:
raise ValueError("No data sources successfully fetched")
combined = list(dfs.values())[0]
for df in list(dfs.values())[1:]:
combined = combined.merge(
df,
on='date',
how='inner',
suffixes=('', f'_{dfs[source_name].name}')
)
return combined.sort_values('date').reset_index(drop=True)Model Retraining Strategy#
A critical question: How often should you retrain your model?
Too frequently → overfitting to recent noise, high operational overhead Too infrequently → model degrades as market structure changes
Recommended strategy for NSE intraday trading:
from sklearn.model_selection import TimeSeriesSplit
import numpy as np
from typing import Tuple
class AdaptiveRetrainingScheduler:
"""
Determine optimal retraining frequency based on performance degradation.
"""
def __init__(
self,
initial_retrain_frequency_days: int = 5,
min_trades_for_evaluation: int = 100,
performance_threshold: float = 0.15 # 15% performance drop triggers retrain
):
"""
Initialize scheduler.
Args:
initial_retrain_frequency_days: Start retraining every N days
min_trades_for_evaluation: Minimum trades before evaluating degradation
performance_threshold: Sharpe ratio drop % that triggers retraining
"""
self.initial_frequency = initial_retrain_frequency_days
self.min_trades = min_trades_for_evaluation
self.threshold = performance_threshold
self.performance_history = []
def should_retrain(
self,
days_since_last_retrain: int,
recent_sharpe: float,
baseline_sharpe: float,
trades_since_last_retrain: int
) -> Tuple[bool, str]:
"""
Determine if model should be retrained.
Args:
days_since_last_retrain: Days elapsed
recent_sharpe: Current Sharpe ratio
baseline_sharpe: Sharpe ratio at last retraining
trades_since_last_retrain: Trades executed since retrain
Returns:
(should_retrain, reason)
"""
reasons = []
should_retrain = False
# Criterion 1: Time-based
if days_since_last_retrain >= self.initial_frequency:
reasons.append(f"Time-based: {days_since_last_retrain}d since last retrain")
should_retrain = True
# Criterion 2: Performance degradation
if trades_since_last_retrain >= self.min_trades:
if baseline_sharpe > 0:
performance_drop = (baseline_sharpe - recent_sharpe) / baseline_sharpe
if performance_drop > self.threshold:
reasons.append(
f"Performance drop: {performance_drop*100:.1f}% "
f"(Sharpe {baseline_sharpe:.3f} → {recent_sharpe:.3f})"
)
should_retrain = True
if not reasons:
reason = f"No retrain needed: {days_since_last_retrain}d old, Sharpe {recent_sharpe:.3f}"
else:
reason = " | ".join(reasons)
return should_retrain, reason
class ModelRetrainingPipeline:
"""Complete retraining pipeline with validation."""
def __init__(
self,
model_class,
data_source: DataSourceRegistry,
feature_engineer,
lookback_days: int = 60
):
"""
Initialize retraining pipeline.
Args:
model_class: Scikit-learn compatible model
data_source: Registry of data sources
feature_engineer: Feature engineering pipeline
lookback_days: Days of history for retraining
"""
self.model_class = model_class
self.data_source = data_source
self.feature_engineer = feature_engineer
self.lookback_days = lookback_days
def retrain(
self,
symbol: str,
end_date: datetime = None,
save_path: str = None
) -> Tuple[Any, Dict[str, float]]:
"""
Retrain model on recent data.
Training data splits:
- Train: [end_date - lookback_days : end_date - 5 days]
- Validation: [end_date - 5 : end_date]
We hold out last 5 days to prevent lookahead bias and verify
the model works on data not used during training.
Args:
symbol: NSE symbol
end_date: End date for retraining (today if None)
save_path: Path to save retrained model
Returns:
(trained_model, performance_metrics)
"""
if end_date is None:
end_date = datetime.now()
# Fetch data
start_date = end_date - pd.Timedelta(days=self.lookback_days)
df = self.data_source.fetch_combined(
symbol, start_date, end_date
)
# Feature engineering
X, y = self.feature_engineer.transform(df)
# Split: reserve last 5 days for validation
val_idx = int(len(X) * 0.917) # ~5 trading days of 252 trading days/year
X_train, X_val = X[:val_idx], X[val_idx:]
y_train, y_val = y[:val_idx], y[val_idx:]
# Train model
model = self.model_class()
model.fit(X_train, y_train)
# Evaluate
train_score = model.score(X_train, y_train)
val_score = model.score(X_val, y_val)
# Calculate Sharpe on validation predictions
val_preds = model.predict(X_val)
val_returns = val_preds * y_val # Simulated returns
val_sharpe = self._calculate_sharpe(val_returns)
metrics = {
'train_score': train_score,
'val_score': val_score,
'val_sharpe': val_sharpe,
'overfitting_ratio': train_score / val_score if val_score > 0 else np.inf,
'training_samples': len(X_train),
'validation_samples': len(X_val)
}
# Check for overfitting
if metrics['overfitting_ratio'] > 1.5:
print(
f"Warning: Possible overfitting detected. "
f"Train/Val score ratio: {metrics['overfitting_ratio']:.2f}"
)
# Save model
if save_path:
import pickle
with open(save_path, 'wb') as f:
pickle.dump(model, f)
return model, metrics
def _calculate_sharpe(self, returns: np.ndarray) -> float:
"""Calculate Sharpe ratio."""
if len(returns) < 2:
return 0.0
return np.mean(returns) / np.std(returns) * np.sqrt(252)From Single-Strategy to Multi-Strategy#
The key to sustainable alpha is diversification across uncorrelated strategies.
Single strategy risk: If your hypothesis breaks (market regime change, competition discovers it), you're done.
Multi-strategy benefit: Even if one signal decays, others persist. Portfolio Sharpe improves through diversification.
from dataclasses import dataclass
from typing import Dict, List
import numpy as np
import pandas as pd
@dataclass
class StrategyPosition:
"""Position from a single strategy."""
symbol: str
quantity: int
target_weight: float
confidence: float # 0-1, how confident we are in this signal
class MultiStrategyPortfolio:
"""Manage multiple independent trading strategies."""
def __init__(self, total_capital: float, max_position_weight: float = 0.10):
"""
Initialize multi-strategy portfolio.
Args:
total_capital: Total capital to deploy
max_position_weight: Maximum weight per position (risk limit)
"""
self.total_capital = total_capital
self.max_position_weight = max_position_weight
self.strategies: Dict[str, Dict] = {}
self.current_portfolio = {}
def register_strategy(
self,
strategy_name: str,
target_allocation: float,
sharpe_ratio: float,
correlation_with_others: List[Tuple[str, float]] = None
) -> None:
"""
Register a new strategy.
Args:
strategy_name: Name of strategy (e.g., 'momentum', 'mean_reversion')
target_allocation: Target portfolio weight (0.0-1.0)
sharpe_ratio: Historical Sharpe ratio
correlation_with_others: List of (strategy_name, correlation) tuples
"""
self.strategies[strategy_name] = {
'allocation': target_allocation,
'sharpe': sharpe_ratio,
'correlations': correlation_with_others or []
}
def combine_signals(
self,
signals: Dict[str, List[StrategyPosition]]
) -> Dict[str, int]:
"""
Combine signals from multiple strategies into single portfolio.
Methodology:
For each symbol, aggregate signals using weighted average:
target_quantity_i = Σ_s (allocation_s * confidence_s,i * quantity_s,i)
Where:
- allocation_s: Capital allocation to strategy s
- confidence_s,i: Strategy s's confidence in symbol i
- quantity_s,i: Quantity from strategy s for symbol i
Then apply position sizing constraints.
Args:
signals: Dict of {strategy_name: [StrategyPosition]}
Returns:
Combined portfolio {symbol: quantity}
"""
# Aggregate signals by symbol
symbol_signals = {}
for strategy_name, positions in signals.items():
if strategy_name not in self.strategies:
raise ValueError(f"Unknown strategy: {strategy_name}")
strategy_allocation = self.strategies[strategy_name]['allocation']
for pos in positions:
if pos.symbol not in symbol_signals:
symbol_signals[pos.symbol] = []
# Weighted quantity = allocation * confidence * quantity
weighted_qty = (
strategy_allocation *
pos.confidence *
pos.quantity
)
symbol_signals[pos.symbol].append({
'weighted_quantity': weighted_qty,
'strategy': strategy_name,
'confidence': pos.confidence
})
# Consolidate to final portfolio
portfolio = {}
for symbol, signal_list in symbol_signals.items():
# Average across strategies
avg_quantity = np.mean([s['weighted_quantity'] for s in signal_list])
portfolio[symbol] = int(np.round(avg_quantity))
# Apply position weight constraints
portfolio = self._apply_position_constraints(portfolio)
self.current_portfolio = portfolio
return portfolio
def _apply_position_constraints(
self,
positions: Dict[str, int],
current_prices: Dict[str, float] = None
) -> Dict[str, int]:
"""
Apply position sizing constraints.
Args:
positions: Proposed positions
current_prices: Current market prices (for weight calculation)
Returns:
Constrained positions
"""
if current_prices is None:
current_prices = {s: 1.0 for s in positions.keys()}
# Calculate current position weights
total_notional = sum(
positions[s] * current_prices[s]
for s in positions.keys()
)
# Scale down if any position exceeds max weight
max_position_notional = self.total_capital * self.max_position_weight
constrained = {}
for symbol, qty in positions.items():
position_value = qty * current_prices[symbol]
if position_value > max_position_notional:
# Scale down this position
constrained_qty = int(max_position_notional / current_prices[symbol])
constrained[symbol] = constrained_qty
else:
constrained[symbol] = qty
return constrained
def calculate_portfolio_metrics(
self,
returns_by_strategy: Dict[str, pd.Series]
) -> Dict[str, float]:
"""
Calculate portfolio-level metrics.
Args:
returns_by_strategy: Dict of {strategy: Series of returns}
Returns:
Portfolio metrics including Sharpe ratio, diversification benefit
"""
# Combine returns using allocations
portfolio_returns = pd.Series(0.0, index=returns_by_strategy[
list(returns_by_strategy.keys())[0]].index)
for strategy_name, returns in returns_by_strategy.items():
allocation = self.strategies[strategy_name]['allocation']
portfolio_returns += allocation * returns
# Calculate metrics
annual_return = portfolio_returns.mean() * 252
annual_vol = portfolio_returns.std() * np.sqrt(252)
sharpe = annual_return / annual_vol if annual_vol > 0 else 0
# Calculate diversification benefit
# Sum of individual Sharpes weighted by allocation
weighted_sharpe_sum = sum(
self.strategies[s]['sharpe'] * self.strategies[s]['allocation']
for s in self.strategies.keys()
)
diversification_benefit = sharpe / weighted_sharpe_sum if weighted_sharpe_sum > 0 else 1.0
return {
'portfolio_sharpe': sharpe,
'annual_return': annual_return,
'annual_volatility': annual_vol,
'diversification_benefit': diversification_benefit,
'num_strategies': len(self.strategies),
'avg_strategy_sharpe': weighted_sharpe_sum
}Module 28.3: The Long Game
Building an Edge That Lasts#
There's a critical distinction between outcome and process.
Outcome (what happened):
- Made 23% this month
- Won 60% of trades
- Beat benchmark by 200 bps
Process (why it happened):
- Systematically tested 47 hypotheses using rigorous methodology
- Maintained consistent position sizing discipline
- Adapted signal parameters when OOS performance degraded
- Diversified across 3 uncorrelated alpha sources
The outcome-oriented trader celebrates when lucky and despairs when unlucky. The process-oriented trader focuses on: Did we execute our system with discipline?
The Mathematics of Process Excellence#
Consider a trader with:
- Win rate: 52% (1% better than 50-50 coin flip)
- Average win: $1000
- Average loss: $980
- Expected value per trade: (0.52 × 1000) + (0.48 × -980) = $49.60
Over N trades, expected total profit = $49.60 × N
With a process that ensures consistent execution:
- Trade 5 times/day × 250 trading days/year = 1,250 trades/year
- Annual profit = 62,000
This seems small. But consider the compounding:
If you maintain this process over 10 years:
- Year 1: 62K + initial stake)
- Year 2: $62K profit on larger base
- By Year 10: With reinvestment at 3:1 leverage, account compounds to $500K+
Key insight: Small, consistent edges compound into massive returns over time.
The trader who obsesses over monthly returns will:
- Abandon profitable strategies after unlucky drawdowns
- Chase recent winners (overfit to recent data)
- Over-trade, increasing transaction costs
- Burn out mentally, making emotional decisions
The trader who obsesses over process will:
- Accept short-term variance while trusting positive expectancy
- Stick with systematic approach through drawdowns
- Gradually compound alpha sources
- Build toward institutional-scale operation
The Research Pipeline Compounding Effect#
Let's model how a disciplined research pipeline creates sustainable advantage:
class ResearchCompoundingModel:
"""
Model the compounding effect of research pipeline on alpha.
Mathematical model:
If you generate H hypotheses per month, with:
- p: probability each hypothesis produces positive alpha
- E[α]: expected alpha from successful hypothesis
- decay_rate: annual alpha decay rate for existing signals
Then total portfolio alpha:
α(t) = Σ_i α_i(0) * (1 - decay_rate)^t + Σ_new p * E[α]
Where first term is existing signals decaying,
second term is new successful signals being added.
With continuous hypothesis generation, portfolio alpha
stabilizes at:
α_steady_state = (H * p * E[α]) / decay_rate
This shows why companies with strong research pipelines
maintain edges indefinitely.
"""
def __init__(
self,
hypotheses_per_month: int = 2,
success_rate: float = 0.20, # 20% of hypotheses succeed
expected_alpha_sharpe: float = 0.50, # Excess Sharpe from good signal
annual_decay_rate: float = 0.15 # 15% annual decay
):
"""
Initialize compounding model.
Args:
hypotheses_per_month: Number of ideas tested monthly
success_rate: Fraction that become profitable signals
expected_alpha_sharpe: Expected alpha contribution
annual_decay_rate: Rate at which signals decay
"""
self.hypotheses_per_month = hypotheses_per_month
self.success_rate = success_rate
self.expected_alpha = expected_alpha_sharpe
self.decay_rate = annual_decay_rate
def project_alpha(self, years: int) -> pd.DataFrame:
"""
Project portfolio alpha over time.
Args:
years: Number of years to project
Returns:
DataFrame with projections
"""
results = []
existing_signals = [] # Track each signal's alpha
total_alpha = 0.0
for month in range(years * 12):
# Add new signals
num_new_signals = int(
self.hypotheses_per_month * self.success_rate
)
for _ in range(num_new_signals):
existing_signals.append(self.expected_alpha)
# Decay existing signals
for i in range(len(existing_signals)):
existing_signals[i] *= (1 - self.decay_rate / 12)
# Total alpha is sum of all signals
total_alpha = sum(existing_signals)
results.append({
'month': month,
'year': month / 12,
'total_alpha_sharpe': total_alpha,
'num_active_signals': len([s for s in existing_signals if s > 0.01]),
'newest_signal_alpha': existing_signals[-1] if existing_signals else 0
})
return pd.DataFrame(results)
def calculate_steady_state(self) -> float:
"""
Calculate steady-state portfolio alpha.
At equilibrium, new alpha added equals alpha lost to decay:
New alpha per month = H/12 * p * E[α]
(where H is annual hypotheses = monthly * 12)
Equilibrium when:
New = Lost
H/12 * p * E[α] = decay_rate * α_ss
α_ss = (H * p * E[α]) / (12 * decay_rate)
Returns:
Steady-state portfolio alpha (Sharpe)
"""
annual_hypotheses = self.hypotheses_per_month * 12
new_alpha_per_year = annual_hypotheses * self.success_rate * self.expected_alpha
steady_state_alpha = new_alpha_per_year / self.decay_rate
return steady_state_alphaExample projections:
- Research rate: 2 hypotheses/month, 20% success rate (0.4 new signals/month)
- Alpha per signal: 0.50 Sharpe
- Decay rate: 15% annually
- Steady-state alpha: (2 × 12 × 0.20 × 0.50) / 0.15 = 5.33 Sharpe
This is extraordinary. A portfolio with 5.33 Sharpe ratio (with moderate leverage) compounds capital at 50%+ annually.
The point: A systematic research pipeline is more valuable than any single signal.
From Personal Project to Institutional Operation#
At what scale does your personal trading system become an actual business?
Milestones:
-
Proof of concept (100K capital)
- Single strategy working consistently
- 6+ months live trading data
- Sharpe ratio > 1.0
-
Product development (1M)
- 2-3 uncorrelated strategies
- Full research pipeline operational
- Documented processes and procedures
- Ready to hand off trading to someone else
-
Institutional scale ($1M+)
- Multi-strategy system running autonomously
- Demonstrated ability to scale without losing edge
- Professional infrastructure (risk systems, compliance, operations)
- Capable of managing external capital
Transition checklist from project to business:
class TransitionChecklist:
"""Evaluate readiness to move from personal trading to institutional."""
def __init__(self):
self.criteria = {
# Profitability
'sharpe_ratio_exceeds_1_5': False,
'live_trading_track_record_months': 0,
'monthly_return_consistent': False,
'max_drawdown_below_15_pct': False,
# Scalability
'can_scale_2x_without_slippage': False,
'can_scale_10x_with_infrastructure': False,
'capital_dependent_on_leverage': True, # Bad if true
# Process maturity
'research_pipeline_documented': False,
'hypothesis_log_maintained': False,
'retraining_automated': False,
'risk_limits_automated': False,
# Operational
'trading_executable_by_other_person': False,
'system_runs_without_manual_intervention': False,
'monitoring_and_alerts_automated': False,
'incident_procedures_documented': False,
# Infrastructure
'redundant_data_feeds': False,
'backup_execution_venue': False,
'pos_sizing_respects_vol_targets': False,
'transaction_costs_modeled': False
}
def readiness_score(self) -> float:
"""Calculate 0-100 readiness score."""
weight_groups = {
'profitability': [
'sharpe_ratio_exceeds_1_5',
'live_trading_track_record_months',
'monthly_return_consistent'
],
'scalability': [
'can_scale_2x_without_slippage',
'can_scale_10x_with_infrastructure'
],
'process': [
'research_pipeline_documented',
'hypothesis_log_maintained',
'retraining_automated'
],
'operations': [
'trading_executable_by_other_person',
'system_runs_without_manual_intervention',
'monitoring_and_alerts_automated'
]
}
scores = {}
for group, criteria in weight_groups.items():
group_score = sum(
self.criteria.get(c, False) for c in criteria
) / len(criteria)
scores[group] = group_score
# Profitability must be demonstrated
if scores['profitability'] < 0.5:
return 0.0
# Average other dimensions
overall = (
scores['profitability'] * 0.4 +
scores['scalability'] * 0.2 +
scores['process'] * 0.2 +
scores['operations'] * 0.2
) * 100
return overallWhat Quant Firms Look For#
You've built your system. Now you're interviewing at a quant fund. What are they actually evaluating?
They don't care about your monthly returns. They care about:
1. Your Thinking Process#
They'll ask:
- "Walk me through your most profitable idea and why you thought it would work"
- "Tell me about an idea that failed completely"
- "How did you validate this hypothesis?"
What they're listening for:
- Do you think in terms of economic rationale or pattern recognition?
- Can you articulate why an edge should exist?
- Do you acknowledge limitations of your work?
Answer structure: Hypothesis → Reasoning → Testing → Validation → Degradation
2. Your Research Discipline#
Questions:
- "How many hypotheses have you tested?"
- "What's your out-of-sample Sharpe?"
- "How do you avoid data snooping?"
What they evaluate:
- Research log and hypothesis tracking
- Backtest methodology (walk-forward validation, proper data splits)
- Out-of-sample verification
- Evidence of not overfitting
This is why maintaining a research log is critical. You can show: "I've tested 47 ideas. 9 made it to production. 3 are currently live. Here's my detailed log."
3. Your Process vs. Outcomes#
They'll dig into:
- "Your Sharpe was 2.5 last year but 1.2 this year. What happened?"
Bad answer: "Bad luck, market conditions changed, the strategy deteriorated"
Good answer: "I tracked this via my research log. OOS Sharpe degraded from 1.8 to 0.9. This was expected based on [economic reason]. I retired it in March and deployed replacement signal with 1.5 Sharpe. Overall portfolio Sharpe maintained at 1.2."
They want evidence that you:
- Monitor signal health continuously
- Have systematic retirement criteria
- Maintain alpha through portfolio evolution
4. Your Scalability Story#
They ask: "Could this manage $1B?"
Red flags:
- "My signal only works with $1-10M capital"
- "I haven't thought about how to scale"
- "I use a small set of micro-cap stocks with low liquidity"
Green flags:
- "My signal works on NSE's top 100 liquid stocks. At $1B, I'd be 0.1-0.5% of daily volume"
- "I've modeled capacity constraints and shown OOS performance with scaled position sizes"
- "I'm diversified across multiple uncorrelated strategies"
5. Your Risk Management#
Questions:
- "What's your maximum daily loss?"
- "How do you handle drawdowns?"
- "What system failures concern you most?"
Good answers show:
- Position sizing based on vol targeting
- Portfolio-level leverage limits
- Monitoring for regime changes
- Graceful degradation (can run at reduced capacity)
Interview Presentation Framework#
class InterviewPresentation:
"""
Structure for presenting your trading research to funds.
"""
# Section 1: Process Overview (5 minutes)
# Your research methodology and discipline
# Show: research log structure, hypothesis format, testing protocol
# Section 2: Case Study: One Successful Signal (10 minutes)
# Hypothesis → Economic Rationale → Testing → Results → Production
# Outline:
# - What was your hypothesis?
# - Why would this work? (microstructure / behavioral / fundamental)
# - How did you test it? (period, data, methodology)
# - What were results? (Sharpe, hit rate, drawdown)
# - How did OOS performance compare?
# - What does signal look like in production?
# - When/why did you retire it? (or current status)
# Section 3: Portfolio Evolution (5 minutes)
# Show how you build multi-strategy portfolio
# Display: correlation matrix of signals, cumulative Sharpe
# Section 4: Infrastructure & Scalability (3 minutes)
# How does your system handle growth?
# Latency, capacity, diversification
# Section 5: What You're Looking For (2 minutes)
# How would capital change your research priorities?
passThe Compounding of Mastery#
Here's the deepest insight about the long game:
Trading skill compounds faster than financial returns.
Year 1: You learn how to backtest properly, avoid overfitting, maintain a research log.
Year 2: You understand your own biases, develop pattern recognition for signal decay, can diagnose model failures.
Year 3: You've tested 100+ hypotheses, understand NSE market structure, can predict which ideas will work.
Year 4: You think in terms of portfolio construction, regime detection, systematic signal replacement.
Year 5: You can train someone else to execute your system. You've built something institutional.
This skill development is non-linearly valuable. A trader with 5 years of disciplined research experience is exponentially more valuable than a trader with 1 year.
This is why quant funds invest so heavily in hiring experienced researchers and training pipelines. Experience and process, compounded, create defensible edges.
Chapter Summary#
Module 28.1 — Research Pipeline
- Alpha research must start with economically motivated hypotheses
- Maintain detailed research logs to track all tested ideas
- A/B test new signals against production signals using rigorous statistics
- Retire signals when OOS performance degrades, market structure changes, or capacity constrains
- The goal: systematic discovery of repeatable trading edges
Module 28.2 — System Evolution
- Integrate new data sources systematically (order flow, fundamental, sentiment)
- Retrain models adaptively based on performance degradation, not calendar
- Scale from single strategy to multi-strategy portfolio for resilience
- Optimize for correlation and capacity constraints
- Build infrastructure that allows autonomous operation
Module 28.3 — The Long Game
- Focus on process, not outcomes. Small consistent edges compound massively.
- A research pipeline with steady hypothesis generation produces steadily compounding alpha
- Transition from personal project to institutional operation through milestones
- Quant firms evaluate your thinking, discipline, and scalability—not just returns
- Skill in quantitative trading compounds faster than financial returns
Interview Questions for Self-Assessment#
- What's the biggest hypothesis you've abandoned, and why?
- If your top signal lost 50% of its Sharpe tomorrow, how would you diagnose why?
- How many out-of-sample periods have you tested? (Good answer: 3+ independent periods)
- What percentage of your capital is deployed in signals you tested <2 months ago?
- Could you hand your system to someone else and have it execute without your involvement?
- What's your correlation matrix between signals? (Should show low/negative correlations)
- If you had $10M to deploy, what would change about your system?
- Can you articulate the economic rationale for each signal in your portfolio?
Further Reading & Research#
- De Prado, M. L. Advances in Financial Machine Learning (2018) — Advanced backtesting methodology
- López de Prado, M. Machine Learning for Asset Managers (2020) — ML in quant finance
- Pardo, R. The Evaluation and Optimization of Trading Strategies (2008) — Walk-forward analysis
- Aronson, D. Evidence-Based Technical Analysis (2007) — Avoiding overfitting
- Taleb, N. Fooled by Randomness (2001) — Understanding variance vs. alpha
End of Chapter 28