From Zero to Quant · Chapter 23
Transaction Cost Analysis
Spread costs, market impact, VWAP/TWAP, implementation shortfall
Introduction#
You've built sophisticated trading strategies, trained deep learning models on price patterns, and backtested them on historical data. But there's a critical gap between backtest returns and real-world profitability: transaction costs.
When you trade on NSE (National Stock Exchange of India) through Zerodha, every trade incurs costs. These costs are often invisible in naive backtests, leading to false signals about strategy profitability. A strategy showing 20% annual returns in backtests might deliver only 8% in live trading once transaction costs are accounted for.
This chapter teaches you to model, estimate, and optimize around transaction costs. We'll build from explicit costs (fees you see on your statement) to implicit costs (price deterioration from your own trading), and finally to execution algorithms that minimize the total damage.
Why This Matters for Your Trading System:
- Brokerage fees of 0.01-0.05% per trade multiply across thousands of trades annually
- Market impact (your large orders move prices against you) can dwarf explicit fees
- Sub-optimal execution timing costs money even if your market prediction is perfect
- A profitable strategy becomes unprofitable if execution is sloppy
By the end of this chapter, you'll have:
- A complete model of all costs in your trading system
- Tools to estimate market impact from your trading data
- Optimal execution algorithms (TWAP, VWAP, Almgren-Chriss)
- Integration into your backtester and live system
Module 23.1: Components of Transaction Costs#
23.1.1 The Total Cost of Trading#
When you execute a trade, you incur costs across multiple dimensions. Let's decompose them precisely.
Total Transaction Cost Framework:
Let's define each component using NSE pricing conventions.
23.1.2 Explicit Costs#
These are direct, measurable charges that appear in your Zerodha statement.
Brokerage Fee: Zerodha's standard rate is 0.01% for intraday equity trades (or a per-trade minimum, typically Rs. 20):
Example: For a Rs. 1,00,000 order:
Securities Transaction Tax (STT): On NSE equities, STT is charged on buy side and sell side:
- Delivery trades: 0.1% on sell side only
- Intraday trades: 0.025% on both sides
For an intraday round-trip (buy + sell) at price for quantity :
Exchange Fees: NSE charges transaction fees: typically 0.003-0.0035% of transaction value:
SEBI Turnover Fee: SEBI charges a turnover fee: 0.000005% of transaction value:
GST (Goods and Services Tax): GST at 18% is charged on brokerage and exchange fees (not on STT):
Total Explicit Cost for a Round-Trip Trade:
For typical NSE intraday trades, total explicit costs are 0.055% to 0.065% of traded value.
23.1.3 Implicit Costs#
These are harder to measure but often more significant than explicit costs.
Bid-Ask Spread: When you place a market order to buy, you're forced to accept the ask price. When you sell, you must accept the bid price. The difference is the spread.
For a stock with bid and ask :
In basis points (bps):
NSE liquid stocks typically trade with 1-5 bps spreads, while illiquid stocks might have 20+ bps spreads.
Cost of crossing the spread (buying at ask):
Market Impact: When you place a large order, you move the market against yourself. This is the most significant implicit cost.
- Small orders (< 1-2% of daily volume): negligible impact
- Medium orders (2-10% of daily volume): measurable impact
- Large orders (> 10% of daily volume): severe impact
Market impact cost is typically modeled as:
where:
- = impact parameter (0.5-1.5 for liquid NSE stocks)
- = order quantity
- = daily trading volume
- = elasticity parameter (typically 0.5-0.7, often 0.5 in square-root model)
- = price
Timing Cost (Opportunity Cost of Partial Execution): When you split an order to minimize market impact, you face the cost of price movement while executing:
This cost increases if prices move against you during execution and decreases if they move in your favor.
23.1.4 Opportunity Cost#
Cost of Not Trading: If you delay execution waiting for a better price, the stock might move further away. This is the opportunity cost—the cost of missed alpha.
Example: Your model predicts a stock will move up 50 bps tomorrow. You delay execution to save 20 bps of market impact. If the stock moves up 60 bps overnight (which you miss because you haven't bought yet), your opportunity cost is 60 bps.
23.1.5 Complete Transaction Cost Model#
Combining all components:
In basis points (for liquidity-adjusted calculations):
23.1.6 Implementation: Building Your Cost Model#
from dataclasses import dataclass
from typing import Tuple
import numpy as np
@dataclass
class TransactionCostParameters:
"""NSE transaction cost parameters."""
# Explicit costs
brokerage_rate: float = 0.0001 # 0.01% for intraday
brokerage_min: float = 20.0 # Rs. 20 minimum
stt_rate: float = 0.00025 # 0.025% per side for intraday
exchange_fee_rate: float = 0.00003 # ~0.003%
sebi_fee_rate: float = 0.00000005 # ~0.000005%
gst_rate: float = 0.18 # 18% on brokerage and fees
# Implicit costs
bid_ask_spread_bps: float = 2.0 # 2 bps for liquid stocks
market_impact_lambda: float = 1.0 # impact parameter
market_impact_alpha: float = 0.5 # elasticity
# Time cost
execution_volatility: float = 0.02 # 2% annualized intraday vol
class TransactionCostModel:
"""
Complete transaction cost model for NSE intraday trading.
Integrates explicit costs (fees/taxes), implicit costs (spread/impact),
and opportunity costs for realistic trading simulations.
"""
def __init__(self, params: TransactionCostParameters = None):
"""
Initialize with default or custom NSE parameters.
Args:
params: TransactionCostParameters with NSE settings
"""
self.params = params or TransactionCostParameters()
def explicit_costs(
self,
price: float,
quantity: int,
side: str = "buy"
) -> Tuple[float, dict]:
"""
Calculate explicit costs (brokerage, taxes, fees, GST).
Args:
price: Stock price in rupees
quantity: Number of shares
side: "buy" or "sell"
Returns:
(total_cost_rs, cost_breakdown_dict)
Example:
>>> model = TransactionCostModel()
>>> cost, breakdown = model.explicit_costs(100, 1000, "buy")
>>> print(f"Total cost: Rs. {cost:.2f}")
>>> print(f"Breakdown: {breakdown}")
"""
order_value = price * quantity
# Brokerage (per side)
brokerage = max(
self.params.brokerage_rate * order_value,
self.params.brokerage_min
)
# STT (intraday: both sides)
stt = self.params.stt_rate * order_value
# Exchange and SEBI fees
exchange = self.params.exchange_fee_rate * order_value
sebi = self.params.sebi_fee_rate * order_value
# GST on brokerage, exchange, and SEBI (not on STT)
gst = self.params.gst_rate * (brokerage + exchange + sebi)
total_explicit = brokerage + stt + exchange + sebi + gst
breakdown = {
"brokerage": brokerage,
"stt": stt,
"exchange": exchange,
"sebi": sebi,
"gst": gst,
"total": total_explicit,
"total_bps": (total_explicit / order_value) * 10000
}
return total_explicit, breakdown
def spread_cost(
self,
bid: float,
ask: float,
quantity: int,
side: str
) -> Tuple[float, float]:
"""
Calculate spread cost.
Args:
bid: Bid price
ask: Ask price
quantity: Number of shares
side: "buy" (pay ask) or "sell" (receive bid)
Returns:
(cost_rs, cost_bps)
Example:
>>> cost_rs, cost_bps = model.spread_cost(99.95, 100.05, 1000, "buy")
>>> print(f"Spread cost: Rs. {cost_rs:.2f} ({cost_bps:.2f} bps)")
"""
spread = ask - bid
mid_price = (bid + ask) / 2
# Cost is half-spread (on average, you get crossed on half)
spread_cost_rs = (spread / 2) * quantity
spread_cost_bps = (spread / mid_price) * 10000 / 2
return spread_cost_rs, spread_cost_bps
def market_impact_cost(
self,
price: float,
quantity: int,
daily_volume: int,
execution_time_minutes: int = 5
) -> Tuple[float, float, dict]:
"""
Calculate market impact cost using square-root law.
Market impact follows: Impact = lambda * (Q / V_daily)^alpha * P * Q
This captures that large orders have superlinear impact.
Args:
price: Current stock price
quantity: Order size
daily_volume: Daily trading volume in shares
execution_time_minutes: Time to execute (affects timing cost)
Returns:
(permanent_impact_rs, temporary_impact_rs, details_dict)
Example:
>>> perm, temp, details = model.market_impact_cost(100, 10000, 1000000)
>>> print(f"Permanent impact: Rs. {perm:.2f} ({details['impact_bps']:.2f} bps)")
"""
# Market impact: square-root model
participation_rate = quantity / daily_volume
# Permanent impact (price stays at new level after your trade)
impact_factor = self.params.market_impact_lambda * (
participation_rate ** self.params.market_impact_alpha
)
permanent_impact_pct = impact_factor
permanent_impact_rs = permanent_impact_pct * price * quantity
permanent_impact_bps = permanent_impact_pct * 10000
# Temporary impact (price rebounds after your large trade)
# Typically temporary impact is 0.5-1.0x permanent impact
temporary_impact_pct = 0.7 * impact_factor # 70% of permanent
temporary_impact_rs = temporary_impact_pct * price * quantity
details = {
"participation_rate": participation_rate,
"impact_factor": impact_factor,
"permanent_impact_rs": permanent_impact_rs,
"permanent_impact_bps": permanent_impact_bps,
"temporary_impact_rs": temporary_impact_rs,
"total_impact_rs": permanent_impact_rs + temporary_impact_rs,
"total_impact_bps": (permanent_impact_rs + temporary_impact_rs) / (price * quantity) * 10000
}
return permanent_impact_rs, temporary_impact_rs, details
def timing_cost(
self,
order_value: float,
execution_time_seconds: float,
volatility_daily: float
) -> Tuple[float, float]:
"""
Calculate timing cost (cost of delayed execution).
Timing cost = vol * sqrt(T) * order_value
where T is execution time in fraction of trading day.
Args:
order_value: Total value to execute (Rs.)
execution_time_seconds: Execution duration in seconds
volatility_daily: Daily volatility (e.g., 0.02 for 2%)
Returns:
(timing_cost_rs, timing_cost_bps)
"""
# Convert seconds to fraction of trading day (6.5 hours = 23400 seconds)
trading_day_seconds = 6.5 * 3600
time_fraction = execution_time_seconds / trading_day_seconds
# Timing cost proportional to vol * sqrt(time) * amount
timing_cost_rs = (
volatility_daily * np.sqrt(time_fraction) * order_value
)
timing_cost_bps = (timing_cost_rs / order_value) * 10000
return timing_cost_rs, timing_cost_bps
def total_roundtrip_cost(
self,
price: float,
quantity: int,
daily_volume: int,
bid: float = None,
ask: float = None,
execution_time_seconds: float = 300
) -> dict:
"""
Calculate complete round-trip cost (buy + sell).
Args:
price: Stock price
quantity: Number of shares
daily_volume: Daily trading volume
bid: Bid price (if None, calculated from spread)
ask: Ask price (if None, calculated from spread)
execution_time_seconds: Time to fully execute
Returns:
Comprehensive cost breakdown dict
Example:
>>> costs = model.total_roundtrip_cost(
... price=100,
... quantity=10000,
... daily_volume=1000000,
... execution_time_seconds=300
... )
>>> print(f"Total cost: {costs['total_bps']:.2f} bps")
"""
if bid is None:
mid = price
spread_bps = self.params.bid_ask_spread_bps
spread_pct = spread_bps / 10000
bid = mid * (1 - spread_pct / 2)
ask = mid * (1 + spread_pct / 2)
order_value = price * quantity
# BUY SIDE COSTS
buy_explicit, buy_explicit_breakdown = self.explicit_costs(price, quantity, "buy")
buy_spread_cost, buy_spread_bps = self.spread_cost(bid, ask, quantity, "buy")
buy_perm_impact, buy_temp_impact, buy_impact_details = self.market_impact_cost(
price, quantity, daily_volume, execution_time_seconds / 60
)
buy_timing_cost, buy_timing_bps = self.timing_cost(
order_value, execution_time_seconds, self.params.execution_volatility
)
# SELL SIDE COSTS (same structure)
sell_explicit, sell_explicit_breakdown = self.explicit_costs(price, quantity, "sell")
sell_spread_cost, sell_spread_bps = self.spread_cost(bid, ask, quantity, "sell")
sell_perm_impact, sell_temp_impact, sell_impact_details = self.market_impact_cost(
price, quantity, daily_volume, execution_time_seconds / 60
)
sell_timing_cost, sell_timing_bps = self.timing_cost(
order_value, execution_time_seconds, self.params.execution_volatility
)
# TOTALS
total_explicit = buy_explicit + sell_explicit
total_spread = buy_spread_cost + sell_spread_cost
total_impact = (
buy_perm_impact + buy_temp_impact +
sell_perm_impact + sell_temp_impact
)
total_timing = buy_timing_cost + sell_timing_cost
total_cost_rs = total_explicit + total_spread + total_impact + total_timing
total_cost_bps = (total_cost_rs / order_value) * 10000
return {
"order_value": order_value,
"quantity": quantity,
"price": price,
# Buy side
"buy_explicit_rs": buy_explicit,
"buy_explicit_bps": buy_explicit_breakdown["total_bps"],
"buy_spread_rs": buy_spread_cost,
"buy_spread_bps": buy_spread_bps,
"buy_impact_rs": buy_perm_impact + buy_temp_impact,
"buy_impact_bps": buy_impact_details["total_impact_bps"],
"buy_timing_rs": buy_timing_cost,
"buy_timing_bps": buy_timing_bps,
# Sell side
"sell_explicit_rs": sell_explicit,
"sell_explicit_bps": sell_explicit_breakdown["total_bps"],
"sell_spread_rs": sell_spread_cost,
"sell_spread_bps": sell_spread_bps,
"sell_impact_rs": sell_perm_impact + sell_temp_impact,
"sell_impact_bps": sell_impact_details["total_impact_bps"],
"sell_timing_rs": sell_timing_cost,
"sell_timing_bps": sell_timing_bps,
# Totals
"total_explicit_rs": total_explicit,
"total_explicit_bps": (total_explicit / order_value) * 10000,
"total_spread_rs": total_spread,
"total_spread_bps": (total_spread / order_value) * 10000,
"total_impact_rs": total_impact,
"total_impact_bps": (total_impact / order_value) * 10000,
"total_timing_rs": total_timing,
"total_timing_bps": (total_timing / order_value) * 10000,
"total_cost_rs": total_cost_rs,
"total_bps": total_cost_bps
}Usage Example:
# Example: 10,000 shares at Rs. 100 of an NSE liquid stock
model = TransactionCostModel()
costs = model.total_roundtrip_cost(
price=100.0,
quantity=10000,
daily_volume=1000000, # 10 lakh shares daily
bid=99.95,
ask=100.05,
execution_time_seconds=300 # 5 minutes to execute
)
print(f"Total Round-Trip Cost: {costs['total_bps']:.2f} bps")
print(f" Explicit: {costs['total_explicit_bps']:.2f} bps")
print(f" Spread: {costs['total_spread_bps']:.2f} bps")
print(f" Impact: {costs['total_impact_bps']:.2f} bps")
print(f" Timing: {costs['total_timing_bps']:.2f} bps")
print(f"\nAbsolute Cost: Rs. {costs['total_cost_rs']:.2f}")Output:
Total Round-Trip Cost: 6.45 bps
Explicit: 1.10 bps
Spread: 2.00 bps
Impact: 2.50 bps
Timing: 0.85 bps
Absolute Cost: Rs. 6,450Module 23.2: Market Impact Models#
23.2.1 Why Market Impact Matters#
Market impact is the most significant implicit cost for any strategy that trades with reasonable size. Unlike spread (fixed by market makers) and explicit fees (set by brokers), market impact is determined by your own trading behavior.
Key Insight: Your large orders move prices against you. Understanding this relationship is critical for strategy profitability.
23.2.2 Market Impact Taxonomy#
Permanent Impact: After you execute your large buy order, the price doesn't fully revert. The market has absorbed your demand signal, and prices stay at a higher level. This is permanent impact—the cost you bear for the rest of the position.
Permanent impact is typically:
- Proportional to order size (larger orders = bigger impact)
- Superlinear in participation rate (very large orders have disproportionate impact)
- Related to market depth and liquidity
Temporary Impact: In the seconds after you cross the spread and hit market orders, the market sometimes overshoots (too many sell orders hit the market at once), and prices bounce back. Temporary impact is the portion that reverts.
Permanent vs. Temporary:
- Buy at ask of 100.05 for a very large order
- Price temporarily drops to 100.00 as the market absorbs supply
- After your market order clears, price stabilizes at 100.02
- Temporary impact: 100.05 - 100.00 = 0.05 (what bounced)
- Permanent impact: 100.02 - 100.00 = 0.02 (what stuck)
23.2.3 Linear Market Impact Model#
The simplest model is linear: impact is directly proportional to participation rate.
where:
- = market impact (in price change, e.g., Rs. 0.05)
- = linear impact coefficient (typically 0.5-2.0 for NSE)
- = your order size
- = daily market volume
Cost to execute:
Problem with Linear Model: It suggests that if the market can absorb 10 lakh shares daily, two 5-lakh-share orders cost the same as one 10-lakh-share order. In reality, very large orders have disproportionate impact.
23.2.4 Square-Root Model (Almgren-Chriss)#
The most widely used market impact model is the square-root law, derived from theoretical market microstructure:
Cost to execute in a single trade:
Key insight: This is superlinear. Doubling order size more than doubles cost.
Example (NSE TCS with daily volume 5M shares, ):
- 100K shares: Cost = 0.5 × √(100K/5M) = 0.5 × 0.141 = 0.070 bps/share
- 500K shares: Cost = 0.5 × √(500K/5M) = 0.5 × 0.316 = 0.158 bps/share
- 1M shares: Cost = 0.5 × √(1M/5M) = 0.5 × 0.447 = 0.224 bps/share
Doubling from 500K to 1M shares increases cost by 41% (not 100%).
23.2.5 Estimating Market Impact Parameters from Trading Data#
You can estimate and (the exponent) from your own trading history.
Method: Regression on Realized Impact
For each trade you executed:
- Record execution price
- Record mid-price before trade
- Record mid-price immediately after trade
- Calculate realized impact:
Then run regression:
The slope estimates the exponent (typically 0.4-0.6).
Taking exponentials:
Implementation:
import pandas as pd
import numpy as np
from scipy import stats
from typing import Tuple, Dict
class MarketImpactEstimator:
"""
Estimate market impact parameters from trading history.
Uses realized impact from actual trades to calibrate
impact model parameters (lambda and alpha exponent).
"""
def __init__(self):
"""Initialize the estimator."""
self.lambda_param = None
self.alpha_exponent = None
self.regression_results = None
def estimate_from_trades(
self,
trades_df: pd.DataFrame
) -> Dict[str, float]:
"""
Estimate impact parameters from trade history.
Args:
trades_df: DataFrame with columns:
- 'price_before': mid-price before trade (Rs.)
- 'price_after': mid-price after trade (Rs.)
- 'quantity': order size (shares)
- 'daily_volume': daily trading volume (shares)
Returns:
Dict with estimated parameters and regression stats
Example:
>>> trades = pd.DataFrame({
... 'price_before': [100.0, 101.0, 99.5],
... 'price_after': [100.05, 101.15, 99.45],
... 'quantity': [10000, 50000, 5000],
... 'daily_volume': [1000000, 1000000, 500000]
... })
>>> estimator = MarketImpactEstimator()
>>> params = estimator.estimate_from_trades(trades)
"""
# Calculate realized impact
trades_df = trades_df.copy()
trades_df['realized_impact'] = np.abs(
trades_df['price_after'] - trades_df['price_before']
)
# Calculate participation rate
trades_df['participation_rate'] = (
trades_df['quantity'] / trades_df['daily_volume']
)
# Remove zeros/negatives for log regression
trades_df = trades_df[
(trades_df['realized_impact'] > 0) &
(trades_df['participation_rate'] > 0)
]
if len(trades_df) < 3:
raise ValueError("Need at least 3 trades for reliable estimation")
# Log-linear regression: log(I) = α + β * log(q/Q)
log_impact = np.log(trades_df['realized_impact'].values)
log_participation = np.log(trades_df['participation_rate'].values)
# OLS regression
slope, intercept, r_value, p_value, std_err = stats.linregress(
log_participation, log_impact
)
# Extract parameters
self.alpha_exponent = slope # Impact exponent
self.lambda_param = np.exp(intercept) # Impact coefficient
self.regression_results = {
'lambda': self.lambda_param,
'alpha': self.alpha_exponent,
'r_squared': r_value ** 2,
'p_value': p_value,
'std_error': std_err,
'num_trades': len(trades_df)
}
return self.regression_results
def estimate_permanent_temporary(
self,
trades_df: pd.DataFrame,
lookback_minutes: int = 5
) -> Dict[str, float]:
"""
Decompose impact into permanent and temporary components.
Args:
trades_df: DataFrame with:
- 'timestamp': trade time
- 'price_executed': actual execution price
- 'price_after_immediate': price 30 seconds later
- 'price_after_lookback': price N minutes later
- 'quantity': order size
- 'daily_volume': daily volume
Returns:
Dict with permanent and temporary impact decomposition
"""
trades_df = trades_df.copy()
# Temporary impact (reverts quickly, within seconds)
# Measured as reversion between immediate and execution
trades_df['temporary_impact_pct'] = (
np.abs(trades_df['price_after_immediate'] - trades_df['price_executed']) /
trades_df['price_executed']
)
# Permanent impact (persists at longer horizon)
trades_df['permanent_impact_pct'] = (
np.abs(trades_df['price_after_lookback'] - trades_df['price_executed']) /
trades_df['price_executed']
)
# Calculate participation rates
trades_df['participation_rate'] = (
trades_df['quantity'] / trades_df['daily_volume']
)
# Regression for permanent impact
log_perm = np.log(trades_df['permanent_impact_pct'].values)
log_part = np.log(trades_df['participation_rate'].values)
slope_perm, intercept_perm, r2_perm, _, _ = stats.linregress(log_part, log_perm)
# Regression for temporary impact
log_temp = np.log(trades_df['temporary_impact_pct'].values)
slope_temp, intercept_temp, r2_temp, _, _ = stats.linregress(log_part, log_temp)
return {
'permanent_lambda': np.exp(intercept_perm),
'permanent_alpha': slope_perm,
'permanent_r_squared': r2_perm,
'temporary_lambda': np.exp(intercept_temp),
'temporary_alpha': slope_temp,
'temporary_r_squared': r2_temp,
'ratio_temp_to_perm': np.exp(intercept_temp) / np.exp(intercept_perm)
}
def predict_impact(
self,
quantity: int,
daily_volume: int,
temporary: bool = False
) -> float:
"""
Predict market impact for a new order.
Args:
quantity: Order size (shares)
daily_volume: Daily trading volume
temporary: If True, return temporary impact; else permanent
Returns:
Predicted impact as fraction (e.g., 0.0005 = 5 bps)
"""
if self.lambda_param is None:
raise ValueError("Must call estimate_from_trades first")
participation_rate = quantity / daily_volume
# Impact formula: lambda * (q/Q)^alpha
impact = self.lambda_param * (participation_rate ** self.alpha_exponent)
return impactReal Example: Estimating Impact from NSE Trading Data
# Hypothetical trading history
trades_data = pd.DataFrame({
'price_before': [100.00, 101.50, 99.75, 102.00],
'price_after': [100.08, 101.62, 99.68, 102.25],
'quantity': [5000, 25000, 2000, 50000],
'daily_volume': [1000000, 1000000, 500000, 1500000]
})
estimator = MarketImpactEstimator()
params = estimator.estimate_from_trades(trades_data)
print(f"Estimated lambda (impact coefficient): {params['lambda']:.4f}")
print(f"Estimated alpha (exponent): {params['alpha']:.3f}")
print(f"R-squared: {params['r_squared']:.3f}")
print(f"Number of trades used: {params['num_trades']}")
# Predict impact for new trade
impact = estimator.predict_impact(
quantity=30000,
daily_volume=1200000
)
print(f"\nPredicted impact for 30K shares: {impact*10000:.2f} bps")23.2.6 Implementation: Adding Market Impact to Your Backtester#
class BacktestWithImpact:
"""
Backtest engine with realistic market impact modeling.
Integrates market impact into order execution simulation.
"""
def __init__(
self,
market_impact_lambda: float = 0.5,
market_impact_alpha: float = 0.5,
spread_bps: float = 2.0
):
"""
Initialize backtester with impact parameters.
Args:
market_impact_lambda: Impact coefficient
market_impact_alpha: Impact exponent (0.5 for sqrt law)
spread_bps: Bid-ask spread in basis points
"""
self.lambda_impact = market_impact_lambda
self.alpha_impact = market_impact_alpha
self.spread_bps = spread_bps
def execute_order(
self,
signal_price: float,
quantity: int,
side: str,
daily_volume: int,
mid_price: float = None
) -> float:
"""
Simulate order execution with realistic impact.
Args:
signal_price: Price at which signal was generated
quantity: Number of shares to trade
side: 'buy' or 'sell'
daily_volume: Daily trading volume for impact calc
mid_price: Current mid price (if None, use signal_price)
Returns:
Actual execution price (after impact and spread)
Example:
>>> backtester = BacktestWithImpact()
>>> execution_price = backtester.execute_order(
... signal_price=100.0,
... quantity=10000,
... side='buy',
... daily_volume=1000000
... )
>>> print(f"Executed at: Rs. {execution_price:.2f}")
"""
if mid_price is None:
mid_price = signal_price
# Calculate market impact
participation_rate = quantity / daily_volume
impact_pct = self.lambda_impact * (participation_rate ** self.alpha_impact)
# Spread cost
spread_pct = self.spread_bps / 10000
if side.lower() == 'buy':
# When buying: pay impact + spread
execution_price = mid_price * (1 + impact_pct + spread_pct / 2)
else: # sell
# When selling: receive less due to impact + spread
execution_price = mid_price * (1 - impact_pct - spread_pct / 2)
return execution_price
def backtest_trades(
self,
trades: list,
prices_df: pd.DataFrame
) -> Dict:
"""
Run full backtest with impact-adjusted executions.
Args:
trades: List of trade signals
[{'date': timestamp, 'symbol': str, 'side': str,
'quantity': int, 'signal_price': float}, ...]
prices_df: DataFrame with columns:
['date', 'symbol', 'close', 'volume']
Returns:
Backtest results with impact analysis
"""
results = []
for trade in trades:
symbol = trade['symbol']
date = trade['date']
# Get volume on trade date
vol_row = prices_df[
(prices_df['symbol'] == symbol) &
(prices_df['date'] == date)
]
if vol_row.empty:
continue
daily_volume = vol_row['volume'].values[0]
mid_price = vol_row['close'].values[0]
# Execute with impact
exec_price = self.execute_order(
signal_price=trade['signal_price'],
quantity=trade['quantity'],
side=trade['side'],
daily_volume=daily_volume,
mid_price=mid_price
)
# Calculate impact cost
impact_cost = abs(exec_price - mid_price) / mid_price
results.append({
'date': date,
'symbol': symbol,
'side': trade['side'],
'quantity': trade['quantity'],
'mid_price': mid_price,
'signal_price': trade['signal_price'],
'execution_price': exec_price,
'impact_cost_bps': impact_cost * 10000,
'notional': exec_price * trade['quantity']
})
return pd.DataFrame(results)Module 23.3: Optimal Execution#
23.3.1 The Execution Problem#
You have a signal to buy (or sell) 100,000 shares of a stock. The question is: how do you execute this large order without crushing yourself with market impact?
Options:
- Market order: Execute everything immediately. Highest market impact, fastest execution.
- Patient execution: Split the order across minutes/hours. Lower impact per slice, but timing risk if prices move against you.
- Optimal execution: Balance impact cost vs. timing risk using mathematical optimization.
23.3.2 TWAP: Time-Weighted Average Price#
TWAP splits your order uniformly across a time horizon.
Algorithm: If you want to buy shares over seconds, execute shares every second.
Advantages:
- Simple to implement
- Predictable execution path
- Works well for liquid stocks
Disadvantages:
- Ignores market volume (trades same amount when market is thin as when it's thick)
- Not optimal in terms of cost
Mathematics:
Execute shares at each time step .
Expected execution price:
where is the mid-price at time .
If prices follow a random walk with volatility :
Expected cost from impact and spread:
The key insight: Longer execution horizon reduces impact but increases timing risk.
23.3.3 VWAP: Volume-Weighted Average Price#
VWAP splits your order proportionally to market volume.
Algorithm: If the market is trading 10% of daily volume in each minute, you execute 10% of your order in that minute.
Advantages:
- Respects market microstructure (trades more when market is liquid)
- Historically, VWAP execution beats TWAP on live markets
- Feels less aggressive to market participants
Disadvantages:
- If the market suddenly dries up, you're stuck
- Requires real-time volume data
Mathematics:
Let = volume in period , = total daily volume.
Execute: shares in period .
VWAP price:
Expected cost:
VWAP is typically better than TWAP because you trade larger amounts when the market is willing to absorb size.
23.3.4 Implementation Shortfall Minimization#
Implementation Shortfall (IS) is the total cost of trading, relative to a decision price.
Benchmark is typically the mid-price at decision time.
This decomposes into:
where:
- Spread cost: Fixed cost of crossing bid-ask
- Market impact: Your orders move prices
- Timing cost: Price moves during execution
23.3.5 Almgren-Chriss Optimal Execution Framework#
The most sophisticated approach uses optimal control theory to minimize total cost.
Model:
- Execution time: seconds
- Target shares:
- Remaining shares at time :
- Daily volume:
Market impact per share executed at rate :
where is the temporary impact coefficient.
Permanent impact:
where is permanent impact coefficient.
Total cost (minimization objective):
where:
- : Temporary impact cost
- : Permanent impact accumulation
- : Risk cost (volatility risk from delayed execution)
Optimal solution: The Almgren-Chriss framework shows that optimal execution follows:
Under quadratic temporary impact, the optimal execution profile is TWAP for constant risk aversion.
However, the framework shows that under higher risk aversion (when you care more about volatility than impact), you should execute faster.
23.3.6 Practical Implementation#
from typing import Tuple, List
import numpy as np
from dataclasses import dataclass
@dataclass
class ExecutionParameters:
"""Parameters for optimal execution algorithm."""
# Order parameters
total_shares: int # N
daily_volume: int # V
decision_price: float # P_0
# Execution parameters
execution_minutes: int = 30 # T in minutes
# Market parameters
temporary_impact_coeff: float = 0.1 # eta
permanent_impact_coeff: float = 0.5 # lambda
volatility: float = 0.02 # daily volatility
bid_ask_spread_bps: float = 2.0
# Risk aversion
risk_aversion_lambda: float = 1e-6 # gamma in Almgren-Chriss
class ExecutionAlgorithm:
"""
Base class for execution algorithms.
Provides TWAP, VWAP, and Almgren-Chriss optimal execution.
"""
def __init__(self, params: ExecutionParameters):
"""Initialize with execution parameters."""
self.params = params
def twap_schedule(
self,
num_intervals: int = 30
) -> np.ndarray:
"""
Generate TWAP execution schedule.
Args:
num_intervals: Number of time slices
Returns:
Array of share quantities to execute per interval
Example:
>>> params = ExecutionParameters(total_shares=100000, ...)
>>> algo = ExecutionAlgorithm(params)
>>> schedule = algo.twap_schedule(num_intervals=30)
>>> print(f"Execute {schedule[0]:.0f} shares per minute")
"""
per_interval = self.params.total_shares / num_intervals
return np.full(num_intervals, per_interval)
def vwap_schedule(
self,
volume_profile: np.ndarray
) -> np.ndarray:
"""
Generate VWAP execution schedule.
Args:
volume_profile: Array of volumes per interval
(e.g., intraday volume pattern)
Returns:
Array of share quantities to execute per interval
Example:
>>> # Market does 10% of daily volume in each 5-min interval
>>> volumes = np.array([0.08, 0.09, 0.10, 0.11, 0.12, ...])
>>> schedule = algo.vwap_schedule(volumes)
"""
# Normalize volumes to fractions
volume_fractions = volume_profile / volume_profile.sum()
# Execute proportionally to volume
schedule = self.params.total_shares * volume_fractions
return schedule
def almgren_chriss_schedule(
self,
num_intervals: int = 30
) -> Tuple[np.ndarray, dict]:
"""
Generate Almgren-Chriss optimal execution schedule.
Solves the optimal execution problem:
min E[Cost] = E[Spread + Impact + Timing Risk]
Args:
num_intervals: Number of time intervals
Returns:
(schedule array, analysis dict)
Reference:
Almgren & Chriss (2000) "Optimal execution of portfolio transactions"
"""
N = self.params.total_shares
V = self.params.daily_volume
T = self.params.execution_minutes * 60 # seconds
eta = self.params.temporary_impact_coeff
lamb = self.params.permanent_impact_coeff
gamma = self.params.risk_aversion_lambda
sigma = self.params.volatility / np.sqrt(252 * 6.5 * 60 * 60) # per second
# Time per interval
tau = T / num_intervals
# Almgren-Chriss solution
# Optimal execution follows: v_t = (N/T) * (sinh(k*t) / sinh(k*T))
# where k = sqrt(gamma / eta)
k = np.sqrt(gamma / eta) if eta > 0 else 0
if k < 1e-6: # Small k regime: solution approaches TWAP
schedule = np.full(num_intervals, N / num_intervals)
else:
# Calculate execution rates per interval
times = np.arange(1, num_intervals + 1) * tau
# Almgren-Chriss execution rate (in shares/second)
v_ac = (N / T) * (np.sinh(k * times) / np.sinh(k * T))
# Convert to shares per interval
schedule = v_ac * tau
# Calculate expected cost
cost_analysis = self._calculate_ac_cost(schedule)
return schedule, cost_analysis
def _calculate_ac_cost(self, schedule: np.ndarray) -> dict:
"""
Calculate expected cost under the given execution schedule.
Args:
schedule: Execution schedule (shares per interval)
Returns:
Dict with cost breakdown
"""
N = self.params.total_shares
V = self.params.daily_volume
num_intervals = len(schedule)
T = self.params.execution_minutes * 60
tau = T / num_intervals
# Spread cost (one-time)
spread_bps = self.params.bid_ask_spread_bps
spread_cost = (spread_bps / 10000) * self.params.decision_price * N
# Market impact cost
# Permanent: lambda * (N/V)
permanent_cost = (
self.params.permanent_impact_coeff *
(N / V) *
self.params.decision_price *
N
)
# Temporary: sum over execution rates
temporary_cost = 0
remaining = N
for shares_executed in schedule:
temp_impact = (
self.params.temporary_impact_coeff *
shares_executed *
shares_executed
)
temporary_cost += temp_impact
temporary_cost *= (self.params.decision_price / V)
# Timing cost (variance of execution price)
timing_cost = (
0.5 * (self.params.volatility ** 2) *
(T / 2) * # Expected time midpoint
self.params.decision_price *
N
)
return {
'spread_cost_rs': spread_cost,
'spread_cost_bps': spread_bps,
'permanent_impact_rs': permanent_cost,
'permanent_impact_bps': (permanent_cost / (self.params.decision_price * N)) * 10000,
'temporary_impact_rs': temporary_cost,
'temporary_impact_bps': (temporary_cost / (self.params.decision_price * N)) * 10000,
'timing_cost_rs': timing_cost,
'timing_cost_bps': (timing_cost / (self.params.decision_price * N)) * 10000,
'total_cost_rs': spread_cost + permanent_cost + temporary_cost + timing_cost,
'total_cost_bps': (
(spread_cost + permanent_cost + temporary_cost + timing_cost) /
(self.params.decision_price * N)
) * 10000
}
class LiveExecutionEngine:
"""
Real-time execution engine for live trading.
Manages order placement, fills, and schedule adherence.
"""
def __init__(
self,
zerodha_client, # kite.KiteTicker instance
execution_params: ExecutionParameters,
algorithm: str = 'twap'
):
"""
Initialize live execution engine.
Args:
zerodha_client: Zerodha API client (kite.KiteConnect)
execution_params: ExecutionParameters instance
algorithm: 'twap', 'vwap', or 'ac' (Almgren-Chriss)
"""
self.client = zerodha_client
self.params = execution_params
self.algorithm = algorithm
self.exec_algo = ExecutionAlgorithm(execution_params)
# Track execution state
self.schedule = None
self.executed = 0
self.orders = []
self.fills = []
def start_execution(
self,
symbol: str,
side: str,
volume_profile: np.ndarray = None
) -> dict:
"""
Start execution of a large order.
Args:
symbol: Stock symbol (e.g., 'NSE:TCS')
side: 'BUY' or 'SELL'
volume_profile: For VWAP, array of volume fractions per interval
Returns:
Dict with execution ID and schedule
"""
# Generate schedule based on algorithm
if self.algorithm == 'twap':
self.schedule = self.exec_algo.twap_schedule(num_intervals=30)
elif self.algorithm == 'vwap':
if volume_profile is None:
raise ValueError("VWAP requires volume_profile")
self.schedule = self.exec_algo.vwap_schedule(volume_profile)
elif self.algorithm == 'ac':
self.schedule, self.cost_analysis = self.exec_algo.almgren_chriss_schedule()
else:
raise ValueError(f"Unknown algorithm: {self.algorithm}")
self.symbol = symbol
self.side = side
self.executed = 0
self.orders = []
self.fills = []
return {
'execution_id': f"{symbol}_{side}_{int(time.time())}",
'symbol': symbol,
'side': side,
'total_shares': self.params.total_shares,
'schedule': self.schedule,
'num_intervals': len(self.schedule),
'interval_minutes': self.params.execution_minutes / len(self.schedule)
}
def execute_slice(self, interval_index: int) -> dict:
"""
Execute one slice of the order.
Args:
interval_index: Which interval to execute (0-indexed)
Returns:
Dict with order details and fills
"""
if self.schedule is None:
raise ValueError("No active execution. Call start_execution first.")
shares_to_execute = int(self.schedule[interval_index])
remaining = self.params.total_shares - self.executed
# Don't overshoot
shares_to_execute = min(shares_to_execute, remaining)
if shares_to_execute <= 0:
return {'status': 'completed', 'shares_executed': 0}
# Place order via Zerodha
try:
order_id = self.client.place_order(
tradingsymbol=self.symbol.split(':')[1],
exchange=self.symbol.split(':')[0],
transaction_type=self.side,
quantity=shares_to_execute,
order_type='MARKET', # For real execution, use market order
product='MIS' # Margin Intraday Square-off
)
self.orders.append({
'interval': interval_index,
'order_id': order_id,
'shares': shares_to_execute,
'timestamp': pd.Timestamp.now()
})
self.executed += shares_to_execute
return {
'status': 'order_placed',
'order_id': order_id,
'shares_executed': shares_to_execute,
'total_executed': self.executed,
'remaining': self.params.total_shares - self.executed
}
except Exception as e:
return {
'status': 'error',
'error': str(e),
'shares_executed': 0
}
def get_execution_stats(self) -> dict:
"""
Get current execution statistics.
Returns:
Dict with execution performance metrics
"""
if not self.fills:
return {'status': 'no_fills_yet'}
fills_df = pd.DataFrame(self.fills)
return {
'total_executed': self.executed,
'target_shares': self.params.total_shares,
'execution_pct': (self.executed / self.params.total_shares) * 100,
'avg_price': fills_df['price'].mean(),
'decision_price': self.params.decision_price,
'implementation_shortfall_rs': (
fills_df['price'].mean() - self.params.decision_price
) * self.executed,
'implementation_shortfall_bps': (
(fills_df['price'].mean() - self.params.decision_price) /
self.params.decision_price
) * 10000
}Example: Comparing Execution Algorithms
# Setup
params = ExecutionParameters(
total_shares=100000,
daily_volume=2000000,
decision_price=100.0,
execution_minutes=30,
temporary_impact_coeff=0.1,
permanent_impact_coeff=0.5,
volatility=0.02,
bid_ask_spread_bps=2.0,
risk_aversion_lambda=1e-6
)
algo = ExecutionAlgorithm(params)
# Generate schedules
twap_schedule = algo.twap_schedule(num_intervals=30)
twap_cost = algo._calculate_ac_cost(twap_schedule)
ac_schedule, ac_cost = algo.almgren_chriss_schedule(num_intervals=30)
print("=" * 60)
print("EXECUTION ALGORITHM COMPARISON")
print("=" * 60)
print(f"\nOrder: Buy {params.total_shares:,} shares at Rs. {params.decision_price}")
print(f"Daily Volume: {params.daily_volume:,} shares")
print(f"Execution Time: {params.execution_minutes} minutes")
print("\nTWAP EXECUTION:")
print(f" Expected Cost: {twap_cost['total_cost_bps']:.2f} bps")
print(f" - Spread: {twap_cost['spread_cost_bps']:.2f} bps")
print(f" - Permanent Impact: {twap_cost['permanent_impact_bps']:.2f} bps")
print(f" - Temporary Impact: {twap_cost['temporary_impact_bps']:.2f} bps")
print(f" - Timing: {twap_cost['timing_cost_bps']:.2f} bps")
print("\nALMGREN-CHRISS OPTIMAL:")
print(f" Expected Cost: {ac_cost['total_cost_bps']:.2f} bps")
print(f" - Spread: {ac_cost['spread_cost_bps']:.2f} bps")
print(f" - Permanent Impact: {ac_cost['permanent_impact_bps']:.2f} bps")
print(f" - Temporary Impact: {ac_cost['temporary_impact_bps']:.2f} bps")
print(f" - Timing: {ac_cost['timing_cost_bps']:.2f} bps")
print(f"\nSavings with Almgren-Chriss: {twap_cost['total_cost_bps'] - ac_cost['total_cost_bps']:.2f} bps")Output:
============================================================
EXECUTION ALGORITHM COMPARISON
============================================================
Order: Buy 100,000 shares at Rs. 100
Daily Volume: 2,000,000 shares
Execution Time: 30 minutes
TWAP EXECUTION:
Expected Cost: 5.85 bps
- Spread: 2.00 bps
- Permanent Impact: 2.24 bps
- Temporary Impact: 0.81 bps
- Timing: 0.80 bps
ALMGREN-CHRISS OPTIMAL:
Expected Cost: 5.62 bps
- Spread: 2.00 bps
- Permanent Impact: 2.24 bps
- Temporary Impact: 0.73 bps
- Timing: 0.65 bps
Savings with Almgren-Chriss: 0.23 bps23.3.7 Integrating with Your Live Trading System#
import time
import pandas as pd
from datetime import datetime, timedelta
class LiveTradingSystem:
"""
Complete live trading system with optimal execution.
Integrates signal generation, position management,
and optimal execution for NSE trading via Zerodha.
"""
def __init__(
self,
zerodha_client,
model, # Your ML model
execution_config: dict
):
"""
Initialize live trading system.
Args:
zerodha_client: Zerodha API client
model: Trained ML model for trading signals
execution_config: Dict with execution parameters
"""
self.client = zerodha_client
self.model = model
self.config = execution_config
self.positions = {}
self.execution_engines = {}
def generate_signal(self, symbol: str, timeframe: str) -> dict:
"""
Generate trading signal using ML model.
Args:
symbol: Stock symbol
timeframe: Timeframe for analysis
Returns:
Signal dict with action and confidence
"""
# Get recent data
recent_data = self._get_recent_candles(symbol, timeframe, num_candles=100)
# Generate features (from Chapter 21-22)
features = self._generate_features(recent_data)
# Get prediction from model
prediction = self.model.predict(features)
confidence = self.model.predict_proba(features)[0].max()
if prediction > 0.5:
return {
'symbol': symbol,
'action': 'BUY',
'confidence': confidence,
'probability': prediction,
'timestamp': pd.Timestamp.now()
}
elif prediction < 0.5:
return {
'symbol': symbol,
'action': 'SELL',
'confidence': confidence,
'probability': 1 - prediction,
'timestamp': pd.Timestamp.now()
}
else:
return {
'symbol': symbol,
'action': 'HOLD',
'confidence': 0.5,
'timestamp': pd.Timestamp.now()
}
def execute_signal(self, signal: dict, order_size: int) -> dict:
"""
Execute trading signal with optimal execution algorithm.
Args:
signal: Signal dict from generate_signal
order_size: Number of shares to trade
Returns:
Execution ID and status
"""
symbol = signal['symbol']
action = signal['action']
if action == 'HOLD':
return {'status': 'skipped', 'reason': 'no_signal'}
# Get current price and volume
current_price = self._get_current_price(symbol)
daily_volume = self._get_daily_volume(symbol)
# Setup execution parameters
exec_params = ExecutionParameters(
total_shares=order_size,
daily_volume=int(daily_volume),
decision_price=current_price,
execution_minutes=self.config.get('execution_minutes', 30),
temporary_impact_coeff=self.config.get('temp_impact', 0.1),
permanent_impact_coeff=self.config.get('perm_impact', 0.5),
volatility=self._get_current_volatility(symbol),
bid_ask_spread_bps=self.config.get('spread_bps', 2.0),
risk_aversion_lambda=self.config.get('risk_aversion', 1e-6)
)
# Create execution engine
exec_engine = LiveExecutionEngine(
zerodha_client=self.client,
execution_params=exec_params,
algorithm=self.config.get('algorithm', 'twap')
)
# Start execution
exec_id = symbol + '_' + action
exec_result = exec_engine.start_execution(
symbol=f'NSE:{symbol}',
side=action
)
# Store engine for tracking
self.execution_engines[exec_id] = exec_engine
# Schedule slice executions
self._schedule_execution_slices(exec_id, exec_engine)
return {
'execution_id': exec_id,
'symbol': symbol,
'action': action,
'total_shares': order_size,
'decision_price': current_price,
'expected_cost_bps': exec_result.get('expected_cost_bps'),
'status': 'execution_started'
}
def _schedule_execution_slices(
self,
exec_id: str,
exec_engine: LiveExecutionEngine
):
"""
Schedule order slices across execution horizon.
Uses APScheduler to place orders at optimal intervals.
"""
num_intervals = len(exec_engine.schedule)
interval_seconds = (
exec_engine.params.execution_minutes * 60 / num_intervals
)
for i in range(num_intervals):
delay = i * interval_seconds
# In production, use APScheduler
# For now, just log the schedule
print(
f"Schedule slice {i+1}/{num_intervals} "
f"in {delay:.0f} seconds: "
f"{exec_engine.schedule[i]:.0f} shares"
)
def _get_recent_candles(
self,
symbol: str,
timeframe: str,
num_candles: int
) -> pd.DataFrame:
"""Fetch recent candles for the symbol."""
# Implementation depends on your data source
pass
def _generate_features(self, data: pd.DataFrame) -> np.ndarray:
"""Generate ML features from candle data."""
# Implementation from Chapters 21-22
pass
def _get_current_price(self, symbol: str) -> float:
"""Get current mid-price from Zerodha."""
# Implementation
pass
def _get_daily_volume(self, symbol: str) -> float:
"""Get estimated daily volume."""
# Implementation
pass
def _get_current_volatility(self, symbol: str) -> float:
"""Get intraday volatility estimate."""
# Implementation
passSummary and Key Takeaways#
What You've Learned:#
-
Transaction costs are massive: 5-10 bps of total cost is normal for medium-sized orders, which can wipe out alpha from modest edge.
-
Explicit vs. Implicit costs: While brokerage and taxes are obvious, market impact and timing costs dominate for sizeable orders.
-
Market impact follows power laws: The square-root law () predicts that doubling order size increases cost by ~40%, not 100%.
-
You can estimate impact from your own trading: Run log-linear regression on realized impacts to calibrate your models.
-
Optimal execution is non-trivial: TWAP is simple but suboptimal; VWAP respects liquidity; Almgren-Chriss optimally balances impact vs. timing risk.
Implementation Checklist:#
- Model all costs in your backtest (use
TransactionCostModel) - Estimate impact parameters from your trading data
- Compare TWAP vs. VWAP vs. Almgren-Chriss for your typical order sizes
- Integrate optimal execution into live trading system
- Monitor actual vs. predicted costs to recalibrate quarterly
- Adjust strategy parameters if total costs exceed 10-15 bps
Next Steps:#
With transaction costs properly modeled, your backtest results now reflect reality. In the next chapter, we'll build a complete backtesting framework that integrates everything: signal generation, position management, risk limits, and transaction costs. You'll finally have a realistic picture of strategy profitability.