Research Pipeline & Deployment: Strategy Lifecycle Guide › Version Control and Experiment Tracking
Quantitative FinanceAll chapters Research Pipeline & Deployment: Strategy Lifecycle Guide › Version Control and Experiment TrackingOne section · Sections in this chapter3 / 9 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
As research progresses, you accumulate hundreds of experiments across multiple strategy variants. Without systematic tracking, reproducing results becomes impossible, and you lose the ability to understand what worked and why.
Git for Code Versioning
Version control with Git is foundational. Every change to strategy code, data processing pipelines, and configuration files should be tracked. A well-organized repository structure separates concerns and makes collaboration easier.
The repository structure below reflects a principle of separation of concerns: data handling lives in one place, signal generation in another, and execution logic in a third. This separation ensures that a bug fix in execution code cannot accidentally break signal calculation. It also enables different team members to work on different components simultaneously without creating merge conflicts. The notebooks directory provides a space for exploratory analysis, while the src directory contains production-quality code that has been tested and reviewed.
# Recommended repository structure for quant research
repository_structure = """
quant-research/
├── README.md
├── requirements.txt
├── setup.py
├── .gitignore
│
├── config/
│ ├── strategy_params.yaml
│ └── data_sources.yaml
│
├── data/
│ ├── raw/ # Original data (often gitignored)
│ ├── processed/ # Cleaned data
│ └── features/ # Computed features
│
├── src/
│ ├── __init__.py
│ ├── data/
│ │ ├── loaders.py
│ │ └── cleaners.py
│ ├── features/
│ │ ├── signals.py
│ │ └── transforms.py
│ ├── models/
│ │ ├── portfolio.py
│ │ └── risk.py
│ ├── backtest/
│ │ ├── engine.py
│ │ └── metrics.py
│ └── execution/
│ ├── paper_trading.py
│ └── production.py
│
├── notebooks/
│ ├── 01_data_exploration.ipynb
│ ├── 02_signal_research.ipynb
│ └── 03_backtest_analysis.ipynb
│
├── experiments/
│ ├── configs/ # Experiment configurations
│ └── results/ # Experiment outputs
│
├── tests/
│ ├── test_signals.py
│ └── test_backtest.py
│
└── scripts/
├── run_backtest.py
└── deploy_strategy.py
"""
# Key .gitignore entries for quant research
gitignore_content = """
# Data files (often too large for git)
data/raw/
*.csv
*.parquet
*.h5
# Credentials and secrets
.env
credentials/
*_secret*
# Experiment outputs (track configs, not results)
experiments/results/
*.pkl
# Python
__pycache__/
*.pyc
.venv/
*.egg-info/
# Jupyter
.ipynb_checkpoints/
# IDE
.vscode/
.idea/
"""Recommended Repository Structure:
quant-research/
├── README.md
├── requirements.txt
├── setup.py
├── .gitignore
│
├── config/
│ ├── strategy_params.yaml
│ └── data_sources.yaml
│
├── data/
│ ├── raw/ # Original data (often gitignored)
│ ├── processed/ # Cleaned data
│ └── features/ # Computed features
│
├── src/
│ ├── __init__.py
│ ├── data/
│ │ ├── loaders.py
│ │ └── cleaners.py
│ ├── features/
│ │ ├── signals.py
│ │ └── transforms.py
│ ├── models/
│ │ ├── portfolio.py
│ │ └── risk.py
│ ├── backtest/
│ │ ├── engine.py
│ │ └── metrics.py
│ └── execution/
│ ├── paper_trading.py
│ └── production.py
│
├── notebooks/
│ ├── 01_data_exploration.ipynb
│ ├── 02_signal_research.ipynb
│ └── 03_backtest_analysis.ipynb
│
├── experiments/
│ ├── configs/ # Experiment configurations
│ └── results/ # Experiment outputs
│
├── tests/
│ ├── test_signals.py
│ └── test_backtest.py
│
└── scripts/
├── run_backtest.py
└── deploy_strategy.pyThis structure clearly separates data, source code, and experimental notebooks. Placing reusable logic in the src directory ensures that both notebooks and production scripts import the same tested code, preventing logic divergence. The tests directory is not optional: automated tests catch regressions before they reach production. The experiments directory separates configuration files, which should be version controlled, from output files, which are often too large and numerous to store in Git.
Experiment Tracking with MLflow
While Git tracks code, experiment tracking tools like MLflow capture the full context of each run, including parameters, metrics, artifacts, and environment details. This allows comparing experiments across different code versions and parameter combinations.
The value of experiment tracking becomes apparent when you need to reproduce a result from six months ago. With only Git, you would know what code was committed but not which specific parameters generated the impressive backtest you remember. With experiment tracking, every run is recorded with its complete configuration, making reproduction straightforward. This capability is essential for auditing, for explaining results to stakeholders, and for building on previous work rather than rediscovering insights that were already found and forgotten.
import hashlib
from datetime import datetime
from pathlib import Path
from typing import Any
import numpy as np
import pandas as pd
class SimpleExperimentTracker:
"""
Lightweight experiment tracking inspired by MLflow.
In production, use MLflow, Weights & Biases, or similar tools.
"""
def __init__(self, experiment_dir: str = "experiments"):
self.experiment_dir = Path(experiment_dir)
self.experiment_dir.mkdir(exist_ok=True)
self.current_run = None
self.runs = []
def start_run(self, run_name: str = None) -> str:
"""Start a new experiment run."""
run_id = hashlib.md5(
f"{datetime.now().isoformat()}{run_name}".encode()
).hexdigest()[:8]
self.current_run = {
"run_id": run_id,
"run_name": run_name or f"run_{run_id}",
"start_time": datetime.now().isoformat(),
"parameters": {},
"metrics": {},
"tags": {},
"artifacts": [],
}
return run_id
def log_param(self, key: str, value: Any):
"""Log a parameter for the current run."""
if self.current_run is None:
raise RuntimeError("No active run. Call start_run() first.")
self.current_run["parameters"][key] = value
def log_params(self, params: dict):
"""Log multiple parameters."""
for key, value in params.items():
self.log_param(key, value)
def log_metric(self, key: str, value: float, step: int = None):
"""Log a metric for the current run."""
if self.current_run is None:
raise RuntimeError("No active run. Call start_run() first.")
if key not in self.current_run["metrics"]:
self.current_run["metrics"][key] = []
self.current_run["metrics"][key].append(
{
"value": value,
"step": step,
"timestamp": datetime.now().isoformat(),
}
)
def log_metrics(self, metrics: dict, step: int = None):
"""Log multiple metrics."""
for key, value in metrics.items():
self.log_metric(key, value, step)
def set_tag(self, key: str, value: str):
"""Set a tag for the current run."""
if self.current_run is None:
raise RuntimeError("No active run. Call start_run() first.")
self.current_run["tags"][key] = value
def end_run(self):
"""End the current run and save results."""
if self.current_run is None:
return
self.current_run["end_time"] = datetime.now().isoformat()
# Simplify metrics to final values for storage
simplified_metrics = {}
for key, values in self.current_run["metrics"].items():
simplified_metrics[key] = values[-1]["value"] if values else None
self.current_run["final_metrics"] = simplified_metrics
self.runs.append(self.current_run)
self.current_run = None
def get_runs_dataframe(self) -> pd.DataFrame:
"""Get all runs as a DataFrame."""
records = []
for run in self.runs:
record = {
"run_id": run["run_id"],
"run_name": run["run_name"],
"start_time": run["start_time"],
**run["parameters"],
**run.get("final_metrics", {}),
}
records.append(record)
return pd.DataFrame(records)
# Demonstrate experiment tracking
tracker = SimpleExperimentTracker()
# Run multiple experiments with tracking
strategies_to_test = [
{"name": "momentum_fast", "lookback": 63, "holding": 5},
{"name": "momentum_medium", "lookback": 126, "holding": 21},
{"name": "momentum_slow", "lookback": 252, "holding": 63},
]
np.random.seed(456)
for strategy in strategies_to_test:
tracker.start_run(run_name=strategy["name"])
# Log parameters
tracker.log_params(
{
"lookback_days": strategy["lookback"],
"holding_period": strategy["holding"],
"universe": "SP500",
"start_date": "2018-01-01",
"end_date": "2022-12-31",
}
)
# Set tags for organization
tracker.set_tag("strategy_type", "momentum")
tracker.set_tag("researcher", "quant_team")
# Simulate running backtest and logging metrics
for month in range(1, 13):
monthly_return = np.random.normal(0.005, 0.03)
tracker.log_metric(
"cumulative_return", monthly_return * month, step=month
)
# Log final metrics
tracker.log_metrics(
{
"sharpe_ratio": np.random.uniform(0.4, 1.2),
"annual_return": np.random.uniform(0.03, 0.12),
"max_drawdown": np.random.uniform(-0.20, -0.05),
"calmar_ratio": np.random.uniform(0.2, 0.8),
}
)
tracker.end_run()# Review tracked experiments
runs_df = tracker.get_runs_dataframe()
display_cols = [
"run_name",
"lookback_days",
"holding_period",
"sharpe_ratio",
"annual_return",
"max_drawdown",
]Tracked Experiments
======================================================================
run_name lookback_days holding_period sharpe_ratio annual_return max_drawdown
momentum_fast 63 5 0.861 0.043 -0.097
momentum_medium 126 21 0.571 0.118 -0.071
momentum_slow 252 63 0.693 0.102 -0.198The tracking results highlight the trade-offs between different strategy variants. The 'momentum_slow' strategy demonstrates a higher Sharpe ratio and lower drawdown compared to the faster variants, indicating that reducing turnover and focusing on longer-term trends improves risk-adjusted returns in this specific test. This pattern is consistent with the economic intuition that transaction costs erode the returns of high-frequency strategies, but it requires validation on out-of-sample data before drawing firm conclusions.
Configuration Management
Separating configuration from code enables running the same strategy with different parameters without modifying source files. YAML configuration files are human-readable and work well with version control.
The benefit of configuration files extends beyond convenience. When parameters live in code, changing them requires a code commit, which triggers the full review and testing process. When parameters live in configuration files, you can experiment more quickly during research while still maintaining an audit trail. In production, configuration files enable running the same strategy with different risk limits for different accounts or market conditions without maintaining separate code branches.
!uv pip install PyYAML
import yaml
# Example strategy configuration
strategy_config = {
'strategy': {
'name': 'momentum_strategy_v2',
'version': '2.1.0',
'description': 'Cross-sectional momentum with volatility adjustment'
},
'universe': {
'source': 'sp500_constituents',
'min_price': 5.0,
'min_volume': 1000000,
'exclude_sectors': ['Utilities']
},
'signal': {
'type': 'momentum',
'lookback_days': 252,
'skip_days': 21,
'vol_adjustment': True,
'vol_lookback': 63
},
'portfolio': {
'rebalance_frequency': 'monthly',
'num_long': 50,
'num_short': 50,
'max_position_size': 0.04,
'target_volatility': 0.10
},
'risk': {
'max_sector_exposure': 0.25,
'max_single_stock': 0.05,
'stop_loss': -0.15
},
'execution': {
'order_type': 'VWAP',
'participation_rate': 0.10,
'max_spread_bps': 50
}
}
# Convert to YAML string for display
config_yaml = yaml.dump(strategy_config, default_flow_style=False, sort_keys=False)Strategy Configuration (YAML) ================================================== strategy: name: momentum_strategy_v2 version: 2.1.0 description: Cross-sectional momentum with volatility adjustment universe: source: sp500_constituents min_price: 5.0 min_volume: 1000000 exclude_sectors: - Utilities signal: type: momentum lookback_days: 252 skip_days: 21 vol_adjustment: true vol_lookback: 63 portfolio: rebalance_frequency: monthly num_long: 50 num_short: 50 max_position_size: 0.04 target_volatility: 0.1 risk: max_sector_exposure: 0.25 max_single_stock: 0.05 stop_loss: -0.15 execution: order_type: VWAP participation_rate: 0.1 max_spread_bps: 50