Machine Learning in Trading Strategy Design › Key Parameters

The key parameters for the Q-learning agent are:

• learning_rate: Controls how much new information overrides old information. In noisy markets, a lower rate helps average out stochastic rewards.
• discount_factor ($γ$): Determines the importance of future rewards. A value close to 1 makes the agent farsighted, while a lower value focuses on immediate returns.
• epsilon ($ϵ$): The probability of choosing a random action (exploration). Decaying this value over time allows the agent to exploit its learned policy.

Reinforcement learning excels in several trading applications where the sequential nature of decisions and the need to balance multiple objectives make traditional approaches awkward:

Optimal execution involves minimizing market impact when executing large orders. A fund that needs to buy a million shares faces a dilemma: trading quickly risks moving the market against itself, but trading slowly exposes the position to adverse price drift. The RL agent learns to balance urgency (completing the order quickly) against market impact (moving the price adversely), adapting its strategy to real-time market conditions like volatility and liquidity. This is explored further in Part VII when we cover execution algorithms.

Dynamic portfolio rebalancing uses RL to decide when and how much to rebalance, accounting for transaction costs, tax implications, and changing market conditions. Unlike fixed-schedule rebalancing (e.g., monthly on the first trading day), RL-based approaches adapt to market volatility and opportunity costs. When drift is small and costs are high, the agent learns to delay rebalancing; when positions have moved substantially or volatility has spiked, it learns to act quickly.

Market making benefits from RL's ability to learn optimal quote placement strategies that balance inventory risk against spread capture. The market maker's challenge is to profit from the bid-ask spread while avoiding accumulating large positions that expose it to directional risk. The agent learns when to widen spreads (during high volatility or low liquidity) and when to compete aggressively for order flow (in calm markets with balanced two-sided demand).