Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Alternative Data and NLP in Quant Strategies › Key Parameters

Michael BrenndoerferJanuary 4, 20266 min read

Quantitative FinanceAll chapters

Part 13 of 15Machine Learning and Alternative Data

Chapter 71Alternative Data and NLP in Quant Strategies › Key Parameters

Reading section 5 of 18 · Key Parameters · 6 minSection 5 of 18 · 6 min

Section 5 of 18 · 6 min read

Alternative Data and NLP in Quant Strategies › Key ParametersOne section · Sections in this chapter5 / 18 ↓

Sections in this chapter18

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Key parameters for TF-IDF vectorization.

max_features: Maximum number of features to extract. Controls vocabulary size and dimensionality of the feature space.
ngram_range: Range of n-gram sizes to consider, such as (1,1) for unigrams only or (1,2) for unigrams and bigrams. Higher n-grams capture phrase-level patterns.
min_df: Minimum document frequency for a term to be included. Filters out rare words that may be noise.
max_df: Maximum document frequency for a term to be included. Filters out overly common words that provide little discriminative power.

Word Embeddings and Semantic RepresentationLink Copied

Bag-of-words and TF-IDF treat each word as an independent dimension, ignoring semantic relationships. The word "profit" and "earnings" would be as dissimilar as "profit" and "banana" in these representations, despite their obvious conceptual overlap. This limitation stems from a core assumption. Each word occupies its own orthogonal axis in the feature space with no connection to any other word. In mathematical terms, the dot product between any two different word vectors is zero, treating synonyms and antonyms with equal indifference.

Word embeddings address this limitation by learning dense vector representations where semantically similar words have similar vectors. Rather than assigning each word its own independent dimension, embeddings map words into a shared continuous vector space where geometric proximity reflects semantic relatedness. The key insight, pioneered by Word2Vec and subsequent models, is that a word's meaning can be inferred from the company it keeps. Words appearing in similar contexts should have similar representations. This distributional hypothesis, which states that words with similar distributions have similar meanings, gives the theoretical foundation for learning meaningful representations from raw text alone.

In embedding space, we might find the following.

"profit" and "earnings" have high cosine similarity
"bullish" and "optimistic" cluster together
Vector arithmetic captures relationships such that "CEO" minus "company" plus "country" approximates "president"

Modern transformer-based models like BERT go further, producing contextualized embeddings where a word's representation depends on its surrounding context. The word "bank" receives different embeddings in "river bank" versus "investment bank", resolving ambiguity that plagued earlier approaches.

Word embeddings capture semantic similarity through vector geometry. The cosine of the angle between word vectors quantifies how semantically related the words are. This geometric interpretation provides an elegant connection between linguistic meaning and mathematical structure. Words that humans judge as similar tend to point in similar directions in embedding space, while unrelated or opposite words point in different or opposing directions.

The cosine similarity between two word vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ is computed as:

\text{similarity}(\mathbf{v}_1, \mathbf{v}_2) = \frac{\mathbf{v}_1 \cdot \mathbf{v}_2}{\|\mathbf{v}_1\| \, \|\mathbf{v}_2\|}

where:

$\mathbf{v}_1$ : embedding vector for the first word
$\mathbf{v}_2$ : embedding vector for the second word
$\mathbf{v}_1 \cdot \mathbf{v}_2$ : dot product of the two vectors, computed as $\sum_i v_{1i} \cdot v_{2i}$ , measuring alignment
$\|\mathbf{v}\|$ : Euclidean norm (length) of vector $\mathbf{v}$ , computed as $\sqrt{\sum_i v_i^2}$
$\|\mathbf{v}_1\| \, \|\mathbf{v}_2\|$ : product of the two vector lengths, used for normalization
$i$ : index over the dimensions of the embedding vectors
Cosine similarity ranges from -1 (opposite meanings) to +1 (identical meanings), with 0 indicating orthogonality

This formula measures the cosine of the angle between two vectors, capturing their directional similarity independent of magnitude. Normalizing by both vector lengths removes absolute magnitude from the comparison. Two vectors could have very different lengths but still point in exactly the same direction, which would indicate semantic similarity. Vectors pointing in the same direction have high positive similarity, opposite directions have negative similarity, and perpendicular vectors have zero similarity. The cosine function provides exactly this behavior: it equals 1 when the angle is zero (parallel vectors), 0 when the angle is 90 degrees (orthogonal vectors), and -1 when the angle is 180 degrees (antiparallel vectors). Semantically similar words have vectors that point in similar directions, yielding high cosine similarity.

In[8]:

Code

# Simulated word embeddings for financial terms (in practice, use pre-trained models)
# These 4-dimensional vectors illustrate the concept
np.random.seed(42)

# Create embeddings where semantically similar words have similar vectors
embeddings = {
    "profit": np.array([0.8, 0.6, 0.1, 0.2]),
    "earnings": np.array([0.75, 0.65, 0.15, 0.18]),
    "loss": np.array([-0.7, -0.5, 0.1, 0.2]),
    "revenue": np.array([0.6, 0.7, 0.2, 0.1]),
    "growth": np.array([0.5, 0.8, 0.3, 0.1]),
    "decline": np.array([-0.5, -0.6, 0.2, 0.15]),
    "bullish": np.array([0.7, 0.4, 0.6, 0.3]),
    "bearish": np.array([-0.6, -0.35, 0.55, 0.25]),
}


def cosine_similarity(v1, v2):
    """Calculate cosine similarity between two vectors."""
    return np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))


# Calculate similarities
pairs = [
    ("profit", "earnings"),
    ("profit", "loss"),
    ("growth", "decline"),
    ("bullish", "bearish"),
]

# Compute similarities for output
similarities = []
for word1, word2 in pairs:
    sim = cosine_similarity(embeddings[word1], embeddings[word2])
    similarities.append((word1, word2, sim))

Out[9]:

Console

Cosine Similarities Between Word Embeddings:
---------------------------------------------
profit     vs earnings  : +0.996
profit     vs loss      : -0.889
growth     vs decline   : -0.803
bullish    vs bearish   : -0.161

The cosine similarity between "profit" and "earnings" rounds to 1.00, so these handcrafted vectors place the terms in nearly the same direction. Conversely, "profit" and "loss" have a strongly negative similarity of about -0.89. The "bullish" and "bearish" vectors are only weakly negative at about -0.16 because their shared dimensions offset much of their opposition. These values illustrate an important limitation of the example: the relationships depend on the geometry learned or specified by the embedding model, not on the words' dictionary meanings alone.

Out[10]:

Visualization

Eight-by-eight symmetric heatmap of cosine similarities for financial terms. The diagonal and the profit-earnings pair are near 1.00, the profit-loss pair is strongly negative, and bullish-bearish is only mildly negative. — Cosine similarity matrix for eight illustrative financial-term embeddings. The diagonal is 1.00, 'profit' and 'earnings' round to 1.00, and 'profit' and 'loss' are strongly opposed at -0.89. By contrast, the handcrafted 'bullish' and 'bearish' vectors are only weakly negative at -0.16, showing that similarity values depend on the embedding geometry rather than dictionary relationships alone.