Word Analogy › Beyond Geometric Analogies: Relationship Composition and Transitivity

The analogy framework extends naturally to more complex reasoning patterns, though with diminishing reliability as the chains grow longer. If the embedding space encodes the capital-of relationship as a consistent offset $\vec{r}_{cap}$, then you can chain relationships: if France is to Paris as Germany is to Berlin, and Paris is to Eiffel Tower as Berlin is to Brandenburg Gate, then you might try to compose the two offsets to answer: "France is to Eiffel Tower as Germany is to ?". This is relationship composition: applying a sequence of relational offsets to follow multi-hop paths in the semantic graph.

In practice, composed analogies work less reliably than single-hop analogies because each step introduces noise. The first offset might point 5 degrees away from perfect, and the second offset adds another 5 degrees of error, compounding the directional drift. After two hops, the target vector may be far enough from the correct answer that a different word is the nearest neighbor.

Researchers have explored using analogy arithmetic for more structured reasoning tasks. Knowledge base completion, where the goal is to predict missing relationships in a structured knowledge graph, can be framed as analogy completion. Given (France, capital-of, Paris) and (Germany, capital-of, ?), the embedding space can sometimes predict Berlin. This approach, explored in TransE and related models, treats relationship types as translation vectors in the embedding space and explicitly trains the model to satisfy the parallelogram structure.

The success of TransE and similar approaches shows that the analogy insight generalizes beyond evaluation: if you design training objectives that explicitly enforce parallelogram structure for known relationships, you can achieve much stronger analogy performance on structured knowledge tasks. The Word2Vec and GloVe analogy results emerge as a byproduct of distributional training; TransE-style models achieve it as a primary objective.