Entity Linking and Knowledge Graph Construction › Unlinkable Mentions and Entity Typing

A closed-KB disambiguation benchmark may assume every gold mention has a KB referent. An open-KB task relaxes that assumption and needs NIL evaluation. Independently of the gold label, an application may abstain when evidence is insufficient. Report which setting the system and its metrics implement.

Possible abstention features include:

• The top-ranked candidate’s score $s(e^*, m)$.
• The margin $s(e^*, m) - s(e^{(2)}, m)$ between the top two candidates.
• The similarity assigned to the top-ranked candidate by the retriever, when the retriever supplies one.

These features indicate uncertainty under the scorer, not proof of KB absence. A low margin may mean two valid KB candidates are difficult to distinguish. An empty candidate list may mean an alias lookup failed for a known entity. To predict NIL itself, validate the decision rule against both in-KB and out-of-KB gold mentions. A threshold can shift when the corpus, candidate generator, or KB changes.

An acceptance policy should reflect error costs. If false accepted links are more costly than delayed assertions, abstention may be preferable to forcing a candidate. Rejecting a link that would otherwise be accepted lowers accepted coverage for a fixed mention set; it does not guarantee a correct remaining graph. Mislabeling an in-KB referent as NIL can also create duplicate local nodes, so record abstention separately when the evidence establishes uncertainty rather than absence.

Fine-grained entity typing predicts categories such as person/artist/musician, with the hierarchy determined by the chosen type inventory. ReFinED uses fine-grained typing jointly with linking. Such predictions can contribute compatibility evidence, but a predicted type should not be silently treated as a verified ontology constraint. Application validation rules must also specify how they handle missing or uncertain types.