NLP Lineage

1992 · Language modelling

Class-based n-grams

Class-Based n-gram Models of Natural Language

P. Brown, V. Della Pietra, P. deSouza, J. Lai, R. Mercer

Date
1992
What the date marks
Journal
Kind
Research paper
Organisation
IBM T.J. Watson
Access
Not applicable
Threads
In the guided tour
Chapter 1. Language as probability

Notable: no later milestone in the data is documented as descending from it. How levels are decided

Full summary in progress. Until then, this page shows the milestone's record from the research data.

What it changed

Before it

n-gram models counted exact word sequences and assigned zero probability to anything unseen.

The problem

Sparsity: most word sequences never appear in any corpus, however large.

What it introduced

Cluster words into classes automatically, so statistics are shared between similar words.

Why it matters

The first practical answer to sparsity by grouping words that behave alike, an idea embeddings later generalised.

What modern models inherit

The instinct that similar words should share statistical strength.

What came next

Bengio's neural language model replaced discrete classes with continuous vectors.

Before and after

The milestones it descends from, and the milestones that descend from it. Work developed alongside it is not counted. Weak links are left out unless you include them.

Before: where it came from

Class-based n-grams has 2 documented ancestors across 1 chapter, reaching back to Information theory in 1948. Showing all of them.

Class-based n-grams has 2 documented ancestors across 1 chapter, reaching back to Information theory in 1948. Showing all of them.

Class-based n-grams has 2 recorded ancestors, including indirect ones, across 1 chapter, reaching back to Information theory in 1948. Showing all of them.

Class-based n-grams has 2 recorded ancestors, including indirect ones, across 1 chapter, reaching back to Information theory in 1948. Showing all of them.

  1. 1992Class-based n-gramsThis milestone.
  2. 1990Statistical MTClass-based n-grams built directly on thisChapter 1. Language as probability
  3. 1948Information theoryClass-based n-grams was influenced by thisChapter 1. Language as probability
  1. 1992Class-based n-gramsThis milestone.
  2. 1990Statistical MTClass-based n-grams built directly on thisChapter 1. Language as probability
  3. 1948Information theoryClass-based n-grams was influenced by thisChapter 1. Language as probability

After: what it led to

No documented descendants are recorded for Class-based n-grams without weak links.

No documented descendants are recorded for Class-based n-grams without weak links.

No recorded descendants are recorded for Class-based n-grams.

No recorded descendants are recorded for Class-based n-grams.

Direct links and their sources

Where it came from

Built directly on

Influenced by

What it led to

None recorded in the sources used.