A bigram language model represents a statistical framework used in natural language processing (NLP) to predict the likelihood of a word based on the immediate previous word, thereby capturing dependencies within text sequences. This model employs Markov assumptions, simplifying the complexity of language by limiting context to just two consecutive words, and is crucial for tasks like automatic speech recognition, text generation, and spelling correction. By reducing computational complexity, bigram models allow efficient language prediction while sacrificing long-range syntactic or semantic dependencies, making them fundamental yet rudimentary in comparison to more advanced models like neural networks.
The conceptual foundation of bigram models can be traced back to the early days of AI and computational linguistics, gaining noticeable traction in the 1980s and 1990s as part of broader efforts to enhance machine understanding of human language through statistical methods.
Key contributors to the development of bigram models include pioneers in computational linguistics and statistical language processing such as Frederick Jelinek, who worked extensively at IBM on speech recognition and probabilistic models of language.