Protein language models (PMLs) are self‑supervised sequence models—typically transformers or recurrent architectures—trained on massive protein sequence corpora using objectives like masked‑token prediction or next‑token prediction to learn rich, contextual embeddings that encode evolutionary, structural and functional information implicit in amino‑acid patterns. In practice PMLs produce representations that improve downstream tasks (remote homology detection, contact/structure inference, function annotation, variant effect prediction) either via fine‑tuning or zero‑shot scoring, and they support generative design workflows by providing likelihoods and conditional decoding of sequences; their effectiveness emerges from transfer learning and scaling laws similar to NLP, but they also have limitations (sequence biases, incomplete modeling of 3D physics and explicit evolutionary coupling) that motivate hybrid approaches combining PMLs with MSAs, structural priors, and experimental validation.
First use: conceptually appeared in the late 2010s (~2017–2019); gained broad popularity from 2019–2022 as UniRep, TAPE, ProtTrans and large transformer efforts (e.g., ESM/ProtBERT family) demonstrated strong zero‑shot and transfer performance on structure and function tasks.