Language Model Harness

A language model harness (sometimes called an agent harness, eval harness, or LM harness) is a software framework that wraps a base or instruction-tuned language model and provides the surrounding plumbing for invoking it: prompt construction with templating, system-prompt and message-history management, tool calling and function-execution loops, structured-output parsing (JSON, tool-use schemas), retry and error handling, observability/tracing, batch evaluation drivers, and orchestration across multiple models or providers. The harness is what makes a model "usable" in production — the raw weights have no concept of messages, tools, or schemas. Notable harnesses include LangChain's orchestration layers, the Braintrust eval harness, lm-evaluation-harness from EleutherAI, and the harness components of agent frameworks like CrewAI, LangGraph, and OpenAI Agents SDK. The choice of harness substantially affects downstream behavior: the same model can produce very different outputs depending on prompt formatting, tool-calling conventions, and retry logic. The term is increasingly used in the context of agentic systems, where the harness's role in defining the agent's observable action space is comparable to the OS's role in defining a process's environment.