AI Coding and LLM Glossary

AI models and inference

Artificial intelligence (AI): Software techniques that perform tasks associated with reasoning, language, perception, prediction, or decision-making.
Large language model (LLM): A neural model trained to process and generate token sequences. Coding LLMs are tuned or trained with source code and technical text.
Inference: The process of running a trained model to generate an output from an input. See local LLM inference.
Local LLM: A language model whose inference runs on a computer or server controlled by the user instead of a third-party hosted endpoint.
Model provider: The service that exposes models to a client. It can be local, self-hosted, or remote.
Model endpoint: The network address used to reach a provider API, such as an Ollama server at 127.0.0.1:11434.
OpenAI-compatible API: An API that follows commonly adopted OpenAI request and response shapes, often including a /v1/models route.
Ollama: A local model runtime and API. Midnight Coder can detect its models and send model-aware context settings.
Quantization: A reduction in model numerical precision that usually lowers memory use and can improve local inference speed, with a possible quality tradeoff.
RAM and VRAM: System and graphics memory used to load model weights, cache context, and perform inference.

Token: A unit of text processed by an LLM. A token can represent part of a word, punctuation, whitespace, or code.
Context window: The maximum token budget a model can consider during one request, including instructions, history, code, tool results, and generated output.
Prompt: The instructions and input supplied to a model. In an agent, the effective prompt also includes runtime instructions and retrieved workspace context.
Context compaction: The process of replacing older detailed history with a shorter representation to keep a conversation within its token limit.
Thread: A persistent conversation that groups related turns and can be resumed.
Turn: One cycle of user input, model reasoning, tool activity, and output within a thread.
Item: An atomic entry in agent history, such as a message, command execution, file change, or tool result.

AI coding agent: An AI system that can inspect a codebase, plan work, invoke tools, edit files, and validate results. Read how a local coding agent works.
Agentic loop: The repeated cycle in which a model selects an action, receives its result, updates its understanding, and chooses the next action.
Tool calling: A structured mechanism that lets a model request actions from the runtime, such as reading a file or running a test.
Workspace: The directory and project files placed within scope for an agent session.
Sandbox: An enforcement boundary that limits filesystem, process, or network capabilities. See agent security and sandboxing.
Approval: An explicit decision required before a sensitive tool action proceeds.
Local-first: An architecture that keeps the runtime and primary workflow on the user's machine. It does not by itself guarantee local model inference.

SDK: A software development kit that exposes agent capabilities through a programming-language API.
JSON-RPC: A structured request/response protocol over JSON used by the Midnight Coder app server.
App server: A long-running interface that exposes thread, turn, configuration, approval, and event APIs to rich clients.
Streaming: Delivery of model output and tool events as they happen instead of waiting for the entire turn to finish.
Structured output: Model output constrained to a supplied schema so application code can parse it reliably.

See the Midnight Coder integration guide for the terminal, app server, Python, and TypeScript surfaces.

Explore how Midnight Coder connects models, context, tools, and workspace controls.