A code archaeology
From next-token prediction to tool-using software agents
IIT Gandhinagar · Agentic Engineering
LLMs did not suddenly “become programmers.” A text-prediction engine was specialized, given better interfaces, trained on feedback, and placed inside an execution loop.
01
Continue text
02
Learn code's distribution
03
Follow intent
04
Emit structured actions
05
Act, observe, retry
Training
context target
The quick brown fox → jumps
def add(a, b): → returnMinimize next-token prediction error across huge text corpora.
Inference
prompt + generated tokens
↓
next-token distribution
↓ sample
one more token ↺Code is text with unusually strong structure—and executable semantics.
1.5B
parameters
1,024
token context
WebText8M pagescausal LM
Diverse web text contains naturally occurring demonstrations: question → answer, English → French, comment → code.
Key idea: a sufficiently broad continuation model begins to behave like a multitask learner—without a labeled dataset for every task.
But: GPT-2's paper did not claim serious production-quality code generation.
An early bridge
A GPT-2-based, all-language code completer put a language model behind editor autocomplete in July 2019.
function slugify(input) {
// cursor ▌
}↓ TAB
return input
.toLowerCase()
.replace(/\s+/g, '-');175B
parameters
2,048
token context
zero-shotone-shotfew-shot
Description: red button saying Stop
Code: <button style=...>Stop</button>
Description: blue box with 3 circles
Code: ...
Description: rainbow buttons
Code: [model continues]The task is specified inside the context; model weights do not change at use time.
PRIMARY-SOURCE SCREENSHOT
Spectacular generation.
Not yet a repository-aware agent.
HUMAN
“Three colorful buttons”
MODEL
GPT-3 continues the pattern
TEXT
Plausible code response
HUMAN
Paste · render · debug · retry
GPT-3 could emit code. It could not inspect your repository, run the build, see the failure, or repair it.
The human was the integration layer and the feedback loop.
GPT-3 API preview; GitHub begins experimenting after release.
Question→snippet prototype gives way to inline IDE completion.
Copilot technical preview—Codex under the hood.
Codex paper: GPT-family models fine-tuned on public GitHub code.
Codex API beta: natural-language instructions to code.
Parallel histories: independent GPT-3 demos revealed a capability; GitHub and OpenAI specialized it into a model + editor product. The exact collaboration start date is not public.
up to 12BHumanEvalGitHub code
Still a probability model: specialization raises the probability of valid code; it does not install a compiler inside the weights.
A · Tab-complete world
code before cursor
▌
code after cursorVS
B · Instruct → copy world
“Write a React component…”
↓
code block
↓
copy / paste / runInsert
prefix + suffix → insertion
prompt: code before cursor
suffix: code after cursor
output: missing middleCompletion track: constrain the generated span with both sides.
Edit
instruction + input → edited copy
input: existing text
instruction: “fix the bug”
output: rewritten textInstruction track: state a desired transformation in natural language.
PRODUCT EVIDENCE
GitHub described Copilot as suggesting whole lines or entire functions as the developer typed.
active file + nearby code
+ cursor position
↓
ghost text
↓
accept / edit / rejectPrefix-only model
def clamp(x, low, high):
# cursor ▌
assert clamp(12, 0, 10) == 10The model sees the function prefix. The assertion below may be invisible to its prompt.
Fill in the middle
PREFIX: function + comment
SUFFIX: assertion below
MIDDLE: ?Generate code constrained by both what comes before and what must still come after.
Original
[ PREFIX ][ MIDDLE ][ SUFFIX ]
PSM
<fim_prefix> PREFIX
<fim_suffix> SUFFIX
<fim_middle> MIDDLE
SPM
<fim_suffix> SUFFIX
<fim_prefix> PREFIX
<fim_middle> MIDDLEThe decoder is still left-to-right. The data transform ensures it has attended to both sides before predicting the hole.
Editor buffer
function total(items) {
▋
}
console.log(total([
{ price: 2 },
{ price: 3 }
])); // 5Prefix-only knows the function name. FIM also sees the call shape, input schema, closing brace, and expected output.
Generated middle
return items.reduce(
(sum, { price }) => sum + price,
0
);The suffix becomes a lightweight specification.
FIM predicts a span. It still does not run the code or verify that the inferred behavior matches the user's real intent.
| Date | Model / work | What advanced |
|---|---|---|
| 15 Mar 2022 | OpenAI Insert | Product API accepted prefix + suffix |
| 12 Apr 2022 | InCoder | Zero-shot arbitrary, multi-hole code infilling |
| 28 Jul 2022 | OpenAI FIM study | PSM/SPM recipe; little loss to normal completion |
| 9 Jan 2023 | SantaCoder | 1.1B prototype; joint PSM + SPM |
| May 2023 | StarCoder | 15.5B, 8K context, FIM on 1T training tokens |
| 24 Aug 2023 | Code Llama | IDE-oriented infilling variants + long context |
| 25 Jan 2024 | DeepSeek-Coder | Project-level data, 16K windows, FIM insertion |
Result: autocomplete evolved from “append the next line” into insertion, refactoring, and multi-line next-edit prediction.
BASE
Predict likely continuation
SFT
Learn desired responses
RM
Model human preference
RLHF
Follow intent more reliably
Making the model bigger did not automatically make it better at following user intent. Post-training shaped the assistant interface.
No repository state, no shell, no permissions, no autonomous action loop.
Conversation added iteration. The human still moved every action and observation.
OpenAI function calling
Fine-tuned models learned to:
decide whether a function is needed;
choose the function;
emit structured JSON arguments.
Proposal, not execution: application code still validates, authorizes, runs, and returns the result.
{
"name": "run_tests",
"description": "Run a test target",
"parameters": {
"type": "object",
"properties": {
"target": { "type": "string" }
},
"required": ["target"]
}
}01
Host sends schemas
02
Model emits call + args
03
Host validates policy
04
Tool returns result
05
Result becomes next message
The security boundary lives in the host: schema validation · permissions · sandbox · timeouts · logging · stop conditions
Model Context Protocol
MCP does not train the model. It standardizes discovery and transport between the host and external capabilities.
Model-facing contract
Schema in request; model emits tool name + arguments.
Integration-facing protocol
Client discovers servers and transports calls, results, and resources.
LLM · native tool-call format
⇅ name + JSON / tool result
Agent host · state · policy · approvals
⇅ bridge
MCP client
⇅ JSON-RPC · tools/list · tools/call
MCP server
⇅ integration
Git · files · databases · APIs
The host bridges the two layers. Neither JSON Schema nor MCP executes anything by itself.
| Stage | Signal | What the model learns |
|---|---|---|
| Instruction SFT | Expert demonstrations | Answer this kind of request |
| Tool-call SFT | Calls + valid arguments | When / which tool / what JSON |
| RLHF | Human preference rankings | Be helpful, safe, cooperative |
| RLVR / execution RL | Tests, compilers, environment outcomes | Prefer actions that actually work |
Coding is unusually suitable for verification: programs can be parsed, compiled, run, and tested.
Not by itself.
agentic behavior = model policy + tools + stateful harness + environment feedback + permissions + stopping rules
Improves instruction-following and interaction quality.
Improves structured action selection and arguments.
Rewards trajectories and outputs that survive verification.
The harness supplies the loop in which these behaviors matter.
Task + repository snapshot
↓
Agent rollout · read · edit · run
⇅ observations · diffs
Sandboxed repository
↓ patch + hidden tests
Verifier → scalar reward
↓ optimize
Update policy θ ↺ next rollout
Parse · typecheck · lint · unit tests · integration tests
Proxy, not truth: passing tests can still miss security, maintainability, performance, or the actual requirement.
At training time, reward updates weights. At inference time, test output becomes context for the next attempt.
One user turn can contain many model/tool subturns. The model proposes; the harness authorizes and executes; each observation becomes the next message.
The arc in one line
patterns · instructions · action policy
state · tools · permissions · loop
tests · diffs · logs · human judgment
The biggest leap was closing the loop between generation and evidence.