Alignment & Direct Instruction (SFT & Post-Training)
The base model learns broad capability. Post-training decides whether that capability behaves like a useful colleague, a tool user, or a liability.
As base capability becomes more available, the differentiator shifts toward preference data, reinforcement signals, tool-use traces, safety shaping, and product-specific behavior.
Pretraining teaches the world, post-training teaches the job
Pretraining builds broad statistical competence. Post-training narrows that competence toward instruction following, refusal behavior, tool use, tone, reliability, and domain performance.
The user experiences the second layer more directly than the first.
SFT: Supervised Fine-Tuning
SFT is the first phase of post-training. The lab collects thousands of high-quality examples of ideal dialogues (prompts and assistant answers) and runs standard training on them.
SFT does not teach the model new facts; it teaches the model formatting, tone, and conversation structure. It aligns the raw simulator to act like an assistant.
Dialogue wrappers and ChatML
Text on the web is raw and continuous. SFT injects specific tokens (like `<|im_start|>` and `<|im_end|>`) to delineate roles (System, User, Assistant). This allows the application to cleanly steer the model's persona and maintain secure chat boundaries.
The limits of imitation
SFT relies on mimicking examples. It struggles to teach the model how to recover when it makes a mistake, how to verify its own logic, or how to refuse unsafe inputs robustly. To build reliable reasoning, the model needs to learn from outcomes, not just examples.