Context distillation
Training a model to behave, without a special instruction, as it would with that instruction. For example, the model is fine-tuned on the safer answers it gives when told "You are a safe and responsible assistant".
Introduced by
HHH assistant, 2021
Used by
- Llama 2, 2023