Apollo Research details practical refinements to Synthetic Document Finetuning

apollo research published practical guidelines for implementing synthetic document finetuning (sdf), a method for effectively implanting beliefs into models by training them on llm-generated documents where a target fact is true. while following the pipeline established by slocum et al. (2025), the team identified specific failure modes in large-scale generation, such as mode collapse where a single name appeared 439,000 times, and resolved it using 'anti-universe' negative prompts and random name seeding. they also advocate for model-graded quality filtering to enforce hard constraints and iterative testing on small document batches to refine the 'universe' description before scaling. qualitative chat-based evaluation remains the most reliable way to confirm that the model has internalized the new beliefs and to understand how they are represented.