Preference Model raises $16M seed round led by Andreessen Horowitz to build reinforcement learning environments for AI alignment.

Preference Model secured $16 million in seed funding led by Andreessen Horowitz, with participation from SignalFire, South Park Commons, Scale Angel Group, and notable researchers including Fei-Fei Li and Ian Goodfellow. The company develops reinforcement learning environments designed to train more capable and better-aligned AI systems while preventing reward hacking. Alongside the funding, Preference Model open-sourced Karotte, a framework that has undergone over one million evaluation runs to provide secure defaults and tools preventing agents from exploiting training infrastructure. The capital will support continued development of RL infrastructure to help AI labs identify model weaknesses and test environments against agents attempting to break them.