Training a coding model to paint watercolours with TRL and OpenEnv
Article image or reusable cover for Hugging Face
Hugging Face has published a guide on training an AI model to paint watercolors by having it write JavaScript code.
A video from August showing a language model creating watercolor paintings using the p5.brush library received over 1.5 million views. This article reproduces the experiment using open tools (TRL and OpenEnv), where the model is rewarded based on aesthetic taste rather than right/wrong answers. The reward comes from two sources: HPSv3, a model trained on human image preferences, and a comparative judge that evaluates paintings against a curated dataset. By adjusting the weighting between these two reward models, the researchers were able to steer the model's painting style.
The real question of the project is whether you can do RL over taste.
Vibekollen prepared this summary with AI from the original publication. The content belongs to Hugging Face.
