The “Draw an SVG of a pelican riding a bicycle” has been very popular to gauge the ability of large language models. Although it is not really a serious benchmark, its popularity to give a ballpark estimate of a model’s intelligence has been such that some have speculated that models will start training on it…

I decided to check how the small open weight models stack up against the free versions of frontier models on a similar prompt: “Please generate an SVG of an octopus trying to solve a Rubik’s Cube”

Here are the results