SHAPES, the full name of Swarm Heuristics based Adaptive and Penalized Estimation of Splines, is a synthetic image dataset that aims to provide a benchmark system for understanding the spatial and logical relationships between multiple objects.

The dataset consists of complex questions about arrangements of colored shapes. The questions are centered around a combination of concepts and relationships, e.g., is there a red shape on top of a circle? Questions contain two to four attributes, object types, or relationships. The dataset contains 244 questions and 15,616 images, all with a yes and no answer (and corresponding supporting images). This eliminates the risk of learning bias.

Each image is a 30×30 RGB image with a 3×3 grid of objects. The properties of each object include shape (circle, square, triangle), color (red, green, blue), and size (small, large).