LegoGPT: Generating Physically Stable and Buildable LEGO Designs from Text LegoGPT is a groundbreaking tool that converts user-generated text prompts into physically stable and buildable LEGO structures. This innovative approach combines the power of large language models with advanced physics awareness to ensure that the designs are not only creative but also feasible for construction. To develop LegoGPT, researchers created a large-scale dataset called StableText2Lego, which includes over 47,000 LEGO structures derived from more than 28,000 unique 3D objects, each paired with detailed captions. This dataset serves as the foundation for training an autoregressive large language model. The model predicts the sequence of LEGO bricks to add by using next-token prediction, ensuring that each step in the design process aligns with physical stability and buildability. One of the key challenges in generating buildable LEGO designs is ensuring that they remain stable throughout the construction process. To address this, the team implemented an efficient validity check and a physics-aware rollback mechanism. During the inference phase, the model prunes out infeasible predictions based on physics laws and assembly constraints. This ensures that the final design is both structurally sound and easy to assemble. Experiments with LegoGPT have shown promising results. The generated LEGO structures are not only stable but also diverse and aesthetically pleasing. They closely align with the provided text prompts, making it possible to create intricate and accurate designs from simple descriptions. For instance, a user could input "a red tower with a blue roof," and LegoGPT would generate a buildable structure that matches this description. In addition to creating structurally sound models, LegoGPT also incorporates a text-based LEGO texturing method. This feature allows the model to generate designs with specific colors and textures, enhancing the visual appeal and customization options. The ability to texture designs adds an extra layer of creativity and personalization, making LegoGPT a versatile tool for both casual users and professional designers. To validate the practicality of the generated designs, the research team conducted tests where humans and robotic arms assembled the predicted LEGO structures. Both manual and automated construction methods demonstrated that the designs produced by LegoGPT are robust and feasible. This dual verification process underscores the reliability and utility of the tool. The researchers have made their dataset and models publicly available, releasing the StableText2Lego dataset and the corresponding code. This resource is invaluable for the community, providing a foundation for further research and development in generating buildable and stable designs from text inputs. The release of these materials encourages collaboration and innovation, helping to advance the field of generative design in construction and manufacturing. In summary, LegoGPT represents a significant advancement in the intersection of language models and physical construction. By generating stable and buildable LEGO designs from text, it opens up new possibilities for creative and functional modeling. The inclusion of texturing capabilities and practical verification through human and robotic assembly further enhances its value. With the release of the dataset and code, LegoGPT invites the broader scientific and engineering communities to explore and expand upon this exciting technology.