Good News for Otakus: Don’t Be Afraid of Having Too Much Lego, a Smart Sorting Machine Can Help You Sort Them

Lego is now almost synonymous with high-quality toys. The most famous product of the brand is Lego bricks. Its rich shapes and diverse ways of playing are loved by both adults and children. However, it is also a big problem to classify and store these complex and diverse parts. A software engineer built a Lego sorting machine with the help of AI technology, which brought good news to Lego enthusiasts.

If you haven’t played with LEGO, you may have heard of it. The LEGO Group was founded in Denmark in 1932, and soon thereafter,"LEGO"This trademark has become synonymous with high-quality toys.

LEGO's most famous building blocks were first released in 1949. 70 years of history.

How difficult is it to sort Lego?

Lego bricks are very attractive to children due to their complexity, diversity and large imagination space. They have fascinated many players and even become teaching aids in many educational institutions. However, the rich variety of parts also makes friends who love Lego often fall into the trouble of classification and induction. At present, there are a total of Lego bricks. 119 series, 111 colors, and each series containsHundreds or even thousands of parts.

There was a Lego enthusiast who bought 2 tons of Lego bricks in an auction on eBay. He felt happy at the time, but he was dumbfounded by such a large amount of Lego. "It would take several lifetimes to sort it all out," he wrote in his blog.

Many players have also done some research on the classification and organization of Lego. Daniel West, an Australian software engineer who is interested in computer vision, handed this "head-scratching" problem to AI.

The world's first universal AI Lego sorting machine

For humans, just by looking at a building block, we can easily identify its shape, color and other features. Even a one or two-year-old child can distinguish the difference between a 2×1 red building block, a 4×1 green building block and those small chain building blocks after a few attempts.

But what if we give the task of classification and induction to machines? It seems that it is not that simple.

As early as 2011, a LEGO enthusiast in Japan akiyuky, invented a Lego sorting machine to solve this problem. However, his database had fewer images and the sorting speed was slower.

Inspired by akiyuky, who also loves Lego and machinesDaniel West, Australian software engineer  I want to design a smarter and faster Lego sorting machine.

So he spent two years usingOver 10,000 LEGO bricks, built a universal Lego brick sorting machine, this sorting machineWith the help of computer vision algorithms, any Lego parts can be sorted outThe device also has 6 Lego motors and 9 servo motors to power the conveyor belts and agitators that move the Lego parts.

West said the machine couldSort 2927 LEGO bricks into 18 different bins,andEvery 2 secondsYou can share a building block. This efficiency is pretty good, right?

While this isn’t the world’s first Lego sorting machine, West calls it theThe world's first universal LEGO sorting machine, “because it uses state-of-the-art artificial intelligence technology to be able to recognize and classify any Lego part that has ever been produced.” West also plans to release the code in the future.

Classification work, the process is divided into three steps

How does this capable device work? Generally speaking, it doesn't seem complicated and consists of three steps.

Step 1: Add the blocks.

A large pile of Legos is poured into a chute on top of the machine, which then places the bricks on a vibrating belt, where they are shaken apart into individual pieces.

Step 2: Take photos and classify them.

The blocks are transported by a Raspberry Pi computer. The Raspberry Pi computer contains a camera that takes a series of photos as the blocks pass by, which are then fed into the CNN model. ResNet-50  Classify on.

Step 3: Classify and put into boxes

After the classification results are fed back to the device, different small gates on the conveyor belt will guide the building blocks into the corresponding boxes. 

The technical implementation has twists and turns

Since there are hundreds of types of LEGO parts, with various colors and different shapes when viewed from different angles,Collecting the right training data set is the hardest part of the whole job.

West said he initially tried to simulate Legos. LDraw Part Library(an open source program that allows enthusiasts to virtually build Lego) 3D Model, and rendered it using the free animation software Blender.

He can then simulate the 3D model bricks at different rotation angles and colors. These individual images are collected into a file containingOver 25 million imagesof synthetic data sets.

But to West's dismay,His AI Lego sorter was trained with fake images but couldn't recognize real parts“I had almost given up on the project completely at that point,” he said. “I had spent months implementing a complex method for converting synthetic images into real ones, but with little success.”

He then turned to another commonly used technique to help reduce the gap between simulation and reality:Domain Randomization.

Domain randomization can train the model to recognize more data variations.The system not only learned different rotations and colors of Lego bricks, but also learned to take into account various lighting effects, textures and noise..

To further improve the performance of the Lego classifier, West also put together a smaller dataset containing snapshots of real Lego parts. “The real dataset only contained parts from my personal collection, which ended up being 544 different types of parts,” he said. These were the more common types in the Lego series, while the synthetic dataset was some of the less common types.

By training on these two datasets, the camera on West’s Lego sorter should, in theory, be able to recognize parts it has never actually seen. However, the actual performance of the model on the different datasets was different:Real DatasetWhen the parts are 93%, and identificationSynthetic DatasetThe accuracy of the parts in 74%.

It is not easy to collect training data sets, and West also wrote an article (Original link: http://985.so/f5G8), which details how he created 100,000 labeled Lego training images. The article mentions that he mainly used sample generation methods, simplified processes, and AI-assisted annotation to complete the task of labeling massive images.

Dreams become reality with the help of AI

West loves Lego, and his dream is also related to Lego. He said,He will finally realize his lifelong dream: building a real, working Lego sorting machine, while gaining real-world experience designing and deploying artificial intelligence systems.

Interest is the best teacher and the biggest driving force. At the same time, the development of technology has also become a booster for interests and dreams, allowing dreams of all sizes to be realized more quickly and perfectly. Nowadays, AI can help you sort Lego, so what else can't it do?

-- over--