Man Buys Two Metric Tons of LEGO Bricks; Sorts Them Via Machine Learning

iStock // Ekaterina Minaeva
iStock // Ekaterina Minaeva

Jacques Mattheij made a small, but awesome, mistake. He went on eBay one evening and bid on a bunch of bulk LEGO brick auctions, then went to sleep. Upon waking, he discovered that he was the high bidder on many, and was now the proud owner of two tons of LEGO bricks. (This is about 4400 pounds.) He wrote, "[L]esson 1: if you win almost all bids you are bidding too high."

Mattheij had noticed that bulk, unsorted bricks sell for something like €10/kilogram, whereas sets are roughly €40/kg and rare parts go for up to €100/kg. Much of the value of the bricks is in their sorting. If he could reduce the entropy of these bins of unsorted bricks, he could make a tidy profit. While many people do this work by hand, the problem is enormous—just the kind of challenge for a computer. Mattheij writes:

There are 38000+ shapes and there are 100+ possible shades of color (you can roughly tell how old someone is by asking them what lego colors they remember from their youth).

In the following months, Mattheij built a proof-of-concept sorting system using, of course, LEGO. He broke the problem down into a series of sub-problems (including "feeding LEGO reliably from a hopper is surprisingly hard," one of those facts of nature that will stymie even the best system design). After tinkering with the prototype at length, he expanded the system to a surprisingly complex system of conveyer belts (powered by a home treadmill), various pieces of cabinetry, and "copious quantities of crazy glue."

Here's a video showing the current system running at low speed:

The key part of the system was running the bricks past a camera paired with a computer running a neural net-based image classifier. That allows the computer (when sufficiently trained on brick images) to recognize bricks and thus categorize them by color, shape, or other parameters. Remember that as bricks pass by, they can be in any orientation, can be dirty, can even be stuck to other pieces. So having a flexible software system is key to recognizing—in a fraction of a second—what a given brick is, in order to sort it out. When a match is found, a jet of compressed air pops the piece off the conveyer belt and into a waiting bin.

After much experimentation, Mattheij rewrote the software (several times in fact) to accomplish a variety of basic tasks. At its core, the system takes images from a webcam and feeds them to a neural network to do the classification. Of course, the neural net needs to be "trained" by showing it lots of images, and telling it what those images represent. Mattheij's breakthrough was allowing the machine to effectively train itself, with guidance: Running pieces through allows the system to take its own photos, make a guess, and build on that guess. As long as Mattheij corrects the incorrect guesses, he ends up with a decent (and self-reinforcing) corpus of training data. As the machine continues running, it can rack up more training, allowing it to recognize a broad variety of pieces on the fly.

Here's another video, focusing on how the pieces move on conveyer belts (running at slow speed so puny humans can follow). You can also see the air jets in action:

In an email interview, Mattheij told Mental Floss that the system currently sorts LEGO bricks into more than 50 categories. It can also be run in a color-sorting mode to bin the parts across 12 color groups. (Thus at present you'd likely do a two-pass sort on the bricks: once for shape, then a separate pass for color.) He continues to refine the system, with a focus on making its recognition abilities faster. At some point down the line, he plans to make the software portion open source. You're on your own as far as building conveyer belts, bins, and so forth.

Check out Mattheij's writeup in two parts for more information. It starts with an overview of the story, followed up with a deep dive on the software. He's also tweeting about the project (among other things). And if you look around a bit, you'll find bulk LEGO brick auctions online—it's definitely a thing!

10 Facts About Silicon

iStock
iStock

How well do you know the periodic table? Our series The Elements explores the fundamental building blocks of the observable universe—and their relevance to your life—one by one.

Silicon is a metalloid: an element with properties not quite like a metal, nor exactly like a non-metal. If you have a cell phone in your pocket or dirt on your shoes, you’re carrying silicon. Learn more about this ever-present element.

1. IT'S JUST ABOUT EVERYWHERE.

It's the seventh most abundant element in the universe and even more prevalent in the Earth's crust, second only to oxygen as the most common element by weight. The layer under the crust—the mantle—is rich in silicon as well. With an atomic number of 14, it sits right below carbon on the periodic table.

2. SILICON ISN'T THE SAME THING AS SILICONE.

silicone breast implant on blue cloth
iStock

The word silicone might make you think of breast implants, but it's actually a general term for a group of synthetic substances made of alternating silicon and oxygen atoms, with carbon and hydrogen molecules bonded on the sides. By mixing up these side groups of molecules and creating links between chains, chemists can create silicones with all sorts of different properties. Yes, you can find silicones in breast implants, but also in car polish, the insulation around electric cables, and even in your hair conditioner, where they help to calm down frizz. We can also thank silicones for Silly Putty, which was invented during World War II, when scientists were trying to create an alternative to rubber—and instead came up with a new national favorite toy.

3. WE USED SILICON FOR THOUSANDS OF YEARS WITHOUT KNOWING ITS ELEMENTAL NATURE.

Silica is the main ingredient of glass, which humans have been making at least since the Egyptians fashioned beads from the material in 2500 BCE. In China, the Qin and Han dynasties used purple and blue pigments made of barium copper silicates for various decorations, including parts of the famous terra-cotta army.

It took many centuries before people realized the substance could be further separated into two different elements—oxygen and silicon. In the late 1700s, French chemist Antoine Lavoisier noticed that certain materials classified as “earth” substances (which were dry and cold) sometimes behaved like metals (hard, dense, and resistant to being stretched, among other qualities). Silica was one of them. Perhaps, Lavoisier mused, some of the earths were really molecules of oxygen and a yet-undiscovered, metal-like element.

At the time, chemists didn’t know how to remove the oxygen atoms, which form strong bonds with the silicon atoms. That changed in the 1820s, when a Swedish chemist named Jons Berzelius finally managed to obtain silicon in his lab by purifying it from a silicon-containing compound. (Which one, and how he did it, isn't clear.) Berzelius's breakthrough came too late for Lavoisier, who had died in 1794, to see his speculations be proven true.

4. SILICA IS THE MOST COMMON FORM OF SILICON.

Also known as silicon dioxide, this molecule is composed of one silicon atom and two oxygen atoms (SiO2). Most of what we call silicon is actually silica, found in both minerals and plants. Many plants create unique microscopic structures called phytoliths using silica they take up from the soil. Scientists aren't sure why: They might offer protection against microscopic harm or provide structural support, or maybe they're just a way for a plant to use up extra silica.

Phytoliths stick around long after a plant decays, which can illuminate the deep history of an area—whether it used to be a forest or grassland, for instance, or how people used the landscape. Dan Cabanes, a phytolith expert and anthropologist at Rutgers University, has used phytoliths to understand how Neanderthals made a home in a cave in northern Spain, creating a sleeping area with grass bedding they used repeatedly. And because phytoliths survive burning, “we can study how they made fire or what type of food they were consuming,” Cabanes tells Mental Floss.

The picture isn’t always perfect, though, because sometimes two different plants make phytoliths of the same shape—and some plants don’t make them at all.

5. IT'S A KEY COMPONENT OF SOME BEAUTIFUL STONES …

close-up of onyx
iStock

Gorgeous gemstones like amethyst, onyx, and agate are all made of silica. In each rock, the silica molecules are arranged in repeating 3D geometries called crystal structures. Different arrangements, as well as small impurities in the rock, give each gemstone its unique appearance.

6. … AND THE DAZZLING BEAUTY OF DIATOMS.

Triceratium polycystinorum diatom
Anatoly Mikhaltsov, Wikimedia Commons // CC BY-SA 4.0

Silica also forms the cell walls of diatoms, a type of algae found all over the world. Diatoms, which come in a mesmerizing variety of shapes, can live in both fresh and saltwater. When they die, their cell walls can accumulate into chalky deposits of "diatomaceous earth," which we use in all sorts of things, from cat litter to toothpaste.

7. SILICON IS VERY USEFUL IN TECH …

Silicon can act as a semiconductor—a material that neither conducts electricity perfectly nor insulates against it, but rather lies somewhere in between. This property is important in many parts of electronics, where you want some control over the flow of electricity. “What's beautiful about semiconductors is that you can tune their conductivity by adding impurities,” Eric Pop, a professor of electrical engineering at Stanford University, tells Mental Floss. Pure silicon is an insulator, but if you ‘dope’ it with tiny amounts of certain other elements, such as phosphorus or arsenic, it becomes better at conducting electricity.

Other materials, including germanium or gallium arsenide, are better semiconductors than silicon, but silicon is the most popular choice among electronics manufacturers (whose concentration south of San Francisco in the 1970s inspired the name "Silicon Valley"). It's cheap, it’s everywhere, and because it likes to oxidize so much, it can conveniently create its own insulating layer when exposed to air.

8. … BUT RESEARCHERS WANT TO FIND BETTER OPTIONS.

Engineers like Pop are looking for materials to replace silicon in our electronics to help keep up with the demand for faster computing. “Silicon is sort of like the Honda Civic of semiconductors,” Pop says. “It gets the job done, but it’s not very fast.” However, Pop thinks that even when pitted against superior materials, silicon won’t completely disappear, thanks to its low cost.

9. SILICON HOLDS UP MANY OF OUR BUILDINGS.

brick building against blue sky
iStock

Many common building materials are based on silicon-containing substances. Clay minerals, which contain silicon, are used to make bricks, as well as Portland cement, which is then used as the binding agent in concrete.

10. AMERICANS LEFT A BIT OF SILICON ON THE MOON.

When Buzz Aldrin and Neil Armstrong became the first humans to walk on the Moon, in 1969, they left a few things on its surface besides their footprints. One was a small silicon disc, inscribed with messages from the leaders of 73 countries, from Afghanistan to Zambia. The disc is housed inside a protective aluminum case and is stashed in a small bag along with a few other items. Silicon was elected official message-bearer because it could endure the huge range of temperatures on the Moon. The disc nearly didn’t make it, though: Aldrin had forgotten all about the bag, tucked into a pocket of his space suit sleeve, and he was already on the ladder to the spacecraft when Armstrong reminded him about it. Aldrin tossed the pouch onto the Moon.

'Museum in a Box' Brings Interactive Museum Collections to Classrooms

Museum in a Box, Vimeo
Museum in a Box, Vimeo

Museums hold a wealth of information, but they're not always accessible to everyone. Visiting a museum takes time, money, and opportunity. Even if you make it to a museum, most collections are so vast that only a tiny fraction of the collection is available to view. But a new initiative aims to make it easier for museums to reach schools and communities outside the exhibition gallery.

Museum in a Box is essentially a mini interactive exhibit that can be sent out to schools and other organizations that serve kids. Each box comes with a Raspberry Pi computer, a speaker, an amplifier, and a near-field communication (NFC) reader—like the kind that lets you pay with your phone at retail checkouts. Museums and cultural institutions can then add their own objects for kids to learn about using that technology, whether it's 3D-printed versions of statues from the museum's collections, postcards, puzzles, or anything else curators think kids might want to see. Students can place these objects on top of the box, triggering the NFC reader to start playing a recording related to the object.

How it works from Museum in a Box on Vimeo.

Much of the content museums might want to include in a box might already be digitized on the institution's website or in its collections, but the box provides a tangible, curated way to present it. For a prototype designed for the Smithsonian Institution, for instance, the Museum in a Box team created Frogs in a Box, combining photographs of North American frogs from the Smithsonian collection with a narrated collection of frog-sound field recordings produced by Smithsonian Folkways in 1958. When you tap a postcard featuring a species of frog on the top of the box, it plays audio from Sounds of North American Frogs, including the commentary from a herpetologist and recordings of frog calls.

Though much of the work is still in the prototype phase, the Museum in a Box team has been commissioned to create boxes for institutions like the Barnsley Museums in the UK, the University of Melbourne, and the Swedish National Heritage Board.

A Museum in a Box with red flowers designed for the Jewish Museum London
A Museum in a Box designed for the Jewish Museum London
Museum in a Box

The technology isn't necessarily limited to museum collections, though. Another prototype Museum in a Box, this one aimed at language learners, allows users to place flash cards featuring different words on top of the box to trigger recordings of native speakers pronouncing those words. The team is also developing a pilot for a DIY box that would let kids come up with their own themes and content for a box. You can currently support that project on Crowdfunder.

SECTIONS

arrow
LIVE SMARTER
More from mental floss studios