CLOSE
iStock // Ekaterina Minaeva
iStock // Ekaterina Minaeva

Man Buys Two Metric Tons of LEGO Bricks; Sorts Them Via Machine Learning

iStock // Ekaterina Minaeva
iStock // Ekaterina Minaeva

Jacques Mattheij made a small, but awesome, mistake. He went on eBay one evening and bid on a bunch of bulk LEGO brick auctions, then went to sleep. Upon waking, he discovered that he was the high bidder on many, and was now the proud owner of two tons of LEGO bricks. (This is about 4400 pounds.) He wrote, "[L]esson 1: if you win almost all bids you are bidding too high."

Mattheij had noticed that bulk, unsorted bricks sell for something like €10/kilogram, whereas sets are roughly €40/kg and rare parts go for up to €100/kg. Much of the value of the bricks is in their sorting. If he could reduce the entropy of these bins of unsorted bricks, he could make a tidy profit. While many people do this work by hand, the problem is enormous—just the kind of challenge for a computer. Mattheij writes:

There are 38000+ shapes and there are 100+ possible shades of color (you can roughly tell how old someone is by asking them what lego colors they remember from their youth).

In the following months, Mattheij built a proof-of-concept sorting system using, of course, LEGO. He broke the problem down into a series of sub-problems (including "feeding LEGO reliably from a hopper is surprisingly hard," one of those facts of nature that will stymie even the best system design). After tinkering with the prototype at length, he expanded the system to a surprisingly complex system of conveyer belts (powered by a home treadmill), various pieces of cabinetry, and "copious quantities of crazy glue."

Here's a video showing the current system running at low speed:

The key part of the system was running the bricks past a camera paired with a computer running a neural net-based image classifier. That allows the computer (when sufficiently trained on brick images) to recognize bricks and thus categorize them by color, shape, or other parameters. Remember that as bricks pass by, they can be in any orientation, can be dirty, can even be stuck to other pieces. So having a flexible software system is key to recognizing—in a fraction of a second—what a given brick is, in order to sort it out. When a match is found, a jet of compressed air pops the piece off the conveyer belt and into a waiting bin.

After much experimentation, Mattheij rewrote the software (several times in fact) to accomplish a variety of basic tasks. At its core, the system takes images from a webcam and feeds them to a neural network to do the classification. Of course, the neural net needs to be "trained" by showing it lots of images, and telling it what those images represent. Mattheij's breakthrough was allowing the machine to effectively train itself, with guidance: Running pieces through allows the system to take its own photos, make a guess, and build on that guess. As long as Mattheij corrects the incorrect guesses, he ends up with a decent (and self-reinforcing) corpus of training data. As the machine continues running, it can rack up more training, allowing it to recognize a broad variety of pieces on the fly.

Here's another video, focusing on how the pieces move on conveyer belts (running at slow speed so puny humans can follow). You can also see the air jets in action:

In an email interview, Mattheij told Mental Floss that the system currently sorts LEGO bricks into more than 50 categories. It can also be run in a color-sorting mode to bin the parts across 12 color groups. (Thus at present you'd likely do a two-pass sort on the bricks: once for shape, then a separate pass for color.) He continues to refine the system, with a focus on making its recognition abilities faster. At some point down the line, he plans to make the software portion open source. You're on your own as far as building conveyer belts, bins, and so forth.

Check out Mattheij's writeup in two parts for more information. It starts with an overview of the story, followed up with a deep dive on the software. He's also tweeting about the project (among other things). And if you look around a bit, you'll find bulk LEGO brick auctions online—it's definitely a thing!

nextArticle.image_alt|e
iStock
arrow
technology
Trash Talk: 7 Ways to Recycle Your Tech Gadgets
iStock
iStock

Our tech gadgets’ lifespans are short. New smartphone models come out at least once a year, and it’s easy to want the latest and greatest computer, gaming console, or 4K TV—without considering what happens to our used devices.

The Environmental Protection Agency estimates that Americans generated nearly 3.4 million tons of consumer electronics waste in 2014 [PDF] and that only around 40 percent of that waste was recycled—the rest went to landfills or incinerators. The U.S. is also a top destination for e-waste from other countries [PDF]—and in turn, we export much of our e-waste to places like China and India. However, more manufacturers and recycling companies are now taking steps to ensure the e-waste they collect is handled responsibly.

To do your part, don’t simply dump the old model in the trash—use one of these methods to resell or recycle.

1. DROP IT OFF AT A RETAIL STORE.

man returns electronics at a store
iStock

Home and office suppliers often have in-store recycling programs that offer cash back or trade-in options. For instance, Best Buy accepts everything from appliances to car GPS units. (Not all products are accepted, though, so check before you go.) Staples offers trades on phones and tablets and will also take most other electronics, from fax machines to shredders, for recycling. Take your rechargeable batteries and cell phones to Lowes.

2. HOST AN ELECTRONICS DRIVE.

pile of electronics
iStock

Work with your employer or a group to put on a tech recycling event. It’s easy enough for people to bring in old TVs, audio equipment, and laptops. Then, you can collect all the items over the course of a few days or weeks and recycle them in bulk with a local organization. A good place to start: the EPA's list of certified electronics recyclers.

3. TRADE IT IN.

Several sites allow you to swap used electronics for cash. These companies refurbish, resell, or recycle old devices. To get started, enter your device’s details to receive a quote, then ship it in using a prepaid label and get money via PayPal, check, or gift card. Amazon’s Trade-In service accepts phones, tablets, speakers, and gaming equipment, provided the items are in good condition; Gazelle takes smartphones, tablets, and Apple computers; and NextWorth buys back tablets, smartphones, and wearables.

4. DOWNLOAD LETGO OR GONE.

Of course, there’s an app for that. Letgo is a free mobile marketplace for a variety of goods, including electronics, and all you have to do is take a picture of your old computer or TV, upload it, and then communicate with potential buyers within the app. Gone deals specifically with used tech, and the app does all the work, including pricing and generating shipping labels, for you—which means you don’t have to limit your sale options to your local area or meet strangers face to face.

5. SELL IT ON CRAIGSLIST, FACEBOOK, OR EBAY.

laptop showing ebay website
iStock

Go old-school: List your old electronics on Craigslist, Facebook’s Marketplace, eBay, or your local classifieds. It’s not uncommon to find people who buy and refurbish gadgets for resale or to repurpose parts—or parents looking for a cheap used iPhone or laptop for their child. This way, you can negotiate the sale price and get cash on the spot. While there’s no guarantee that the buyer will dispose of your old phone or tablet responsibly once they’re done with it, selling does give the device a second (or third) life and hopefully will replace the purchase of a new product.

6. DONATE IT.

pile of electronics
iStock

While a new phone or gaming console seems like a no-brainer to some, there are many who can’t afford to purchase tech gadgets at all—new or used. If you aren’t able to find a recycling or donation center locally, consider one of these mail-in donation options:

Computers and peripherals: Goodwill has a partnership with Dell called Dell Reconnect. The program takes old computers—and anything you can connect to them, from keyboards to scanners—and refurbishes them for resale. Any parts that can’t be fixed are recycled. The National Cristina Foundation connects consumers to local nonprofits that need computers, and the World Computer Exchange accepts most computer equipment through a local chapter or by mail.

Cell phones: Several organizations collect old cell phones to refurbish, re-sell, and recycle in bulk and then use the funds to support their programming. The National Coalition for Domestic Violence will provide a prepaid shipping label for your phone, laptop, or gaming system, as will Lifecell —the latter purchases Lifestraws for those who lack access to clean water. Cell Phones for Soldiers takes gently used phones to provide communication services to troops and veterans.

Gaming gear: AbleGamers, which provides accessible gaming technology to people with disabilities, accepts donations of used consoles and games via mail. Gamers Outreach and Charity Nerds will take your donated gaming equipment to children who are hospitalized.

7. SEND IT BACK TO THE MANUFACTURER.

packages
iStock

Many companies, including Apple, Dell, HP, and IBM, offer branded recycling programs, which means they’ll take back used devices, recycle them responsibly, and often give you a gift card or a credit towards the purchase of a new device. Take your Apple products to your nearest store or create a prepaid shipping label online. IBM facilitates shipping of its branded products to preferred recyclers in certain states. Because Dell’s recycling program is in partnership with Goodwill, their take-backs aren’t limited to branded devices.

nextArticle.image_alt|e
iStock
arrow
Live Smarter
The Google Docs Audio Hack You Might Not Know About
iStock
iStock

To the uninitiated, Google Docs may take some warming up to. But although it may seem like any other word processor, Docs offers its fair share of nifty features that can make your life a whole lot easier. The only problem is that few people seem to know about them.

The Voice Typing function is one such example. As Quartz discovered, this tool can be used to drastically cut down on the time it takes to transcribe an interview or audio recording—a feature that professionals from many fields could benefit from. Voice Typing might also be useful to those who prefer to dictate what they want to write, as well as those with impairments that prevent them from typing.

Whatever the case may be, it's extremely easy to use. Just open a blank document, click on "tools" at the top, and then select "voice typing." A microphone icon will pop up, allowing you to choose your language. After you've done that, simply click the icon when you're ready to start speaking!

Unfortunately, it's unable to pick up an audio recording played through speakers, so you'll need to grab a pair of headphones, plug them into your phone or voice recorder, and dictate what's said as you listen along. Still, this eliminates the hassle of having to pause and rewind in order to let your fingers catch up to the audio—unless you're the champion of a speed typing contest, in which case you probably don't need this tutorial.

According to Quartz, the transcription is "shockingly" accurate, even getting the spelling of last names right. For a how-to guide on the Voice Typing tool, check out Quartz's video below.

[h/t Quartz]

SECTIONS

arrow
LIVE SMARTER
More from mental floss studios