Man Buys Two Metric Tons of LEGO Bricks; Sorts Them Via Machine Learning

iStock // Ekaterina Minaeva
iStock // Ekaterina Minaeva

Jacques Mattheij made a small, but awesome, mistake. He went on eBay one evening and bid on a bunch of bulk LEGO brick auctions, then went to sleep. Upon waking, he discovered that he was the high bidder on many, and was now the proud owner of two tons of LEGO bricks. (This is about 4400 pounds.) He wrote, "[L]esson 1: if you win almost all bids you are bidding too high."

Mattheij had noticed that bulk, unsorted bricks sell for something like €10/kilogram, whereas sets are roughly €40/kg and rare parts go for up to €100/kg. Much of the value of the bricks is in their sorting. If he could reduce the entropy of these bins of unsorted bricks, he could make a tidy profit. While many people do this work by hand, the problem is enormous—just the kind of challenge for a computer. Mattheij writes:

There are 38000+ shapes and there are 100+ possible shades of color (you can roughly tell how old someone is by asking them what lego colors they remember from their youth).

In the following months, Mattheij built a proof-of-concept sorting system using, of course, LEGO. He broke the problem down into a series of sub-problems (including "feeding LEGO reliably from a hopper is surprisingly hard," one of those facts of nature that will stymie even the best system design). After tinkering with the prototype at length, he expanded the system to a surprisingly complex system of conveyer belts (powered by a home treadmill), various pieces of cabinetry, and "copious quantities of crazy glue."

Here's a video showing the current system running at low speed:

The key part of the system was running the bricks past a camera paired with a computer running a neural net-based image classifier. That allows the computer (when sufficiently trained on brick images) to recognize bricks and thus categorize them by color, shape, or other parameters. Remember that as bricks pass by, they can be in any orientation, can be dirty, can even be stuck to other pieces. So having a flexible software system is key to recognizing—in a fraction of a second—what a given brick is, in order to sort it out. When a match is found, a jet of compressed air pops the piece off the conveyer belt and into a waiting bin.

After much experimentation, Mattheij rewrote the software (several times in fact) to accomplish a variety of basic tasks. At its core, the system takes images from a webcam and feeds them to a neural network to do the classification. Of course, the neural net needs to be "trained" by showing it lots of images, and telling it what those images represent. Mattheij's breakthrough was allowing the machine to effectively train itself, with guidance: Running pieces through allows the system to take its own photos, make a guess, and build on that guess. As long as Mattheij corrects the incorrect guesses, he ends up with a decent (and self-reinforcing) corpus of training data. As the machine continues running, it can rack up more training, allowing it to recognize a broad variety of pieces on the fly.

Here's another video, focusing on how the pieces move on conveyer belts (running at slow speed so puny humans can follow). You can also see the air jets in action:

In an email interview, Mattheij told Mental Floss that the system currently sorts LEGO bricks into more than 50 categories. It can also be run in a color-sorting mode to bin the parts across 12 color groups. (Thus at present you'd likely do a two-pass sort on the bricks: once for shape, then a separate pass for color.) He continues to refine the system, with a focus on making its recognition abilities faster. At some point down the line, he plans to make the software portion open source. You're on your own as far as building conveyer belts, bins, and so forth.

Check out Mattheij's writeup in two parts for more information. It starts with an overview of the story, followed up with a deep dive on the software. He's also tweeting about the project (among other things). And if you look around a bit, you'll find bulk LEGO brick auctions online—it's definitely a thing!

Surprise! Microsoft Admits That People Are Listening In on Your Cortana and Skype Conversations

mabe123/iStock via Getty Images
mabe123/iStock via Getty Images

In a turn of events that, at this point, probably shocks no one, Microsoft has confessed that human contractors have been listening in on some people's private conversations via Skype and Cortana.

Vice first broke the news on the privacy breach earlier this month, after a contractor passed along documents, screenshots, and actual audio files of some conversations. “The fact that I can even share some of this with you shows how lax things are in terms of protecting user data,” the contractor, whose name was withheld (for obvious reasons), told Motherboard. Unlike Apple’s recent Siri snafu, these conversations didn’t include potential criminal activity, but they did catch intimate exchanges about weight loss, love, and relationship problems.

Also unlike Apple: Microsoft is not suspending its practices. Instead, the tech monolith has updated its privacy policy to clarify that humans might, in fact, be eavesdropping on you.

“We realized, based on questions raised recently, that we could do a better job of clarifying that humans sometimes review this content,” a Microsoft representative told Vice. Before, the Skype website had mentioned that your content could be analyzed in order to improve the technology, but it never explicitly stated that humans would be listening to it.

Microsoft only records Skype conversations that use its translation features, in order to “help the translation and speech technology learn and grow,” according to the Skype FAQ section. If you’re not using translation features, your sweet nothings are reportedly as private as you want them to be. The updated FAQ section also now states that “Microsoft employees and vendors” may be transcribing the translated audio, and the procedures are “designed to protect users’ privacy, including taking steps to de-identify data, requiring non-disclosure agreements with vendors and their employees,” and more.

But Cortana’s data gathering isn’t limited to translation. According to its support page, Microsoft can collect your voice data literally any time you “use your voice to say something to Cortana or invoke skills.” If that worries you, we recommend spending some time adjusting the settings on your Microsoft Privacy Dashboard.

[h/t Vice]

Amazon Is Making It Easier for Sellers to Donate Returned or Unsold Items Instead of Trashing Them

ronstik/iStock via Getty Images
ronstik/iStock via Getty Images

After mailing back an unwanted Amazon order or bringing it to a drop-off location, you may assume your return will find its way to a good home. But not every item returned to Amazon is re-listed on the website. Many third-party sellers working through Amazon simply toss returned products in the trash along with any surplus goods they can't sell. That's about to change: As CNBC reports, the retailer will make its new Fulfillment by Amazon (FBA) Donations program the default option for independent shops based in the U.S. and the UK.

Starting in September 2019, sellers that have unwanted returns or unsold items sitting in an Amazon warehouse can depend on the company to donate them to a good cause. In the U.S., products that are eligible for donation will be sent to various nonprofits through a charity group called Good360, and in the UK, groups including Newlife, Salvation Army, and Barnardo will distribute the goods.

“We know getting products into the hands of those who need them transforms lives and strengthens local communities,” Alice Shobe, director of Amazon in the Community, said in a statement. “We are delighted to extend this program to sellers who use our fulfillment services.”

Amazon charges independent sellers 50 cents to ship unsold items back to them and just 15 cents to throw them away, meaning that donating or trying to resell returned items wasn't always cost-effective for businesses. Amazon reportedly wasted 293,000 products over a nine-month period in just one French distribution center alone.

The company now aims to incentivize sellers to donate their unsold and returned inventory by making it the cheaper option, according to sellers who spoke to CNBC. As part of this program, Amazon will also manage the logistics and work with charities in an effort to "streamline the donation process for independent sellers."

The Fulfillment by Amazon Donations program will be the default for U.S. and UK sellers starting on September 1, but stores will still have the opportunity to opt out if they wish.

[h/t CNBC]

SECTIONS

arrow
LIVE SMARTER