This Christmas, you might find yourself elbow-deep in frosting and candy canes, trying to construct a gingerbread house that doesn’t collapse. But it turns out that creating a gingerbread house isn’t just a Christmas construction project—it’s a ritual with sometimes surprising connections to royalty, brutal fairy tales, and global trade.

Although versions of gingerbread date to ancient Egypt and Greece, the gingerbread we eat today has its roots in the Middle Ages, when cakes became all the rage in Europe as an increasingly global world opened up to new spices and ingredients. First there was fruitcake. The once-hot, now played-out treat came into favor after medieval cooks finally got access to dried fruit from Spain and Portugal thanks to increased trade in the 13th century.

That led to a vogue in cakes and breads, which spread as better construction made having an oven in your house less terrifying. Trade with the East also made the ingredients in gingerbread available for the first time. Early gingerbread recipes contain spices that were once coveted and expensive, like cinnamon, sandalwood, and saffron, which became increasingly accessible after the Crusades. Gingerbread became big business, and local variations began to arise. Lebkuchen, a gingerbread-like spiced treat, became popular in Germany, and guilds of gingerbread makers began to emerge in the 15th and 16th centuries.

As gingerbread makers got better at their craft, they began to press the luxurious creations into intricate molds and even paint them. The sweet treat became a popular way for rich rulers to impress visitors, as when Elizabeth I handed out gingerbread men to visiting dignitaries.

Then, a simple story thrust gingerbread from yummy treat to full-blown cultural phenomenon. Though the original doesnt reference gingerbread specifically, the Brothers Grimm's “Hansel and Gretel” told the story of two children who are left to starve by their poor, hungry parents, then enticed and imprisoned by a wicked witch in a house “built of bread and covered with cakes.”

After the Grimms published the tale in 1812, building gingerbread houses became a popular pastime in Germany. Food historians debate whether the Grimms’ story simply drew on gingerbread houses that were already popular, or whether it gave people the idea in the first place, but it certainly seems that constructing gingerbread houses became a popular activity among Germans right around the time the Grimms began publishing their bestsellers.

Wikimedia // Public Domain

By then, of course, gingerbread was already associated with Christmas. And nobody celebrates Christmas like the Germans, who pioneered everything from Christmas presents to Christmas trees to some of the most popular carols.

In the 19th century, many gingerbread house-making Germans, armed with their favorite holiday traditions, moved to the United States in multiple mass waves of immigrationThe opera “Hänsel and Gretel” by Engelbert Humperdinck, which premiered in Germany in 1893 and in the United States two years later, featured a gingerbread house that may also have increased the popularity of the confectionery construction.

By 1909, Good Housekeeping was suggesting that moms make a “Jack Horner pie” (the term was used as a catch-all for any pastry that had goodies inside) featuring a miniature gingerbread house for a Hansel and Gretel children’s party. And with each Christmas, more elaborate gingerbread creations could be found.

These days, gingerbread structures are so popular that many become tourist destinations, as in the case of a three-ton gingerbread village built each year by a New York chef. And next time you see one, you might want to remember its convoluted history—one that belies its sugar-sweet looks.