At 142 years old, the Kentucky Derby is one of the longest-running continuous sporting events in the United States. A case could be made that it’s also the longest-running fashion show. Each year on the first Saturday in May, locals, wealthy out-of-towners, and celebrities flock to Louisville’s Churchill Downs with a singular goal: to be the best-dressed person at the track. Wide-brimmed straw hats adorned with flowers, feathers, and lengths of ribbons, and petite, impossibly shaped fascinators sit atop the heads of the well-to-do, and the resulting looks are covered by the media nearly as breathlessly as the three-year-old thoroughbreds who are there to compete.

The history of hats at the Kentucky Derby is as old as the race itself, and can be traced back to one man: Colonel Meriwether Lewis Clark Jr., who opened Churchill Downs in 1875, envisioned a day at the races as a high-society event, one on par with England’s famous Epsom Derby, where men and women attended in full morning dress. “Around that time, there had been a lull in thoroughbred racing due to the destruction of the Civil War,” Chris Goodlett, the curator of collections at the Kentucky Derby Museum in Louisville, tells mental_floss. To compensate, Clark dispatched rich, fashionable women to recruit other upper-echelon types to attend the race. It worked, and the American horse racing industry was transformed from a den of gambling and drinking—a place no proper 19th-century woman would venture to—into the high-class spectacle we know today.

After the runaway success of the very first Kentucky Derby, racetracks became a place to see and be seen—and for ladies to show off their finest hats and clothes. “By 1916, you had ads appearing in the local paper for ladies to come into Stewart’s department store and buy their complete Derby Day looks,” says Goodlett. “But up through the 1950s, it was customary for women to don hats to society events, so wearing them to the Derby didn’t necessarily receive the level of attention we see now.”

That changed in the 1960s, says Goodlett, when everyone virtually stopped wearing hats—except at the Kentucky Derby. “At that point, the Derby was the only institution really keeping hats on the map,” notes Bri Mott, founder and CEO of Fashion at the Races. And the media, especially the visual medium of television, loved it. “The hats became more of a novelty, and people started going in for more avant-garde creations,” says Goodlett.

The tradition has stuck around, even while everyday hats have not, because getting dressed up for the races is a simple, fun way to lose yourself for a little while, like putting on a costume, says Mott. “Horse racing is known as the Sport of Kings, and when you put on something special and go to the track, you feel like a queen or king for a day.”

In 2012, Churchill Downs, together with luxury watch brand Longines, launched its first fashion contest, held at the Kentucky Oaks, which takes place the day before the Derby. “That really started changing the game for women,” says Mott. “Now, they weren’t just dressing up for themselves, they were competing on a stage for a prize.” The first year had about 50 women contestants; last year that number jumped to 200. And this year’s fashion competition, on May 6, will have photographers from both Vogue and Vanity Fair on hand to cover it.

Though he couldn’t have foreseen it at the time, Clark not only changed the American horse racing industry—opening Churchill Downs, creating the Derby, and writing many racing rules that are still followed today—he changed the face of it, too. “The Derby has never missed a year since 1875, not due to war or the Great Depression,” notes Goodlett. “And with tradition comes expectation. If you don’t know anything about the Derby, people will say, ‘Oh, it’s customary to wear a hat. You better get one.’”