He was obsessed with money. He didn’t have time to bother with things like tact or good taste. Undoubtedly one of history’s greatest showmen, P.T. Barnum birthed some of America’s greatest amusements. So why did two of his most famous landmarks burn down in spectacular fashion?

By the 1860s, Barnum was one of the most famous men in the world. His tours and shows may have been sensationalistic and even trashy, but they were huge hits in a country that was learning how to enjoy its rare moments of leisure in a new way. Among his accomplishments were hoaxes, freak shows, and something he called The American Museum.

From a modern perspective, Barnum’s “museum” wasn’t a museum of all. It was a cabinet of the weird and the gauche—a bizarre mishmash of history, taxidermy, technology, and outright exploitation. Its rooms boasted oddities like “THE GREAT MODEL OF NIAGARA FALLS, WITH REAL WATER!” and a tiny doll house in which “General Tom Thumb,” a 32-inch man coached and trained by Barnum himself, lived and entertained.

To say the museum was a hit would be an understatement: thousands of visitors forked over a quarter to visit each day. But hard times hit when Barnum’s museum burned not once, but three times during the 1860s.

William England // Getty

The first fire, in 1864, was perhaps the weirdest: as the Civil War peaked, members of the Confederate Secret Service plotted to burn down several prominent hotels in New York City in a bid to disrupt that year’s presidential election, rob treasuries, and free Confederate POWs. They didn’t succeed (the plot was foiled in part by rebels who were intimidated by upped police presence in the city), but a couple of weeks later they went to try again, and one would-be arsonist decided to deviate from the plan when he set fire to Barnum’s museum, which was across the street from the famed Astor House.

The crime is thought to have been partially motivated by Barnum’s outspoken anti-slavery views (even though Barnum, always a man of contrasts, actually owned slaves at one point and did much to further blackface and minstrel shows in the U.S.). Drunk and angry, arsonist Robert Cobb Kennedy walked into the museum, threw an incendiary device known as “Greek Fire,” and walked out again.

Chaos broke out in the museum as thousands of visitors poured out onto the street, but nobody died and property damage was not catastrophic. That wasn’t the case in 1865, when all hell broke loose at the museum. A furnace in an adjacent restaurant sparked the blaze—or did it? By that time, Barnum was even more popular and visible: he was speaking in his capacity as Republican member of the Connecticut House of Representatives when he was told the museum was completely destroyed.

The New York Times mourned the fire, which, "while greatly injuring and materially impoverishing its enterprising and public-spirited proprietor, did a damage to this and the adjacent communities, which neither time nor money can replace.” It memorialized the museum with an extensive catalog of its contents, which included everything from a fortune teller to an aquarium complete with whales to a woman demonstrating a new-fangled sewing machine.

Barnum vowed to rebuild the museum, and soon he had reopened his curiosity. But in 1868, it burned down a third time—in the middle of a cold snap in March, a huge blaze swept through the building. As firefighters battled the fire, the museum froze over in an eerie spectacle almost as amazing as the rebuilt museum itself. Again, Confederate spies were suspected, although the cause of the blaze is uncertain. What is clear is that “a menagerie” of rare animals perished.

The showman suffered even more fires as the years went on. Though the 1868 fire was the last straw in terms of the museum business, Barnum switched his attention to the circus business. But in 1872, a grand circus building called the Hippotheatron burned to the ground, too. Barnum’s circus animals may have died, but his dream of a gigantic entertainment palace didn’t: the site of this final blaze ended up becoming Madison Square Garden.