We all have those friends on Facebook who threaten to move to Canada when politics don’t go their way. As it turns out, things weren’t that different back in the 1800s—but unlike your pals on social media, these disillusioned Confederates actually followed through with their threats to leave.

When the Civil War ended in 1865, many Confederate supporters decided that the South was no longer hospitable. Tempted by Brazil’s Emperor Dom Pedro II, a Confederate ally who took out newspaper ads offering land subsidies, more than 10,000 Southerners moved their families to South America to start over, planting cotton, sugarcane, and coffee. Though many eventually returned home, some stayed and became known as the “Confederados.” The most successful colony was known as the Norris Colony, founded in 1865 by former Alabama Senator Col. William Norris and his wife, Isabel. It eventually grew to include 100 families.

Descendants in the area still pay homage to their heritage with celebrations called “Festa Confederada,” which are held up to four times yearly in the Confederado graveyard. Participants arrive dressed in hoop skirts and gray Confederate jackets, then dance to vintage tunes such as “The Yellow Rose of Texas” and “Dixie’s Land.” They eat fried chicken and fly the Confederate flag with abandon. They say there are no racist intentions to their celebrations, but that they fly the Stars and Bars simply to recognize their very unique past.

That past is something shared by President Jimmy Carter and First Lady Rosalyn Carter. In 1972, the Carters, then the Governor and First Lady of Georgia, visited the Santa Barbara d’Oeste region of Brazil, in part to visit the grave of Rosalyn’s great uncle, who was one of the original Confederados. The Carters were shocked to find so many residents milling about with their own South Georgia accents. Even now, five generations after the first Confederados settled in Brazil, many bilingual descendants speak both Portuguese and English—the latter with a pronounced twang.