In coming years, leaf peepers might still able to enjoy a road trip through fall foliage—but the display might not look the same. While foliage is notoriously difficult to predict, researchers suggest climate change may dull autumn’s vibrant red and yellow hues.

According to Nicole Cavender, vice president of science and conservation for Illinois’s Morton Arboretum, unpredictability caused by climate change—including extreme droughts and floods—could reduce the number of leaves on a tree, hindering the spectacle of foliage colors. One bad storm can change the entire year’s outlook for fall foliage.

Another effect will be the timing of the leaf change: Warmer temperatures generally cause trees to change colors later in the season. A group of researchers recently studied the effects of climate change on autumn phenology, or seasonal changes, and found that 70 percent of their study area (the Northern Hemisphere) experienced delayed foliage. Only the arid and semi-arid regions stayed unchanged.

Of course, Cavender notes, fall foliage doesn’t hinge on climate alone. Factors like the genetics of a tree, environmental conditions such as wind and strong rains, and the tree’s overall health all play an integral part.

While specifics remain up in the air, Cavender predicts that the growing number of frost-free days anticipated in the next 50 years will definitely dull the colors of some of the United States’ most popular fall trees.

The sugar maple, for example, known for its vibrant shades of yellow, orange, and red, is expected to decrease in abundance in America’s natural forests by 2100. It’s also one of many species whose natural habitats will shift north due to warming temperatures. The sugar maple isn’t alone: The yellow birch, beloved for its bright yellow fall leaves, will also migrate north—possibly above the Canadian border by the early 22nd century, according to National Geographic.

Trees aren’t the only species which will move as a result of changing climate: The range of insects are also predicted to dramatically change. The ash tree, which typically has yellow, red and even purple leaves in the fall, is particularly sensitive to insect-borne diseases. One such insect, the emerald ash borer in North America, has been decimating ash trees—although cold winters could help control the epidemic by reducing the insect population, very cold days have decreased by more than 30 percent in the last century, according to NOAA's National Centers for Environmental Information.

These somewhat dire predictions have already changed this fall's display: This year’s extended summer temperatures caused foliage delays in popular leaf peeping spots from Massachusetts to Indiana, among other locations.

But there are some things—besides reducing greenhouse gas emissions—that we can do to help. Fall foliage effects—such as colors, vibrancy, and longevity of those colors—varies by type of tree. Cavender says that by planting a diverse assortment of tree species, we can pave the way for colorful autumns well into the future.

“As the climate changes, it’s critical to plant the right tree in the right environmental conditions,” Cavender tells mental_floss. “The more tree diversity you have, the more likely you are to get fall colors every year.”