Given the Midas touch he has with movies nowadays, it’s easy to forget that Robert Downey Jr. used to be one of Hollywood’s biggest liabilities. Though he has always been a formidable talent, a series of drug-related arrests and failed attempts at rehabilitation made him a tough sell to studios, particularly in the mid-1990s. (Even in the case of Iron Man, the filmmakers had to aggressively sell Downey for the role after the studio rejected the idea. Twice.) In honor of Downey’s 50th birthday, we’re recounting six films that might have featured him on the movie poster had life gone differently.

1. PRETTY IN PINK (1986)

Imagine if Duckie had gotten the girl at the end of Pretty in Pink? That was, in fact, how the movie was intended to end. But the script changed after Jon Cryer landed the part, despite Molly Ringwald pushing for Robert Downey, Jr., who was competing with Cryer for the role. In the book You Couldn’t Ignore More if You Tried: The Brat Pack, John Hughes, and Their Impact on a Generation, Ringwald explained that the changed ending made more sense to her once Cryer came on board, as she just wasn’t feeling the romantic vibe. As Ringwald put it, “I mean, if they remade the movie now, [Duckie] would be, like, the gay friend who comes out at the end. … I feel bad saying that I really fought for Robert Downey, Jr., because it sort of seems like I don’t appreciate Jon’s performance, which I totally do—it’s just, it really did affect the movie.”

Cryer addressed Ringwald’s comments in the DVD commentary on Pretty in Pink: Everything’s Duckie Edition, saying that, "Molly dropped the bomb that she would’ve been fine with the original ending if Robert Downey Jr., had played Duckie … But since it was me, she just couldn’t see it. It was like, wow, so I’m that unattractive? Thanks, Mol!”

2. SAY ANYTHING… (1989)

Had Downey been cast as Lloyd Dobler in Cameron Crowe’s beloved high school romance, he probably would have owned the role. But today, it’s hard to imagine anyone but a lovestruck John Cusack hoisting that boombox up over his head, Peter Gabriel’s “In Your Eyes” blasting. Though Crowe wrote the role specifically with Cusack in mind, he wasn’t sure that Cusack (who was 23 years old when the film was released) would be interested in playing another high school student. So he went against his gut and auditioned other actors for the role, including Kirk Cameron! The role was actually offered to Downey, who turned it down—prompting Crowe to go back to his original idea and offer it to Cusack, who signed on immediately.

3. BACKDRAFT (1991)

Truthfully, we’re not exactly sure how high up on Ron Howard’s list Downey was to play firefighter extraordinaire Brian McCaffrey. But Downey’s consideration for the part is particularly amusing considering that you can actually watch his audition tape (right here), and that he got beat out for it by William Baldwin. In Downey’s defense, Brad Pitt also unsuccessfully auditioned for the part.

4. WILD THINGS (1998)

Much like Showgirls, there are two schools of thought when it comes to this strange erotic-teen movie hybrid: love it or hate it. Which is something that Kevin Bacon, who both starred in and executive produced the movie, understands. “When I first picked up the script, I thought, ‘Oh, my God, this is the trashiest piece of crap I’ve ever read,’” Bacon told Entertainment Weekly. “But every few pages, I kept discovering that it wasn’t what it seemed. Every few pages, there was another surprise.” One surprise was having to replace Downey, who had been cast as Sam Lombardo, with Matt Dillon. “It was during his rehab, and he’d just been on Diane Sawyer’s show,” the movie’s director, John McNaughton, told Entertainment Weekly. “And to the people in Hollywood, that was a great career move. That made him hot.” But that didn’t matter to movie’s insurance bonders, who were reluctant to cover Downey. “There were just too many lawyers and insurance people and bond-company people involved,” said producer Rodney Liber.

5. AMERICA’S SWEETHEARTS (2001)

In 2000, after spending a year incarcerated, Downey’s career seemed to be on the upswing. After landing a regular role on Ally McBeal, he was being touted as the next great comeback story … until he was arrested again in November. The show’s producers stuck by him, but the question mark hanging over his availability and fitness to film led producers on America’s Sweethearts to rethink his casting as Eddie Thomas, and eventually replace him with John Cusack. (When Downey was arrested again just a few months later, he was fired from Ally McBeal, too.)

6. MELINDA AND MELINDA (2004)

Woody Allen faced double trouble when it came down to the paperwork on his 2004 dramedy, Melinda and Melinda. “We were originally going to work with Winona Ryder and Robert Downey Jr. and I couldn’t get insurance on them,” Allen said in Conversations with Woody Allen. “The insurance companies are very prissy and sticky and gave us a hard time. We were heartbroken because I had worked with Winona before [on Celebrity] and I thought she was perfect for this … And I had always wanted to work with Bob Downey and always thought he was a huge talent.” Ryder and Downey were replaced with Radha Mitchell and Will Ferrell.