Retroactive Jealousy & Ex-Fantasies: a reality check for your peace of mind

A girl fondly looking at a photo of her ex - he winks back at her

Worried that your partner thinks about their ex? Or even fantasises about them?

Here’s a little thought experiment: right now, your partner is thinking about their ex. They’re reminiscing about someone they had sex with before you.

How does that feel?

Maybe they’ve come across an old photo and they’re looking at it. Or they’re chatting away with a friend and their ex has come up in conversation. There’s laughter and smiles.

I think it’s natural to feel some discomfort with that.

But if you struggle with retroactive jealousy, it’s not plain old discomfort.

It’s wave after wave of intense anxiety, sadness or anger. It’s a deep resentment of some aspect of your partner’s past, and their attachment to it.

If this resonates with you, let’s explore the feeling and see what we can do to prevent it sabotaging you and your relationship.

The grip of the past: understanding intense retroactive jealousy

When people come and talk with me about these feelings, it helps to understand the doubt: the ‘what if’ that’s setting them off.

They can tell me easily. “Well, what if she misses him – or more specifically – misses sex with him? What if she’s still attracted to him?

Or “What if she thinks about him when she’s having sex with me…?

Interestingly, it usually isn’tWhat if he comes back, what if they hook up again…?

How might you describe the ‘what if’ or ‘maybe’ around your partner’s past?

Then I ask about the felt consequences. “This will sound like a stupid question” I say, “but… so what? What if she enjoyed sex with him, and she still thinks about it? What if she thinks about him during sex? What would that mean?”

A typical answer might be “Well it means she prefers him to me. She’s not into me. I’m not exciting enough. And it also means she’s faking, pretending…“.

Often I see the anger, the despair, rising as the thoughts flow.

How might you reply to the ‘so what’ question? Do you find yourself making comparisons between yourself and the ex?

Dispelling doubts: what if your partner does think about exes?

“Is there any evidence of this, I ask?” And sometimes people say “Yeah I can tell she’s tuning out, she goes quiet on me. Even during sex, she’s distracted sometimes. I bet it’s him“.

I’m not sure this evidence would stand up in court. But like most sticky doubts, it’s technically possible. She might be lying there thinking of… Craig.

There’s nothing I can say that proves otherwise. In therapy, there’s no point in me saying “Oh that’s very unlikely. She’s not thinking of her ex… she’s probably just tired…”.

That 100% won’t help.

What I could say is quite the opposite. Our minds DO wander during sex – all over the place.

Research shows that, of all sexual fantasies, thinking of someone other than our actual partner is extremely common.

The truth about Partner-Replacement Fantasy

Ouch. I appreciate when we’re battling with retroactive jealousy, we don’t want to hear the words partner-replacement fantasy.

Well I don’t do that” one guy said to me. “My mind wanders a bit during sex… sometimes I make it wander so I can hold on for longer. But I never think of my exes… why would I?

We don’t KNOW that your partner does it. But it’s possible.

And if they do, research says it wouldn’t mean much. A 2017 study concluded that erotic mental imagery – including thoughts of past partners – is a normal part of sexual cognition.

It’s NOT about dissatisfaction or any comparison to you.

In his excellent book Tell Me What You Want, Dr. Justin Lehmiller explains that the content of our sexual fantasies doesn’t really equate to what we want in reality.

The brain just goes where it goes when we’re aroused. We’ve all experienced this.

So if your partner’s mind IS wandering during sex – even to that past experience that you weren’t a part of – it’s not any rejection of you.

When intimacy needs a tune-up: addressing sexual connection

Tuning out during sex MIGHT suggest that intimacy could be better. Has your sexual connection become a bit lazy or routine?

The way to approach this isn’t insecure interrogation (‘what are you thinking about?‘) but genuine interest (‘what might make us feel closer during sex?‘ or ‘tell me what you want‘ – as the book title goes).

Tweaks to the physical or emotional connection might be all we need here.

Fantasies aren’t the problem. And to go full circle on this, having old photos isn’t a rejection of you either.

A secret stash of photos, loads of explicit photos, more recent photos? OK it would be reasonable to have some concerns about that. But it’s rarely the case.

Retroactive jealousy: a mirror to our insecurities

Here’s what these fantasy worries ARE telling us: that we’re not so secure in ourselves.

If we demand to be our partner’s one and only in their thoughts and their fantasies, we’re way too dependent on our partner to keep us feeling validated and desired.

It’s one of the common traits of people who struggle with retroactive jealousy. Our sense of our own worth is too easily threatened by the recognition that our partner is a different person to us. With a different past, different experiences, and different people in their lives.

We take reminders of this, real or hypothetical, very personally – and retroactive jealousy this way lies.

So developing a healthier appreciation of ourselves is a significant part of overcoming retroactive jealousy. A sense of self we can hold on to, even when times are tough.

And it starts with recognising the old narrative: “my partner is thinking about their ex… I’m with someone who’s faking and pretending, I don’t really know my partner at all. I’m trapped, stuck, I’ll never be enough. How can they do this to me…?

It’s a compelling, well-worn story. It flows, it’s creative, it’s compelling the the most self-doubting of ways.

And understanding how we reasoned ourselves into this doubt is helpful. This habit of needing a positive reflection of ourselves from others – we’ve probably had it a long time.

Putting thoughts, photos and fantasies into perspective

Does understanding partner-replacement fantasies help in this respect? Sometimes, if fears of our partner thinking or reminiscing about their ex are part of our retroactive jealousy narrative.

It can feel incredibly unfair that our partner’s past – something we had no part in – affects us the way it does. We don’t WANT our partner fantasising about other people (well some of us might, but it’s a niche interest).

But even IF our partners do, research confirms that it’s not a threat to us or our relationship.

If you relate to any of this, I hope it’s helped bring some perspective. And I know I used the male pronoun here – but retroactive jealousy isn’t specific to any gender or sexuality, of course. Women can struggle with these feelings too.

Getting clear on the doubt – the ‘what if‘ – is the starting point. And then ‘so what‘ if they think of their ex sometimes?