The grass isn’t always greener. Although Ben Affleck moved on with Saturday Night Live producer Lindsay Shookus, 37, after splitting from Jennifer Garner in 2015, he’s now doubting that decision. “He’s not as happy as everyone thinks,” an insider exclusively tells In Touch.

“The truth is, Ben has huge regrets about how everything went down with Jennifer. He misses his old life with her and the kids,” the insider continues. “The decision to move on from Jen and the kids has been very hard on him.” As he pines for the past, the 45-year-old thinks his behavior may have been the result of a “midlife crisis.” The insider adds, “He’s just trying to concentrate on co-parenting and being the best dad possible, but it isn’t easy.”

Despite everything that’s happened between them, Ben and Jen decided to end last year on the right foot. As In Touch previously reported, they even spent the holidays together, just like they used to! Now, the Batman actor is doing everything he possibly can to stay on good terms with his former love. The father-of-three even went back to rehab in December to prove he’s changing for the better.

According to a source, Lindsay can feel that she’ll soon be a gone girl. “She thought Ben was over Jennifer, because he truly convinced her that was the case,” the source said. “Now she’s telling friends that she knows ‘his heart isn’t with me.’”

Have a tip? Send it to us! Email In Touch at contact@intouchweekly.com.