Fiancé messaging other females. Ive been so angry with him lately

Fan question. Id like to stay anonymous.
My fiancé and i have been together for a while, A few years. And about last year i caught him sending a very vulgar message to another female. I confronted him and he denied and lied. But i kept bugging him about it because i knew that wasnt the truth. He pulls his friend into it and helps him lie. So fast forward im out of town and i ask again. And he finally tells me he did it. So after that convo i decided i needed some breathing space to think and figure it out and once i get back into town then we can talk more. A few weeks later im Back in town and hes doing everything he can to try to make up. Letters apologizing the whole 9. But keep in mind still no reason as to why right. It took a while but i did forgive him. Not saying that one i wasnt pissed off and two i was extremely hurt. Now fast forward about 3-4 mo and once again i see a vulgar message to a female. Now I’m fed up over it and done. So once again i confront him about it and he still isn’t telling the truth. So for about 2 months we werent together but still coparenting. We come to terms and start our family back. And since last year things have been good. We’ve celebrated many accomplishments together. But in the back of my head everyday i just cant get over how dirty he did me. How wrong it was even if it was just a message. And i find myself beating myself up about dwelling in the past just to remind myself things are better now. But lately ive been angry at myself for not putting my foot down harder and im still very very angry at him. Any advise is welcome and appreciated.