Confessions: Ex-husband is back in my life, wants a fresh start

I am 38 and a divorced mother of two. I met my ex-husband in the university where we fell in love and married immediately after. However, he had a child with another woman who I didn’t think was a threat then despite the fact that he was taking care of both the mother and child. After our second child, their relationship became intimate and he started mistreating me so much that I divorced him. He moved in with her and it has been four years now. The problem is that I still love him so much that I am no longer bitter. Whenever he comes to see our children and he says he still loves me and he married the other woman out of the circumstances I put him in. He wants me to be his lawful wife for life. He has still not married the other woman -- they have only been in a living-together sort of arrangement. I need your advice on what to do in this situation.