Should I be able to move? Custody issues

Should I be able to move or is it wrong of me? I currently share two children with my ex-husband. We currently have 50-50 custody although he is required to travel for his job and will often times be gone for week‘s at a time. I have recently been offered a position that will require me to move about 45 minutes from where I currently live and where he also lives. I have asked him to do a modified schedule and he is telling me that I am being selfish and not thinking of my children. Although I feel like the move would be best for all of us considering that I do not have to travel for work like he does so realistically I am here for them all of the time. His ideal scenario would be for me to leave them here where we currently live and when he travels they will be staying with his mother. There have been a lot of incident’s in his home where inappropriate things have happened to my children CPS gives him a slap on the hand and they have had access to adult content on multiple occasions he also has a DWI. Somehow it has been written in our court ordered custody agreement that we have to reside in the same county. I am remarried have another child and I’m ready for a change for my whole family including my other two children. The past 8 years o have had to love with a few miles of two baby mommas of his, his entire family and himself and his current new wife because of the court order … is it wrong for me to want the fresh start and move the 45 minutes and take him to court to try and get a modified schedule,or should I stay in the county where I currently reside with him. I feel torn, I feel like I have been controlled by the current court order for 8 years and as I see him advancing in his career taking flights and traveling for his job freely. I don’t understand how I’m selfish for wanting to make a move with a 45 min difference for my job.