What should I do?

My daughters father is being difficult. He was not a good person and I regret ever meeting him but our daughter is perfect in every way. Anyways…. He has no stable home, hasn’t had a job in over a year. We do 50/50 but I recently started tracking him via apple airtag and have other proof he’s had our daughter out at all hours… like 2,3,4 am! He swears he doesn’t but obviously I wouldn’t say it without proof. I suggested a lesser parenting time so he could get his life together and he says he can’t do that because he’s depressed when she’s not with him and he’s rather struggle for a few years to get the time with her. He lives in a crap hole camper, has brought her to me with no car seat, her car seat in the front seat, dirty, same clothes as days before etc. I’m not saying he’s a bad dad but he has his issues. My daughter is two and has recently started begging to come home no matter where she goes. He got her today and 6 hours later he was calling me saying she wants mama. So, am I in the wrong for wanting to keep her more till he proves stability?