What should I do?

I need advice. My son’s dad called me the other day. Saying he was changing his last name to his step dad’s (no problem hes close with him) but he wants me to change my two boys last name to the same thing he’s changing his too. My kids don’t know his step dad. My oldest hadn’t seen him in 2 1/2 years or more and he’s 3. My youngest has never met him. I told him I wouldn’t be changing their name and if I were it would be to mine as the person that’s been there since day one. Was I to harsh? I don’t feel like it’s would make since nor would it be right as they know their name and aren’t even close with their dad.( He just doesn’t show up) I don’t think names are something that should be changed on children just because. Was I to harsh? Should I consider? Or should I keep it where the conversation was left and what their name is? They got his last name at birth but now he wants me to change it to his new one bc he thinks his kids should have the father’s last name…