AITA for wanting to take a break from TTC?

My SO and I have been TTC a second child. We got pregnant with our first child right away, but this time has been much harder. I gained about 50 pounds since my son was born and I am older now (35+ club), but I still was hoping it would be easy.

The past few months have been rough on me with all the ovulation testing, getting excited just to not be successful, etc. so I decided I want to take a TTC break for 2-3 months and work on losing a bit of weight. If we conceived this month it would also make the due date line up with Christmas, and I do not really want to tackle the holidays on one income (because of maternity leave) or possibly miss Christmas with my toddler by being in the hospital.

Despite saying this to my husband, and him knowing that I started a diet and exercise routine that I’ve already seen results with, he refuses to wear a condom saying I “won’t get pregnant anyway” (which I think is a really insensitive thing to say to me after all the recurring disappointment I’ve felt these past months) and just generally being dismissive by saying that a “baby would be a blessing if it did happen.” Which it would, but I really was in a better headspace lately thinking we were taking a break from this, and I was feeling good about myself for the first time in a long time with the weight loss plan. I feel like he is completely disregarding me so he can have slightly better bedroom enjoyment.

AITA for being really upset about this?