leave the house when dad is on duty. baby feels nerves and tension from all. maybe dad feels like he’s doing it wrong sort of speak. it takes time.
I’m not sure if maybe dad is a new parent, but my son did the same. He was my second baby but daddy’s first baby. They sense daddy’s energy just like they do mommy’s energy. First time parenting is an anxious time, and negative emotions can stress baby out. Just a thought if this sounds like your own situation. I used to be right at my partner’s hip when I passed baby over. Or I would lay baby in my spot in bed while daddy laid next to him. Now, they’re inseparable lol.
Try having your husband drape one of your freshly worn tshirts over his shoulder. So when he holds the baby, the baby will smell you instead of dads smell. If he wears a powerful cologne or something maybe refrain from that for awhile too.
Time. And patience. My now 4 month old was the same in the beginning. There are still moments when she very much prefers me but she loves when he holds now. 
1 Like
Place a couple burp cloths near you in bed so they get your scent, then when your husband hold him have him use them. This way he has something with your scent too; it will help him get used to both scents. Eventually he will become more familiar to your husbands scent too.
Infants are very sensitiveand sometimes they seem to pick up on certain peoples touch or chemicals put out by every person- as the infant gets a few weeks older and begins to interact with everyone - he or she is less likely to react to those stimuli and instead respond to voice and eye contact- just hang in there- it will get better as the baby becomes more mature!
Skin to skin with dad and mom leaving the room. If baby senses you, he’ll cry for you (especially if you breastfeed and he smells your milk). May just take time to warm up to dad
Is your husband stressed at all ? Our babies can feel that
Does daddy talk and make sounds babies like that
Have you tried skin to skin contact. Have him hold him without a shirt