If you've never cheated I don't see why he'd think that, unless he's doubting it because you have 1) cheated on him, 2) cheated on exes or 3) cheated on your ex with him.
Either way y'all need to talk about this as adults. If he can't come to terms with it because of your past infidelities (cause let's face it, almost everyone worries about someone who's cheated whether during or before the current relationship) maybe he should step back from the relationship. It's not healthy.
If he can’t leave the past in the past y’all can’t have a future and if he is denying the baby tell him to pay for a DNA test when the baby is born if he’s 100% positive that baby isn’t his and when it comes back it is his id put him on child support. But that’s just me.