So I'm worried my son looks too much like me and not enough like my fiance that people may question…

So I'm worried my son looks too much like me and not enough like my fiance that people may question if he is actually my fianc?'s. I got pregnant before we were engaged and his dad was very negative towards us the first few months of the pregnancy. He once said that I only got pregnant to get my fianc? to marry me. Then the other day he asked if any of his genes got into our son because he's pale with light eyes like me and my fianc? is dark. Am I reading to much into it?