why don't I feel comfortable working out with my boyfriend but I feel comfortable in every single other aspect?!! I'm…

why don't I feel comfortable working out with my boyfriend but I feel comfortable in every single other aspect?!! I'm fairly fit, just haven't worked out since last summer and out of practice a little. I have very little excuses to be uncomfortable yet for,some reason the way he talks about working out intimidates me. I try to explain this to him and the answers I get back from him are only discouraging. tried pointing that out to him also and he got offended.. I think he's not listening and is offended only because he wants to work out with me and can't understand why I don't feel the same way. I do want to though and I try to verbalize this and explain it in a way I think will be easy for him to understand and it still,need gets through, it's so petty for us to fight over this because it's not a big issue, not does it define our relationship but it really bothers me that he just can't get it and that I can't break out of my uncomfortable-ness and every time we talk about it I feel less and less likely to and then he brings out comments like "you'll never do it" to try and be competitive and feisty but I know he's trying to manipulate me and I'm stubborn and don't want to give him what he wants even more and it makes me mad he really thinks I'll fall for this and give in just because of some simple,truck I've seen him use time and time again with so many people. Urg am I crazy?!