I just need to vent for a second...

I don't care that my husband goes out. I don't care that it's occasionally to a strip club. With that being said he works so much and alot of the time we just skate passed each other. What bothers me is that his brother asks him to go out like 3 times a frickin week and I'm the bad guy for asking to spend time with my husband. I'm a stay at home mom of two littles it's nice to have adult interactions. we are about to get a house his brother will be moving with us and I'm not sure I can handle being the bad guy all the time. I just want to spend time with my husband.