Yeah, that's going to happen. You can't be hard on yourself about it. Just be mindful of why you are really angry and try not to take it out on the baby. I'm not a single mom and I get stressed and angry at my daughter at night time, too. Well, I guess that's a little different because she's four and HATES going to bed, but still. Things get easier.
It's okay to feel stressed as any kind of mom! It's part of the job description. Unfortunately, it makes it that much harder when you're doing it alone. 😔