I had the same issue with my oldest. He would leave with his brother or friends band would leave me alone with the baby ALL THE TIME! Finally I say him down and calmly explained to him that I felt isolated and alone. He apologized and made it a point to either take us out or let me go out by myself when I needed it. ☺