My boyfriend (father of my child) has his 3 year old this year on Halloween, he really wants me and…

My boyfriend (father of my child) has his 3 year old this year on Halloween, he really wants me and our (will be) month old daughter to go with them trick-or-treating but I don't want to take out our young baby in the cold. Should I just bundle up our daughter and go with them or should I stand my ground and stay home? I do want to go I just worry about baby getting sick.