To go or not to go

Hello ladies i really need some help. I am debateing on wether or not i should go to a rock concert with my SO, his friends, and some of their wives. My main problem is leaving my son for too long. The concert with be about 2 hours away and my SO says if we go we might be back home at around 2-3 am. I do have a babysitter in mind which is my aunt and grandma who live next door. My aunt and grandma have alot of babysitting experience taking care of babies of all ages and they are very responsible and love having my son. My son is a very calm and happy 4 (almost 5) month old baby. Some days he sleeps from 10 pm-7am or 12am-9pm so he sleeps very well and is a heavy sleeper. I would want to go with my SO but i just do not know if it is right to leave him for so long. Even though i know people right next door to babysit, my SO and i have not had any dates or alone time since our son was born so it is not like we always leave him. So what are your ladies opinion? My SO wants to go as do i but my only problem is the time we will be back. Opinions please.