I think that's up to you. It's not a given for just anyone. I packed mine probably around 35 or 36 weeks. You can start and add to it. I just got everything and then did it and left it in his room and then when I finished adding little things here and there, I put it in the trunk, along with his diaper bag and set up his car seat in our car. :) But whenever you wanna do it, that's up to you. I don't think it's ever really too soon.
Honestly you could never pack it too sokn but I would see if you know of you are doing c section which could keep you there for 3 days to a week, or vaginal birth which usually is 2 or 3 days. unless there is a complication with baby, and right now until birth he seems healthy and fine? You shouldnt worry about packing more than a bag for at least 4 days.