Is or has anyone ever had a problem with breastfeeding? My son is almost about 3 weeks now and my…

Is or has anyone ever had a problem with breastfeeding? My son is almost about 3 weeks now and my milk came in on day 5 but then I got a cold and with all the hormones and emotions (stress) my milk supply has gone way down when it was never really established in the firat place. Any suggestions or tips would be appreciated!