So I've been giving my son whole milk for about a month now. And I've gotten so many mixed opinions…

So I've been giving my son whole milk for about a month now. And I've gotten so many mixed opinions on how much is "too much" or "enough" milk. I've heard it can only be 24 oz a day and others saying it doesn't matter. My son is pretty tall and heavy for his age, so I'm not really sure if I'm giving him enough. Any advice?