I think it's about the specifics of being big. Speed and reflexes are less relevant because they're all kinda slow because of the weight anyway. In that situation skills, experience, power, mental aspects and stamina become key and those either don't go so fast with age or are actually improving, may even peak in mid to late thirties. Also, they don't cut weight which gets harder on the organism the older you get.
I think it has to do with the weight loss process. By the time they get to 35 there's going to be a couple lbs of loose skin, bits of fat around organs, scar tissue, etc that you can't cut anymore so you're going to move up and face bigger competition. There is also the mental aspect. When you're 21 it's easy to do weird stuff because you don't know what normal is. Ten or fifteen years later, do you still want to be shadowboxing in a rubber suit under heat lamps? So you either go up another weight class or you force yourself through the grind and it takes that much more of your energy. I don't think there's much else to it. GGG, who seems to have always been disciplined and careful with his weight management, is doing quite well at an age that would be advanced for a HW.
They get old man strength, mass & toughness Heavyweight boxing more than any other division is about physicality and mentality over boxing skill balls, heart, toughness and unbreakable mentality This is why Joyce Fury is the best fight at HW right now. Fury is not the young whipper snapper who'd box circles around Joyce all night anymore. Joyce can match his workrrate and handle his punches. So Fury has to force himself to box with agility which is fading from him to edge a decision. It's a great fight.