No, a fighters physical prime or peak is 25-30. The difference is boxers rarely fight 50 times in a career. Back in the good old days, it wasn't uncommon for a 30 year old fighter to have 100 fights. You think guys like Mayweather, G, Pac and Hopkins would have fought into their 40's if they had 80-100 fights by the time they were 30? The human body has not evolved. Boxers fight less, but fight longer.
That's it. They last longer because they fight less. Prime is still the same age but the decline doesn't have to be severe. In the old days when a fighter started to slip a bit he'd probably soon be beaten down and turned into a has-been .... because he was still expected to fight regularly and against good fighters. Nowadays aging fighters can rest and cherry-pick through their declining years, so they last longer.
I hate this "past prime" comment that if youre past a certain age cant physically be as good as were when you when you were younger. Countess times this been proven wrong eg Chisora was physically the best shape he has ever been in his last last fight and you could see the difference physically how better and fitter Wlad looked against AJ than he did in his previous couple of fights. Training, diet and discipline play a big part.