Mate, this is one of those matchups that makes me sit back and smile. Two absolute engines of violence. Two men that actually lived to fight. I think they would have clashed beautifully. It's a crying shame they didn't come along in the same era. 'imagine those two fighting a trilogy?! I can't even begin to imagine who'd come out on top .
Holyfield is an immensely robust man. Look at the rounds he did with punchers, to little mid-term affect, it's crazy. Joe, on the other hand, started to fall apart pretty early. Holyfield would Ali him I feel, but I do think Frazier would get over on him until he dropped all the way off.
This would be 50/50 up until the late rounds where Holyfields superior offense, skill and physical strength take their tool
Holyfield's fight against Bert Cooper is a good example of what a fight against Frazier probably would look like. Holyfield had some major issues against Cooper and damn near got K'oed. Frazier was a level or two better than Cooper. Frazier was defensively better than he was while being more aggressive. I can easily see Holyfield getting caught up in the moment ( As he had a tendency to do) meet Frazier's aggression, with his own, playing right into Frazier's hands, getting battered and stopped with in 9-10 rds. In a 3 fight series I see Frazier winning 2-1. And not because Holyfield didn't have the ability to beat Frazier the majority of the times they fought, but because his heart and him being so headstrong wouldn't allow him to. Similar to Holyfields three fights against Bowe.
In the rematch Frazier beat Bonavena without question, their first fight happened in Joe's 2nd year as a pro- Whereas Holyfield struggled against Cooper's power in his 7th year as a pro and 4th year as a heavyweight.
Would be a fight for the ages but I'd bet on Frazier TKO or eventual decision. Higher workrate, better skill, and the power/style to shake up and frustrate Evander. If it were a trilogy, Holyfield would eventually get the better of Joe, since he aged better as a fighter. 1-1-1 or 2-1 for Holyfield in that case.