Due to Ali's stance on the Vietnam war and refusal to enter the draft Ali lost more then three and a half years of his CAREER!
It's very important. He has one of the best resumes in the history of the sport ALTHOUGH he lost 3 years of his prime.
It's tough to say because we don't know how Frazier would have done against him if Ali wasn't stripped. You have to take the layoff into account of course, but I've always had the impression Frazier was destined to win at least one against Ali. A meeting in the 60's could have been that one.
When considering his place amongst other heavyweights, not that important. His resume is so good that I immediately have him at number 1 ATG amongst the heavies. Some would argue for Joe Louis, but I have Ali at 1. If you want to argue ATG in a P4P sense, then having 3 years taken out of his prime really hurts him there, because he doesnt have what would have been some really great dominating performances during that time.
or maybe he would have had some off nights and underperformed and it would have hurt his legacy. you can never tell really. You can only assume. Maybe having 3 years off helped his legacy more than hurt it because people just assume he would have destroyed everyone.
Also frazier wouldnt have time to build himself up as much. He wouldnt have been "a champion" like he was by the time they fought in fight of the century. So then in that case, if Ali beat him say in 68 or 69, the win wouldnt have been as meaningful. And if Ali lost to him, it would have hurt his legacy more sense Frazier wouldnt have been ranked as highly. Its all very hard to try to guess what would have happened. Maybe Ali wouldnt have lasted as long into the 70s as he did. Or maybe Ali would have had a long undefeated streak and retired at the top. You never know.