You would have to make Frazier a favorite over all light heavies. He was very dynamic, and feasted on smaller boxer types without a lot of power. The light heavies with the best chance to beat Frazier would need good power, and good chins. I would say Langford has a very good chance. Tunney could do it. Maybe Moore, and Kovalev.
Gus Lesnavich ... Greg Peralta... Murio Mina... If you cast your net far and Stylistically wide, you are still left with Lt Heavy's who may be competitive for a round or two... Whilst Joe is stoking his Furnace.. but once Joe gets up a Semblance of a head of Steam... Its all over ..The Fat Lady is hitting the Showers, Watching an emerging Joe Run over Big Manuel Ramos or Blast out Dave Zygilwitz....(???) I dont even feel Joe has to be at his March 8th 71 Peak... I'm afraid it just aint Happening.. Er..Maybe Bob Fitzimmons Leaves a Horse shoe in a Glove... Then..it still aint Happening... " Hit him in the Slats Bob.." " Bob...Bob...Wheres Bobs HEAD.."
Right after the FOTC, I seem to recall reading where Charley Rose said that Bob Fitzsimmons and Jack Dillon would have beaten either of Ali or Frazier. And also, that even though a middleweight, Stanley Ketchel would have beaten both of them. Not saying that I agree, but the opinion of a man who personally saw most, if not all of these men fight is worthy of consideration.
Gene Tunney could perhaps pull an Ali..... But NO one at Light heavy is a favorite against Prime Frazier. But if I had to go with one it would be him, Holyfield deserves a mention here.
I will add in another underated fighter. Billy Conn..... Any man that gives Joe Louis Hell has a chance to beat Frazier. He has the speed and Movement to not get caught early, and the pop to keep Frazier off for a few moments...... Not a favorite, but perhaps he is the best true LTHVY I can think off with a chance.
Bob Foster got destroyed by Frazier and he was an ATG LTHVY. I can't see many hanging with him. The best chance might be Fitzsimmons but I doubt it...... Prime Joe likely gets any of them out of there within the 15 rounds