They were tops at the time, but they were oft-beaten tops. There is nothing any more remarkable in the win than beating Chisora, who has been in the top ten for like 14 years now.
It is very debatable. Chisora has contended for the title and been in the top ten for something like 15 years now. You are thinking of Chisora at the end of his career.