A lot of really tough guys never quite make it to the top. When I was a kid I was always impressed by Colin Jones and Tony Sibson. They may not have been elite, but they sure were tough.
Dwight Muhammad Qawi. I'm telling you, this is the hardest, toughest, meanest son of a b*tch inside a boxing ring in modern times. Including HWs. This guy was only 5'5" and gave everybody he faced at LHW and CW hell. Michael Spinks and Evander Holyfield can tell you a story about that. He even faced George Foreman later in his career. Bernard Hopkins and Evander Holyfield should also get a mention in this thread.