Montel Griffin, Virgil Hill, Antonio Tarver, Reggie Johnson and Clinton Woods etc, were better than most of Joe's defences. Most of Joe's defences were against non world level opponents.
Most of those were mandatories and keep busy fights. But they were still better than the non world level guys who Joe fought. If we compared Roy's better wins from MW-HW, then there'd be no comparison.
Joe should have moved to LHW when he knew that he couldn't get the Ottke fight in the early 00's. Yet he didn't do that. He just happily stayed at SMW fighting whoever Warren and the WBO gave him, long before Lacy and Kessler came on the scene. Joe had zero ambition back then.
He fought an end of career Eubank on just 11 days notice. He obviously didn't fight in the Eubank era.
Joe is massively overrated by many. How was Lennox's career managed? Lennox fought just about everyone who he could fight.
Lennox is not a good enough boxer to be ATG HW No.3 or No.4 (most), but his resume is. Because his career is very well managed; both losses avenged; beat everyone he stepped into the ring with; Lennox retired as the Lineal HW Champion. Joe Calzaghe's career is even better managed. Undefeated lineal undisputed SMW Champion; beat two ATG LHW "Names" late in his career.
In my opinion Calzaghe, besides being the best super middleweight ever, is also a great champion. He defeated 10 boxers who were world champions (and you don't become world champions by luck) and at the end of his career he solidified his legacy by defeating a strong boxer like Kessler (and if you are not a strong boxer you cannot beat someone like Carl Froch) and a great champion like Hopkins (the executioner was 43 years old but he was still very strong, after the defeat with Calzaghe he dominated the undefeated Pavlik and won the world light heavyweight title against Pascal).
I think it's debatable that he's the best. But it depends on the criteria you use. It's okay saying that he beat 10 world champions, but they weren't champions at the time, so we need to apply context.
Joe's resume and undefeated career are even better timed than Lennox Lewis' situation. It also makes Joe look bigger than he really is It's just that the difference is bigger in Joe's situation than in Lennox's.