At the time it was picked apart because a lot of those guys had little experience. I think Holmes made "11 title defenses" against guys with 20 wins or less. That's not a lot of fights (as every Anthony Joshua fan will tell you all day every day). You beat a number of young guys who later become champs, and your legacy looks better down the road. If they don't become champs, people throw you under the bus for fighting so many guys with only 10 or 12 or 14 wins.
And let's not forget Holmes was himself 42 years old and twenty pounds overweight. With this particular opponent Holmes' feat is definitely more impressive.
Yeah, that sounds about right -though I'm not sure if any single opponent of Lewis is as good as Witherspoon (at his best, anyway). Edit: Scratch that; Vitali is probably better than the best version of 'Spoon