I actually feel the opposite. Without these people, boxing wouldn't happen. And some even meet the #1 factor in getting into the HOF, which of course is to be FAMOUS. I feel that it's a BOXING, not boxers, hall of fame; non participants are important and should be honored along with the fighters. But I totally respect your opinion.
It’s not the Boxer Hall of Fame, it’s the BoxING Hall of Fame, and there’s no boxING without these people. The public will always glom onto the fighters, obviously, but that doesn’t mean greats in the sport who played other roles should not be enshrined, appreciated and recognized.
If the rules stipulate that non participants can be entered then of course he should be in. If his dubious background is an issue then it should be for some of the boxers that are in there.