Simple solution, get rid of both, and use the Ian John-Lewis clone Carson Jones in his place, no one would tell the difference apart from the refereeing improving.
Why are boxing refs not scrutinised and constantly being evaluated? If they fail on British level fights then they are demoted to Area and so on. I don't even think you should be allowed to ref if you are overweight. You need be moving like a boxer and ready to act in a split second. Much like a premier league ref runs all day just like the players.
If they fail at British level they should be banned, far too much danger in boxing to think it's fine to drop them down a level or two as a punishment like they do in football