For me, this is ridiculously easy: Dave Parriss. I can't say of all time because there have been so many bad ones out there throughout the history of the sport. But Parriss is far and away the worst I have ever seen. I have only seen him ref 3 fights: Tszyu-Hatton, Barrera-Khan and Rogan-***ton 1, but he was absolutely incompetent and biased in all 3. There was not a redeeming quality on his part in any of those 3 fights that could ever justify his getting employment as a boxing ref again, but he did. Here's to Dave Parriss, the worst of the worst!!!
I think Weeks is normally a solid referee, but I just shake my head when I think about his decisions in Hopkins Jones II. It wasn't exactly an easy fight to referee though. With Cortez, the issue was that he didn't let Mayweather and Hatton fight on the inside. Weeks let Hopkins and Roy actually fight, and did a good job in that sense. So it's very subjective as to what you think is worse, and if you are judging a ref on a specific fight or his entire career.
Cortez dosent even watch the fight and hes the closest in there, Ive seen him in replays and hes staring in the other direction HAHA .
Consistancy if Cortez was always the same fine, But he had been incharge for Hatton fights before and had no problem with Hatton style in the Hatton fight he allowed 0 inside fighting, Then watch Hopkins V Calzaghe
The ref in Eder Gonzalez vs Juan Martin Coggi takes the cake. He was banned from ever refing right after. http://www.youtube.com/watch?v=oL6eq_NxKbY