Carnera opposition on average would have been a lot smaller than Fury`s, Carnera was far bigger than all his opponents except Baer who was still a good bit smaller.
Carnera fought more elite opponents. He also had more padding on his record, and perhaps as you say a few swims. Realistically, I don't think that either of them is in the argument for being the lightest punching lineal heavyweight champion of all time.