Aside from the fact that there was a lot of talk about his outside-the-ring activities, I think Chocolate was thrown in with too many high quality fighters too quickly into his career. Throwing him with a much bigger, more accomplished veteran like Kid Berg so soon into his career was absolutely nuts IMO, and a lot of reports at the time thought the body beating Berg gave him must've taken something out of him. And yet Chocolate was right back fighting LaBarba and Battalino in short order after that, and then Canzoneri up at 135 not long after that.
That was the way it was in those times, as the careers of Canzoneri, LaBarba and Battalino also tell. Chocolate I think would be known as one of the greatest ever had he fought in the "modern" ring. Longer layoffs, stricter officiating and better match-making would have done him well, considering his vast skillset. It was factors other than that which let him down. He could hang in there today and not look dated one bit.
Even in his own time though, Chocolate's matchmaking was considered extreme. For example, a Ring magazine article from the early '30s (printed shortly after Chocolate's losses to LaBarba and Battalino) cited Chocolate as a prime example of a fighter who was matched too tough too soon, and appeared to burn out as a result.