I'd lean toward Palomino, but I don't think the difference is as much as some people seem to think. Remember that Carlos was 0-2 when he stepped it up against HOF competition, with those losses being rather decisive.
Although i though Benitez beat him convincingly Palomino was competetive and it was a split decision loss in Puerto Rico. Duran definitely dominated him but he hung tough. But in Vargas favor all his losses were to ATGs and he gave them all they wanted except for the 2nd Mosley fight, when it was clear that Vargas best days were over.