Right! Even Pac himself wasn't sure he could compete well against Oscar. I could still recall him saying on local tv (Manila) that "Oscar might just kick his ass" when told of the probability of fighting De la Hoya a few days after he won the LW belt from Diaz. He wanted to unify the LW division but I guess the lure of the mighty $$$ was too strong.
He wanted to fight the best of the slick African-American boxer but the guy didn't want to spoil his extended vacation!