Because his listed height of 5'7" isn't that small for a lightweight. But fans and media always bring up how incy wincy he is.
Hes 2-3 inches smaller than most of the top guys at LW. Its more his build that makes him look small in the ring though. Hes not heavy set or broad shouldered like Davis or Lopez.