That's 100% true. Wilder was consensus number 2, that's why The Ring didn't want to put their title on the line for the AJ vs Parker fight.
It is the sanctioning bodies that are at fault, it is their belts Usyk holds, Usyks legacy is indelibly written in the annals of boxing history, any future fight is not going to diminish or improve his legacy, so if sanctioning bodies not forcing him to face the likes of Kabayel, Wardley etc, then who can blame Usyk for playing the money game. I am strong believer in the title being defended against the best, and defended regularly, but the heavyweight title has been held and not defended for long periods since the days of James J Corbett.
Did Wardley call him out? Was there an offer on the table? No? So no, he couldn't. And Wardley isn't the champion, he's a paper title holder.
Neither does people refusing him as the 2nd biggest name hw, make it a reality. who do you think were the biggest names in the division at the time?
Kabayel, Hrgovic, Zhang I would all favor to beat him, probably Parker as well. Usyk would be favored anytime after his move to heavyweight. His popularity was far beyond his proven ability