No, probably not. But that doesn't discredit the legitimacy of Fury's claim to the lineage. Beyond that, I certainly don't think Fury's the worst lineal champion.
He is going to be in the discussion if he never comes back TBH. His resume hangs entirely on one good win, and his other wins are going to look increasingly unimpressive, as time obscures the details. It is always difficult to persuade a modern fan, that a fighter like Chisorah from the 1950s, was any sort of a win!
Your points are valid, no doubt. But I don't think that invalidates what I said about the legitimacy of Fury's claim. At the end of the day, guys who end dominant streaks tend to be remembered, if for nothing else, for being the man who ended a dominant reign.