What's tragic about Taylor's conditioning is that it hurts his activity in the ring, and an active Taylor is a virtually unbeatable Taylor. With Taylor's handspeed, flashy combinations, and head-snapping punches, he always ALWAYS wins the rounds in which he is the most active fighter. Not only that, but he gets touched less himself when he is on the offensive. The one exception to this may have been Winky Wright. Both men took turns boxing each other in that fight and Winky was clearly the more skilled boxer, but less powerful puncher. But as a general rule, when Taylor is throwing punches, he gets the better of the exchanges and is usually winning the fight.
Rounds 4 and 5 of the first Pavlik fight are good examples of what you said. Jermain got his ass handed to him in the 3rd, but then came out in 4 and 5, kept up with Kelly's work rate but with greater accuracy and better defense, and consequently won those rounds. If he could fight like that for 12 rounds, he would be damn near unstoppable.