How would Tyson Fury fare against a good swarmer or an aggressive counterpuncher?

If Mihai Nistor turned pro earlier he might have been that guy.

 

He already fought Chisora three times. The first time he went toe-to-toe as well, wasn't very defensive.
Chisora is bigger and stronger than Frazier and he might not be 'elite', but he'd beat plenty of the old 'elites'.
Hammer is similar size and again not 'elite', but would beat plenty of oldies.
They both came at Fury swinging away and struggled to even land on him when he was being defensive.
Eddie Chambers also described how hard it was to hit him clean.

Fury got nailed by Cunningham because he was playing around and hopped in front of him keeping his guard down, lingering too long.
Now if Haye had been in Cunningham's place, that might have been different. However, Fury wouldn't fight Haye like Cunningham because he knows when he's fighting a big puncher.

Fury can neutralise swarmers with his size and hugs, so I don't think he's especially vulnerable to them. Remember how much Cunningham was complaining about dealing with that and how huge Fury was.

AJ is a different story. He got stopped by Ruiz.