I'm at 170lbs currently and the fat around my stomach is dissapearing much faster than I expected, I think if I try for middleweight I will look anorexic
You MUST add roadwork. As far as Boxing goes your caloric "money" will be made on the HB and sparring. Depending on your style, the "catch" about Boxing is that the better you are at it the less it takes to be effective. Unless you have an observer or count punches, this understanding slowly creeps in and infects all of your training. IMHO, this is why you see many fat heavyweights. At a certain level they are near untouchable in the gym, but can't beat an elite opponent to save their lives. This isn't a scientific observation, but IMHO, road work is a great "iron clad" way to get you aerobic money's worth.