There are effectively two fist fights between Bane and Batman in The Dark Knight Rises: the first one in the sewers and the second one in the streets.
In the first one, Batman is very clearly overwhelmed and nearly killed in six separate ways. In the second one, however, Batman overpowers Bane in just a couple minutes because he aimed for the mask.
Which begs the question: why didn't Batman go for the mask in the first fight? The CIA Agent recognized the importance of the mask in the first scene of the movie, both laying out its importance to the audience and also showing that a relatively uninformed agent could piece together Bane's weakness, which should make it trivial for The World's Greatest Detective to figure out.
EDIT: Having watched both fight scenes again, Batman does go for some head blows during the first encounter. However, the blows sound like fleshy sounds compared to the metallic punching sounds of the second fight. Presumably to show that Batman's hitting the actual mask as opposed to the face behind it.
Also worth noting that Batman had way more clean punches in the first fight than the second one, which means way more of an opportunity to do some damage to the mask specifically.
For reference: First fight:
Second fight: