Ever since their introduction in Season 3 of Game of Thrones, The Unsullied had the reputation of being the finest trained army in Westeros. But in S05E04, how come a bunch of Unsullied soldiers (who were trained to be killers since they were children) led by Grey Worm are overwhelmed by untrained Sons of The Harpy so easily?
Unsullied battle tactics are based on the legions of the old Ghiscari Empire, involving large groups of them fighting in lock-step phalanxes using spear and shield, though they are also trained to use shortswords for close-quarters combat. Slave-eunuchs who have been trained from birth to fight, the Unsullied are renowned for their utter discipline on the battlefield, both in their usage of incredibly coordinated large unit phalanx formations, and because they will never break in the face of overwhelming odds, even to the point of death.
They were simply overwhelmed. That they survived so long and took down so many Sons of the Harpy is a tribute to how skilled at arms they are, and how fearless. Such was their down-fall in this instance. A brave warrior stands and fights, and dies, but a smart one (someone like Bronn) would flee and live another day.
The Sons of the Harpy know this, which is why they tend to attack fewer and less organised members of the Unsullied in tight streets and alley ways. First one person in a brothel, then two, then five and so on.
And from the Books:
They fight in formation as light infantry, equipped with short spears, swords, round shields, and distinctive spiked caps. They fight fearlessly and obey without question. Their elite, highly specialized training makes them most effective in their phalanx formation. 
Ser Barristan and Dany have this discussion about this exact problem in the books:
Daenerys pushed her hair back. “Find these cowards for me. Find them, so that I might teach the Harpy’s Sons what it means to wake the dragon.”
Grey Worm saluted her. His Unsullied closed the shroud once more, lifted the dead man onto their shoulders, and bore him from the hall. Ser Barristan Selmy remained behind. His hair was white, and there were crow’s-feet at the corners of his pale blue eyes. Yet his back was still un-bent, and the years had not yet robbed him of his skill at arms. “Your Grace,” he said, “I fear your eunuchs are ill suited for the tasks you set them.”
Dany settled on her bench and wrapped her pelt about her shoulders once again. “The Unsullied are my finest warriors.”
“Soldiers, not warriors, if it please Your Grace. They were made for the battlefield, to stand shoulder to shoulder behind their shields with their spears thrust out before them. Their training teaches them to obey, fearlessly, perfectly, without thought or hesitation … not to unravel secrets or ask questions.”
-A Song of Ice and Fire: Book 5 - A Dance With Dragons, Chapter Two (Daenerys I).
Just as additional speculation that ties into the accepted answer, there are more reasons the Unsullied are less able to fight well against the Sons of the Harpy.
In your example of S05E04, we see that the unsullied all continue fighting with spears, despite the fact that they are in close quarters. This is not ideal for them, as they have less room to maneuver and swing such large weapons.
It shows that the Sons of the Harpy are generally using knives and short blades, which are much better for confined spaces as they are less cumbersome in the limited space.
This leads to 3 possibilities:
- The Unsullied are poorly equipped for their job, and aren't carrying round smaller blades for the potential close quarters combat they may face.
- The spear is their favored weapon, and they will choose to fight with it no matter the conditions.
- The Unsullied are not well trained with anything other than spears, and so could not use shorter weapons, as they are used to training to fight in wide open spaces.
I can't imagine Daenarys allowing her soldiers going out with less equipment than they need, and from Mooz's answer they apparently are trained to use short-swords, so it seems they prefer to use spears to fight.
Whichever reason it is that they are using spears, the fact is that they are unable to fight as well as they would like to in open battle, so they were always at a disadvantage.
I think the fact that they were heavily outnumbered, surrounded and in unfavorable conditions for their fighting style, and still managed to kill all of the Sons of the Harpy is a testament to the fact that they are still incredible fighters, it just happened to show them at a moment where they were unlikely to emerge victorious from the confrontation.
The Unsullied had not been truly tested. What battles have we seen them in? Dany was told to "Blood them early" which never happened. Their rep is a sales-pitch at best. These dudes are soft. The first one in the season five opener needed a head rub and comfort before having his throat slit. Greyworm is in love? Meanwhile Ser Barriston slew like 9 guys including several after he was injured! He killed 9 in like 15 seconds going Babe Ruth on them. Theon said Jaime Lannister killed 12 men at the Whispering Woods just working his way to slay Robb Stark.
TV show just sucks when it comes to military reasoning. The Unsullied were not just trained in spear but in shield & sword too. Every unsullied is trained not just in formation fighting but in melee close quarter sword fighting. The fact that many unsullied serve as Guards for Masters and Merchants shows that they must know how to defend themselves & their charges in small group/urban environment otherwise they would soon not be working as Guards if they continuously failed to kill/defeat would-be attackers. Also warring Masters/Merchants would use unsullied for attacks too so plenty of Unsullied would know how to attack/fight in a small unit or as an individual. The Unsullied have spent every waking hour training in their weapons. Some would have died initially being surprised but pretty immediately unsullied would be battle ready. They are trained to fight in units so if always patrolling as a squad of 5 or ten they would be unassailable. Especially if equipped with horns/whistles to call other squads in vicinity. I think they would be very effective policing the city. Also the shield is not just defense it can be used as a weapon as well as Tyrion so violently showed. Also each unsullied has a dagger so I think trained soldiers would beat part-timers like the Mereen Masters. I believe though that the harpies are mercenaries ...it would a wicked twist if the Harpies are one of the sellsword companies doubletiming working for Dany during the day & for the Masters at night.