I would say that, if there had to be one, Cap would be the obvious choice. As others have mentioned, he has a history of leading others from his work before he was encapsulated in the ice. He also is very mission-oriented and very much interested in the greater good at his own personal cost. Whether this makes someone a good leader or not is debatable but when considering the goal of The Avengers they seem to be positives.
Why not the others?
Thor tends to wander off back to Asgard to take care of things there and it's difficult to lead a group when you have multiple priorities. He may have more battle experience but Earth is (arguably) secondary to Asgard.
Banner tends to wander off because he's not in control of the Hulk's power and doesn't want to hurt others, and lacks confidence. He's a great scientific mind but his heart isn't in the fighting.
Stark has more money than god and more brains than Einstein but he's very self-centered. Sometimes it takes a bit of prodding for him to do the "right" thing and his version of what's right isn't always the same as what's conventionally considered "right". Plus, as Fury was the one setting up The Avengers and intentionally tried to keep Stark out, he doesn't seem to be trusted as a leader.
Romanoff is used to working alone. She'll work as part of a team when necessary but her background is as an assassin/spy, a character type prone to working solo. And, yes, I'll play the misogyny card... she's also a girl. Girls don't generally get to lead men in superhero media.
Barton shows a lot of leadership qualities and the team does look to him often, which is shown pretty clearly in Age of Ultron. But he's also very modest and his family is clearly very important to him, so I think he'd avoid the position of leadership even if it was offered to him.
Plus, if you look at the end of Age of Ultron, at who's left for The Avengers team, it's a completely different group of people and the only two from those above are Cap and Romanoff.
Oh, and what about Fury?
Well, I'm pretty sure they make it clear a couple of times that, while SHIELD may have been behind creating the Avengers, they're not going to run the group. Fury has his own things to work on, managing what's left of SHIELD (as discussed in Agents of SHIELD). They'll be there for back-up if needed, as seen in Age of Ultron but they're two separate entities.