Congratulations! It looks like you spotted a subtle clue towards a plot twist originally planned to be revealed in season 2, that was later dropped.
Pycelle was originally imagined by the showrunners (possibly by author GRRM too, it's unconfirmed as far as I know) as a meek careerist toady, someone who maintains his comfortable high-status position by being useful and loyal to those with power and by avoiding letting himself be seen as a threat.
Part of that, as they saw it at least in early seasons, was that he deliberately acted more physically feeble than he really was, in a conscious attempt to appear harmless. They filmed a (later deleted) scene, which was to be aired in season two, where Tywin calls Pycelle out on his act.
As soon as he realises he's been found out, Pycelle stands up straight, drops the stammer from his voice, and admits (rather awkwardly...) that his mannerisms were a phony ploy. It's an extreme transformation - he looks like a different man, suddenly almost as healthy and sturdy as Barristan.
This scene never made the cut. I don't think they explicitly state why, but given that in Season 6...
...he doesn't particularly stand up straight and fight even when being murdered to death by children.
Maybe he lasts a little longer than you might expect for someone so seemingly frail, but he's quickly overwhelmed.
...it's likely they re-imagined or toned down this aspect of the character before releasing the final cut of Season 2. This presumed toning down came well after the scene you refer to in Season 1, so it's likely that when season 1 was filmed, they were still imagining him in this way.