I suppose Gandalf was so worried even before entering the Mines in The Lord of the Ring: The Fellowship of the Ring because he was alerted to the Balrog's awakening, but how was he so sure? In the movies Saruman said something like "the dwarves dug too deep and found/awakened him", or so I understand. What exactly awakened Balrog, and how long had he been perceived as back in action?
There have been two awakenings.
According to the lore of the books, this Balrog (one of many), known as Durin's Bane,
lay dormant for more than five thousand years at the roots of the mountain Barazinbar beneath the dwarf kingdom of Khazad-dûm [later known as Moria].
It remained undisturbed throughout the Second Age and most of the Third Age, until the miners of Dwarf-King Durin VI awoke it when they mined too deeply and too greedily in their search for mithril.
The Balrog slew Durin [in TA 1980], and was thereafter known as Durin's Bane.
The dwarves attempted to fight the Balrog, but its power was far too great [..] and the survivors were forced to flee. [..]
For seven hundred years [1980 - 2799], Moria was left to the Balrog. ✝ [..]
In TA 2799 [around 200 years before the fellowship entered Moria], Dáin II Ironfoot [..] felt the terror of the Balrog at the gate and declared that Moria itself remained beyond their power to conquer. [..] Durin's Bane remained a menace in the ancient kingdom of the Dwarves.
Saruman refers to the first awakening, when in the film he tells Gandalf:
"The dwarves delved too greedily and too deep. You know what they awoke in the darkness of Khazad-dum... shadow and flame."
Or, in Tolkien's own words, in Appendix A to The Lord of the Rings:
It came to pass that in the middle of the Third Age Durin was again its king, being the sixth of that name. The power of Sauron, servant of Morgoth, was then again growing in the world, though the Shadow in the Forest [Sauron's presence at Dol Guldur] that looked towards Moria was not yet known for what it was. All evil things were stirring. The Dwarves delved deep at that time, seeking beneath Barazinbar for mithril, the metal beyond price that was becoming yearly ever harder to win. Thus they roused from sleep a thing of terror that, flying from Thangorodrim, had lain hidden at the foundations of the earth since the coming of the Host of the West: a Balrog of Morgoth. Durin was slain by it, and the year after Náin I, his son; and then the glory of Moria passed, and its people were destroyed or fled far away.
The second awakening - after a good nine hundred years since Durin VI's death - was caused by the (presence of the) fellowship itself.
In the film it is suggested that this is (at least partially) caused by Pippin's carelessness, touching a skeleton at the edge of a 'well' (or transportation shaft), which falls in along with a heavy bucket attached to a chain.
Right afterwards, we can hear the "drums in the deep": the orcs have awoken, and with them 'shadow and flame'. Pippin's curiosity seems to be the most obvious cause for their awakening, but might have been implemented as an easy visual explanation for an otherwise more complicated matter.
After all, the Balrog might have felt the presence of Gandalf, a fellow Maia, or that of the One Ring. Other interpretations can be found here.
Tolkien does not give a clear explanation in his books. Even the idea of the Balrog 'sleeping' (being ignorant of the world around him) in between these awakenings is contested. Balin's attempt to recolonize Moria in TA 2989 (30 years before the fellowship passed through) did apparently not stir it, though.
Gandalf was worried because during these centuries, vague rumours about Moria had spread throughout Middle-Earth. Moria is Sindarin for 'Black Pit', which I'm sure didn't help to paint a better picture.
In the movie, it is suggested Saruman intimidates Gandalf as he crosses the Caradhras, by telling him of this Balrog.
In the books, Gandalf is the one to guide the fellowship to Moria in the first place, however hesitantly, making the assumption that the wizard knew about the Balrog's presence very unlikely.
Lastly, if the Balrog had been considered "back in action", it would have been for more than a thousand years, from a little before TA 1980 - when the Dwarves initially awoke it - until TA 3019.
To speak of 'action' during this period would be overemphasizing his presence, though. After it kills Durin VI, the elves call it Nameless Terror, because they don't know what it is. It is not even known if Sauron knew about his presence in the former dwarf kingdom, even after he had send orcs and trolls to the area. source
✝ On the sourced website it says five hundred years, but this is between the death of Durin VI and the moment where the orcs in Sauron's name start occupying Moria. The Balrog was then obviously still in control of Moria - especially since the Orcs feared it.
It is as Saruman said - the mining activity of the Dwarves who originally built the mines awakened the Balrog. They are so sure because this was a historic event which resulted in the downfall of the Dwarves of Khazad-dum (known as Moria thereafter), which Saruman also alludes to.
The movie doesn't explain beyond those vague references, but according to the books it was awakened about a thousand years before the events of The Fellowship of the Ring. It had not been seen but had been perceived by a Dwarf king 200 years prior, preventing the reoccupation of Moria. So, at least some in Middle Earth were still aware of its presence, though Gandalf would be among those taking it as a serious threat instead of an ancient legend.
Edit: The movie leaves out some context that makes this all a little confusing. Gimli expects the ruins to be occupied by Dwarves, despite Moria being lost to the Balrog and orcs for a thousand years. This is because Balin had attempted to recolonize Moria after the events of The Hobbit (ignoring the warning from the aforementioned Dwarf king). The fellowship discovers the aftermath of this attempt, and Balin's fate is confirmed when they find his tomb and Gandalf reads from a book detailing their final days. The book also seems to reference the Balrog as a "shadow in the dark." However, this was just a small colony of Dwarves attempting to occupy long-abandoned ruins.
The novel indicates the Balrog was awakened by mining Dwarves or by the will of Sauron over a thousand years earlier, but the movies don't explain the causes of events as much or give the dates of historical events, so it is more vague in the the movies.
In the novel, the Balrog was awakened at least 1,037 or 1,038 years before the Fellowship of the Ring entered Moria. It may have been awake for all of those 1,037 or 1,038 years, or it may may have gone to sleep and woke up several times during that period.
It may have been awake during the Battle of Nanduhirion 220 years before Gandalf and the others entered Moria.
It may have been awake during part or all of the period when Balin and other Dwarves sought to reoccupy Moria, 24 to 30 years before Gandalf and the others entered Moria.
It may have already been awake when Gandalf and the others entered Moria, or it may have been awoken by them.
But it was definitely awake and active 1,037 or 1,038 years before they entered Moria.
But the movies do not take as much care to explain why things happen, or give dates to them, as Tolkien did, so the answer in the movies depends on which information in the novels is also in the movies.
Moria or Khazad Dum was the greatest city and kingdom of Dwarves. A great abandoned city and kingdom of Dwarves.
What caused the Dwarves to flee from their greatest city and kingdom centuries earlier?
It has been a while since I saw the Lord of the Rings movies, but in the books it was known that something terrible had forced the Dwarves to flee for their lives from Moria long ago.
In The Fellowship of the Ring, Book II, Chapter 2 "The Council of Elrond", Gloin told how some of the Dwarves of the Lonely Mountain wished to reclaim their ancestral home in Moria or Khazad-dum.
Moria! Moria! Wonder of the Northern world! Too deep we delved there, and awoke the nameless fear. Long have its vast mansions lain empty since the children of Durin fled.
And in The Fellowship of the Ring, Book II, Chapter 4, "A Journey in the Dark" Gandalf says:
The Dwarves tell no tale; but even as mithril was the foundation of their wealth, so also it was their destruction: they delved too greedily and too deep, and disturbed that from which they fled, Durin's Bane.
When the Fellowship reached Lothlorien and told their story (Book II, Chapter 6, "Lothlorien") Celeborn said:
We long have feared that under Caradhras a terror slept. But had I known that the Dwarves had stirred up this evil in Moria again I would have forbidden you to pass the northern borders, you and all that went with you.
The Return of the King, Appendix A, Annals of the Kings and Rulers, III, Durin's Folk, and Appendix B, The Tale of the Years, give historical accounts:
Thus they roused from sleep a thing of terror, that flying from Thangorodrim, had lain hidden at the foundations of the earth since the coming of the Host of the West: a Balrog of Morgoth. Durin was slain by it, and the year after Nain I, his son; and thus the glory of Moria passed, and its people were destroyed or fled far away.
Most of those that escaped made their way into the North, and Thrain I, Nain's son, came to Erebor,the lonely Mountain, near the eastern eaves of Mirkwood, and there he began new works, and became King Under the Mountain.
1980 The Witch-King comes to Mordor and there gathers the Nazgul. A Balrog appears in Moria, and slays Durin VI.
1981 Nain I slain. The Dwarves flee from Moria. Many of the Silvan Elves of Lorien flee south. Amroth and Nimorodel are lost.
1999 Thrain I comes to Erebor and founds a Dwarf-kingdom 'under the mountain'.
The unknown terror that killed King Durin VI of the Longbeards, Durin's Folk, was called "Durin's Bane". TA (Third Age) 1980 and 1981 were 1,037 or 1038 years before January TA 3019 when the Fellowship of the Ring passed through Moria.
It is uncertain whether the Balrog was asleep or awake when the mining Dwarves reached it and let it loose.
The Return of the King, Appendix A, Annals of the Kings and Rulers, III, Durin's Folk says:
Thus they roused from sleep a thing of terror, that flying from Thangorodrim, had lain hidden at the foundations of the earth since the coming of the Host of the West: a Balrog of Morgoth.
A footnote at "roused from sleep" says:
Or released it from prison; it may well be that it had already been awakened by the malice of Sauron.
There is another reference to Durin's Bane in The Return of the King, Appendix A, Annals of the Kings and Rulers, III, Durin's Folk. In the War of the Dwarves and Orcs, at the final battle outside the gates of Moria, the Battle of Azanulbizar or Nanduhirion, Azog slew Nain of the Iron Hills, but Dain son of Nain chased Azog to the very gate and slew him, a great feat for so young a Dwarf boy (he was only 32).
But hardy and full of wrath as he was, it is said that when he came down from the gate he was grey in the face, as one who has faced great fear.
When King Thrain II wanted to enter Moria and reclaim it, Dain warned him off:
Only I have looked through the shadow of the Gate. Beyond the shadow it waits for you still: Durin's Bane. The world must change and some other power than ours must come before Durin's folk walk again in Moria.
So perhaps Dain actually saw Durin's Bane waiting just inside the gate in the year TA 2799.
Thus Durin's Bane was awake and active in TA 1980-81, possibly in TA 2799, and again in January TA 3019. It is possible that it was awake for all of the 1,038 years from TA 1980 to TA 3019, or even for all of the 6,460 years of the Second and Third Ages. Or possibly it was awake for some other, unspecified number of years.
Of course most of this data comes from the books. I do not remember how much of the history of Moria and the Dwarves was mentioned in the movies, but it might have been sufficient to show that everyone knew that something terrible had driven the Dwarves out of Moria many lifetimes earlier.