Gandalf did not know then that it was the One Ring. He simply suspected it. He realised Smeagol had called it that in the past and understood then there was an even deeper magical power behind this than he realised. He suspected it could be the One Ring, but he needed proof and to know for sure.
If you remember, he left the Shire immediately to learn more information. Whilst in the movies this departure is a fairly short one, in the books Gandalf is gone for 17 years, searching for answers.
You may remember in the movies he finds an account of Isildur, buried deep within a library (at Minas Tirith, in Gondor). It states as this script shows:
It has come to me...the ring of power! It shall be an heirloom of my
Kingdom...all those who follow in my bloodline shall be bound to its
fate, for I will risk no hurt to the ring...it is precious to me,
though I buy it with great pain...
The marking upon the band begin to fade...the writing which at first
was as clear as red flame, has all but disappeared...a secret now that
only fire can tell...
Gandalf rushes back to Frodo and has him cast the ring in the fire, as a final test. At first it seems the situation may not be as feared, as the ring remains plain. As Frodo starts to notice the Elvish lettering though, Gandalf's fears are confirmed:
Frodo: It's some form of Elvish...I can't read it.
Gandalf: There are few who can...the language is that of Mordor, which
I will not utter here.
Frodo: Mordor?
Gandalf: In the common tongue it says, "One ring to rule them all, One
ring to find them, One ring to bring them all, and in the darkness
bind them."
It was at that moment that Gandalf knew.
It's also worth pointing out that Gandalf's suspicions were low before this event, as Saruman had led him and all the wizards to believe the ring had been lost (from Tolkien Gateway):
In T.A. 2953, the Wise gathered once more to debate about the Rings of
Power. Saruman quieted the Council claiming to have the knowledge that
the One Ring was lost forever in the Belegaer.
As Saruman was at the time a wise and trusted leader, none of the wizards like Gandalf had reason to doubt him. To give some context to the time, Bilbo's birthday at the start of the Fellowship of the Ring was in 3001 (48 years later).
Furthermore, to delve into the lore of the books a little more, Gandalf started searching for Gollum when he left the Shire in 3001 (with the help of Aragorn). He returned to visit Frodo in 4 year intervals until 3008. He didn't return again until 3018.
Some time between 3009 and 3017 Gollum is captured by Sauron and tortured. He is eventually released in 3017 and is captured by Aragorn in the Dead Marshes. In the same year, Gandalf reads Isildur's scroll.
Finally, in 3018, Gandalf returns to Frodo and the events unfold as you know.