From Chapter 2 of the first book:
To the everlasting credit of the people of District 12, not one person
claps. Not even the ones holding betting slips, the ones who are
usually beyond caring. Possibly because they know me from the Hob, or
knew my father, or have encountered Prim, who no one can help loving.
So instead of acknowledging applause, I stand there unmoving while
they take part in the boldest form of dissent they can manage.
Silence. Which says we do not agree. We do not condone. All of this is
Then something unexpected happens. At least, I don’t expect it because
I don’t think of District 12 as a place that cares about me. But a
shift has occurred since I stepped up to take Prim’s place, and now it
seems I have become someone precious. At first one, then another, then
almost every member of the crowd touches the three middle fingers of
their left hand to their lips and holds it out to me. It is an old and
rarely used gesture of our district, occasionally seen at funerals. It
means thanks, it means admiration, it means good-bye to someone you
love. [-Katniss' Narration]
So it's effectively used to say goodbye.
In the first book, when little Rue dies, the following scene takes place:
A few steps into the woods grows a bank of wildflowers. Perhaps they
are really weeds of some sort, but they have blossoms in beautiful
shades of violet and yellow and white. I gather up an armful and come
back to Rue’s side. Slowly, one stem at a time, I decorate her body in
the flowers. Covering the ugly wound. Wreathing her face. Weaving her
hair with bright colors.
They’ll have to show it. Or, even if they choose to turn the cameras
elsewhere at this moment, they’ll have to bring them back when they
collect the bodies and everyone will see her then and know I did it. I
step back and take a last look at Rue. She could really be asleep in
that meadow after all.
“Bye, Rue,” I whisper. I press the three middle fingers of my left
hand against my lips and hold them out in her direction. Then I walk
away without looking back.
So Katniss makes this symbol and as the movie shows, the first riots and protests of the uprising against the Capitol begin.
From then on, the significance of the gesture completely changes and it becomes a symbol of the revolution - effectively, a way for the citizens of the Districts to say goodbye to the overbearing force and power of the Capitol.
The whistle that accompanies it is the noise that the mockingjays make. From the Hunger Games wiki:
A mockingjay is a bird that was created through the mating of
jabberjays and female mockingbirds. When Katniss wore a pin with this
bird on it in the 74th Hunger Games, the Capitol was angry. They
believed it would result in an uprising because the Capitol saw the
mockingjay as a failure. This is why Katniss became "The Mockingjay"
and led the rebels in Mockingjay.