In a interview with Seth MacFarlane the same question was made:
Can all the characters hear Stewie when he talks?
The single most asked question about the show. They can, they can hear him and understand him, its just that he's a baby and for that reason they just don't take what he says with any seriousness.
On The Stewie Griffin Wikpedia page they mention a 2011 Comic Con appearance:
MacFarlane went on to say that Brian always hears Stewie, and more recently so does Chris, but the writers usually strive for Peter, Lois, and Meg not to hear him. Once Stewie leaves the house, the question of who can understand him depends very much on the story. MacFarlane also states that these rules can be broken for the sake of comedy, so this could change from one episode to another.
There are more details in Stewie's wiki:
A running gag has occurred over whether or not the family can understand Stewie's speech, especially given his tendency towards homicidal remarks. The Griffins appear to ignore most of what Stewie says, though on occasion they have responded to his speech.
This has been referred in "E. Peterbus Unum" as a meta-joke. The end of the episode reveals it actually being a historical video, projected to a group of students in the future. When the presenter asks if there are any questions, one of the students responds with "I don't get it. So.... like... can the family understand the baby or... what's the deal with that?" The other students are seen nodding in agreement with the apparent confusion.
Brian and Jillian are the only adults that seem to fully understand what he's saying; however there have been occasions when other characters have evidently understood him. Brian can always understand him though, and they often have conversations between themselves, including musical numbers, arguments and bad advice. Strangers such as Connie DiMico can also understand Stewie in "McStroke", as can Jillian. Lauren Conrad is even able to understand him. On the rare occasions he speaks directly to Meg or Chris they also sometimes reply.