YES - They can speak.
For the TV series, a language was invented called Skroth.
In an interview with Entertainment Weekly during Season 2, Executive Producers David Benioff and D.B. Weiss stated that "Skroth" is indeed the official name of the language used by the White Walkers. According to the interview, the inhuman (and unpronounceable) sounds of Skroth, "ice-cracking chatter", were created by Sound Designer Peter Brown.
The language they speak in the books is not named:
The novel A Song of Ice and Fire clearly states that "the Others" ("White Walkers" in the TV series) speak a language with the voice "like the cracking of ice on a winter lake", but the name of the language is not given.