I think Vin Diesel was a good choice.
Let’s not forget that much of what an actor does comes in the form of how he says his lines. Most of communication occurs outside of just words. Much of that is in motions and gestures, but much of it is also in intonation, timing, stress, volume, etc. Plus, Vin Diesel has a unique sounding voice that fits the role of a large, powerful tree being.
Some reinforcing points for thought are these. One of the shortcomings of email is you simply receive the words someone says, and lose much of the full meaning coming from the sender. Another example close to home for me is when you talk to your pet. They don’t understand language for the most part, but they understand a lot based on how you speak it.
As far as an A-list actor goes, the handicap of only speaking several actual words means that your full acting talent has to come to bear in order to pull it off. Maybe an A-lister is what is needed here. By the way, Groot also says “We are Groot”.