....can Asgardian Gods die in the MCU?
Yes
Frigga died in Thor: A Dark World from being stabbed and she was the wife of Odin.
What their death actually entails has not been clarified although it seems to involve transcendence to another plane of existence (perhaps Valhalla, Hel or possibly Fólkvangr.)
Note. Immortal does not mean unkillable...at least for the Asgardians in the MCU. It just means very long-lived.
Immortality: They usually don't age, but can be killed - although more than likely it will be incredibly difficult and takes a great deal of effort or special item to do so. Sometimes only a god can kill another god. And it's not uncommon for them to be resurrected, either - although they may lose their memories or be forced to take a new form.
TV Tropes