Yes, and most of the time they are killed through physical means. Although highly resistant to the average gunshot, demodogs in season 2 are quite clearly vulnerable to blunt trauma. The "magic" you're referring to, I assume, is Eleven using her powers, but here's the thing, it's not her powers that killed it, it was the force of impact after she swung it around and threw it through a window. It might seem like it's the "magic" doing the work, but take anything capable of hitting them with similar force, and it will kill them just like El killed that one (or possibly more than one that we didn't see). Resistant at times, they can be killed by gunshots, such as when hopper took a stand while trying to defend El as she was closing the gate.
Quite frankly, the only "magic" that killed them throughout the show, was when the gate was sealed and they started dying off due to their connection to the upside down being cut off. As for the demogorgon, although even more resistant to bullets (seemingly bulletproof), it still is vulnerable to blunt force (Steve's bat), and it's also highly vulnerable to burning/fire, which is possibly even lethal if exposed to for long enough. El's abilities may have killed the demogorgon which may qualify as a supernatural death, but it's not exactly confirmed since they both disintegrate and disappear in the exact same way. Only one is shown to re-materialize, and that's Eleven.
Last but not least, even the mind flayer is shown to be susceptible to blunt trauma, again when El swung it around, and so quite possibly it's once again vulnerable to fire (fireworks). Though, I wouldn't count on whether it can be killed that way or not.