It depends on the show in question.
First of all, apart from the obvious live programs, TV Shows are filmed well in advance, and usually have at least a half-dozen episodes in some stage of post-production at a time. Dramas that are cancelled in this middle of their production season will definitely have leftover episodes. (This is one reason networks prefer to send shows on "hiatus", or let the season end, before cancelling them.)
What happens to those episodes depends on the contract negotiations between the network and the producer. Those contract terms could literally say anything that both parties are willing to agree to, so every show will be slightly different.
However, in a typical scenario, the network orders a fixed number of episodes up-front, before the first one airs, and is thus forced to pay for all of them. If the network chooses to cancel the order prematurely, they may be required to pay for the rest of the initial order, or may not, but generally the production company is only going to finish producing episodes they're getting paid for.
At that point, the network has the option to do whatever they feel is the best business decision for them. Often times they will "burn off" remaining episodes of a program on weekends, using them as filler for otherwise empty hours of programming. This may be put on the schedule immediately, or the network may hold onto the programming until they need it. Since they have purchased the first-run broadcast rights to the episode, they have sole discretion on when, how, and if the episode runs. The only major alternative would be syndicated re-runs, and even then, that assumes the network is willing to sell rebroadcast rights, and that anyone's willing to buy them. These days, getting syndication at anything less than ~4 years worth of episodes is a tough sell.
It's also likely that completed but unaired episodes of a program will show up on a DVD, but that assumes a DVD will be released in the first place. In the case of a program with only two aired episodes, it's likely the network won't even consider it worth the cost to produce DVDs. So odds are, we're probably never going to see the rest of those episodes unless someone's willing to buy the network's broadcast rights from them and air them.