To summarize my answer:
- Harry's perceived physical presence has to do with the visual medium of TV. The alternative (Dexter thinking to himself) is not as interesting to watch, even though it would be more realistic.
- Notice that when Dexter reminisces about something that (the real) Harry said in the past, that we do get to see a flashback. For things that Harry actually said, Dexter relies on his memory.
- Everything imaginary Harry says is actually what Dexter thinks Harry would say.
- Harry intentionally appears to Dexter. This can be subconscious, e.g. at a time where Dexter doesn't want to talk to Harry but his subconscious does so anyway. But Dexter controls Harry's appearance. Harry is not a sentient being and cannot control when he appears, nor what he says.
- Dexter uses different people (Rudy) if he needs someone to argue a different point (one that Harry would not make).
This delves into some spoilers, though I haven't seen past season... 6 I think (so no spoilers for later seasons).
Harry taught Dexter how to behave. When left to his own devices, Dexter would be an undisciplined mass murderer (this is exactly why Harry teaches Dexter when he's still alive). Harry taught him to control himself and steer his urges.
Harry generally appears at times where Dexter is struggling between what he would do, and what Harry would want him to do. Visually, for the viewer, it's easier to have "ghost Harry" argue his point, rather than Dexter continually think "Harry would say that...", "but I think Harry would disagree", etc.
When Dexter sees Harry, Dexter is thinking like Harry. He labels the thoughts as Harry's, to give that train of thought (which is different from Dexter's personal train of thought) an identity.
In real life, there are people who are capable of having imaginary arguments. I'm one of those people. In such an argument, I always include people I know (because I can estimate their response to a certain topic). To me, in my mind, this looks exactly like how Dexter "sees" Harry.
- If I'm thinking about e.g. buying something expensive I want, and I am currently erring on the side of making the purchase, then I will have an imaginary argument with my grandfather (who is notoriously frugal).
- However, if I were erring on the side of saving my money, I will have an imaginary argument with a specific friend (whose general approach to life is "carpe diem").
- If I am not favoring either option, and I really don't know what to do, I will create an imaginary argument between my grandfather and my friend. (This is a common trope, it's basically the "devil and angel on your shoulder" trope).
Based on my current stance, I engage in an argument with someone who would disagree, but whose opinion I respect. Whenever Dexter sees Harry, Dexter is doing the same.
If I remember correctly, there is a scene where imaginary Harry and imaginary Rudy are discussing something, at a time where Dexter really doesn't know what to do.
Later in the show, Dexter also starts seeing Rudy. And, unsurprisingly, ghost Rudy's opinion on things is what Dexter expects (real) Rudy's opinion would be.
When Dexter met Rudy, he was friendly to Rudy. It took some time for Dexter to realize that Rudy needed to be put down. Before Dexter came to that conclusion, Rudy made sense to Dexter. Dexter disagrees, but understand Rudy's reasoning (and thinks there's some merit to it, even if he ends up disagreeing on the final conclusion).
Dexter understands how Rudy approaches things, well enough that Dexter is able to think like Rudy even after Rudy died.
It's also interesting to see when Rudy started appearing to Dexter. This happened after Dexter started thinking that Harry's way of thinking, while it did serve a purpose for him in the past, is no longer applicable to Dexter's current life situation.
Harry's advice is essentially outdated, and Harry had been appearing less and less to Dexter (and when he appeared, Dexter generally ignored his advice more and more as time went on).
If Dexter doesn't value Harry's input; he may resort to getting feedback from others. This is where Rudy enters the scene. Rudy didn't just appear out of nowhere against Dexter's will, Dexter (subconsciously) invited Rudy into his thoughts.
We later also see another person who does something similar. Travis Marshall sees professor Gellar, in the same way that Dexter sees Harry.
However, there are some key differences here.
- Dexter is aware that Harry is dead, and that the argument is in his head. Travis, however, imagined Gellar to an unhealthy degree. He may have still known that Gellar is dead, but he also responded to ghost Ghellar as if he was a sentient presence. Travis got upset with Gellar for making him do something he didn't want to do, whereas Dexter always knew that he was the one making the decision in the end. There is even a whole episode subplot dedicated to the fact that Travis tries to sabotage Gellar, which makes no sense if Travis was actively aware that Gellar is imaginary.
- Harry keeps Dexter on the right path (compared to senseless murdering). Gellar doesn't serve that same purpose to Travis. Travis simply uses Gellar as a way to shift personal responsibility. He sees Gellar as the real evil, and himself as the right hand to Gellar. Harry represents the good part of Dexter, that which keeps the dark passenger in check. Gellar represents Travis' dark passenger.