You need to understand clearly one crucial point.
Only a Jedi who adheres to the 'good' side becomes part of The Force in the event of his death.
A Jedi adherent of the dark side who perishes is truly dead; he cannot survive as a part of The Force. Hence Vader, having adopted the dark side, had no knowledge that a Jedi was capable of surviving beyond death. This is why Kenobi warns him that he has a surprise coming if he strikes Kenobi down.
We learn later, in 'Revenge of the Jedi', that the Emperor has imbued Vader with awesome powers, by courtesy of the Dark Side. It gives tremendous power in life, but there is no existence after death. Vader has learned about the Dark Side from the Emperor, so they are both ignorant of the truth about The Force.
Ultimately, Vader saves Luke from the Emperor, turning against the Dark Side, thereby redeeming himself; hence Anakin is accepted into The Force, and Luke can see him - or his 'ghost' or spirit - become a part of it.
It is interesting to note, from Leia's reactions in that final scene, that only Luke can see Kenobi, Yoda and Anakin. Only another Jedi has the power to communicate with one who has passed into The Force. Others, such as Leia, cannot see or communicate with them.