In the "Annotated Screenplays," Lucas states that: "One of the things that will never get explained in the films is how Ben was able to retain his identity, because it happened somewhere in between the third and fourth movies. I set up that this is a discipline that he learned from Yoda; Yoda told him how to do that. We don't ever get to see how he does it, but the idea of retaining your identity after you've passed on is something that Ben learned as a Jedi." (Annotated Screenplays, p.269)

But, over the course of EPISODES II and III, Lucas has stated that it will be explained why some Jedi (like Obi-Wan, Anakin, and Yoda) disappear at the moment of death, while some Jedi (like Qui-Gon) do not.