For a perfect clone, down to the quantum level, both would be the real Angier. That is the essence of his fear. The real Angier will be killed every time.
When you download a movie what you are doing is actually making a perfect copy of the movie. Which is the real movie? They all are. This is very different from making copies of film reels where only the original have the quality of the original (each copy adds a little imperfection).
Now, the question The Prestige asks is what if you can make a perfect clone of a person the way you can perfectly copy digital data? In one sense, if we don't want to get too uncomfortable we can always say the remaining copy is now the real data and the deleted copy is not real anymore. But if you really think about it and get out of your comfort zone you will realize that both people are real and what you have done is murdered yourself.
In The Prestige this point is much more clearly made as the old copied Angier becomes the one who is killed. Not the newly created clone. Every time the clone will have only the memory of the surviving Angier. But every time Angier experiences death, only the Angier that experienced death has no way of passing that memory to the surviving Angier so the survivor can never know what dying feels like.