Some knowledge of the book may give a greater understanding.
Clearly the plots do not directly follow each other, and the most obvious difference is that in the book, The Grasshopper Lies Heavy is a book, whereas in the series it is a series of motion pictures*. However, if we assume the central premise is the same then
the world portrayed is a fiction, and the films exist to reveal to the characters that they are not real
In such an interpretation, it does not really matter who "the man" is, or what the in-universe motivations are.
On the other hand, I believe there is not intended to be a definitive answer and the audience is free to come to their own conclusions. Your own explanation is perfectly reasonable, but I prefer one based on a multiverse theory of realities, where the films slip through from other realities
like Tagomi does in the final scene
and Hitler is "the man" (his residence is the only thing presented that could qualify as a literal "high castle") collecting them in order to understand the various possibilities and steer history to his design.
*Crucially, it is the same medium in which story is presented to us.