"Hate" is a big word. Especially in earlier seasons, you can argue that Dwight "hates" Jim more than the other way around, since Dwight is often the instigator.
Dwight plays the role of "office police" when he in fact does not have the authority to do so. Dwight antagonizes people while he's busy trying to "win" the game (in Dwights mind, he is competing for a position of authority).
Dwight consistently behaves as a superior to others, and often introduces (verbal) hostility towards others for no reason other than Dwight wanting to be in charge.
Jim subverts Dwight's alleged authority. This has two consequences:
- Dwight is knocked down a few pegs.
- The people who suffer from Dwight's competitive nature (including Jim himself) get a reprieve by seeing Dwigth knocked down a few pegs.
- As she sits behind him and is therefore usually in shot whenever Dwight is, the most common instance of this is Phyllis smiling whenever Dwight is ridiculed. Dwight is rarely kind to her, and while she's not the type of person to fight back (she's too meek), she appreciates seeing Jim put Dwight down.
So, to answer your question, it would be more accurate to say that Jim hates Dwight's self-aggrandizing behavior more than he hates Dwight himself. However, the two can in some cases overlap, especially in cases where Dwight is constantly antagonistic.