Near the start of Jojo Rabbit whilst at the Hitler youth camp, there is a book burning scene. There is a shot where it zooms in on two of the books being burnt in the fire. I didn't recognize the covers or names before the scene moved on. Is there significance to these books, any joke attached to it or are they just fake prop books?
The book in the image in Sanpaco's answer looks like it's "Journal Littéraire" by Paul Léautaud.
You can see the end of the words in the title in the screenshot.
JOURNAL LITTÉRAIRE PAUL LÉAUTAUD
The author was a French writer and theater critic who kept a large diary of his eccentric musings (the journal in question) through much of his life.
The Nazis may have found his literary work to be crass or rude, hence its significance here.
As for jokes, Léautaud would have been the early-20th-century equivalent of a contemporary film critic (and a notorious one at that). Like other directors, perhaps Waititi has a grudge against certain critics?
There aren't any clear shots that allow you to tell which ones they are burning, which leads me to conclude that there wasn't a joke implied as to which books they were. The only joke being that they are having so much fun doing it.
There is only one shot where partial text can be made out, and the only letters shown are "NAL" and "MRE" which could be the end of the author's name or of a word in the title.
The fact that Nazis held book burnings is documented so we can assume, without more specific reference or evidence, that they were burning the same books that were burnt historically.
The Nazi book burnings were a campaign conducted by the German Student Union (the "DSt") to ceremonially burn books in Nazi Germany and Austria in the 1930s. The books targeted for burning were those viewed as being subversive or as representing ideologies opposed to Nazism. These included books written by Jewish, pacifist, religious, liberal, anarchist, socialist, communist, and sexologist authors among others. The first books burned were those of Karl Marx and Karl Kautsky.