In the original timeline, when the movie starts and Doc tells Marty to meet him at the mall, he doesn't yet know if the experiment will work. Time travel in Back to the Future works by changing history, and at that point, no one had time traveled yet.
In the new timeline, shown at the end of the movie, it's unknown whether all of the 1985 events from the beginning of the movie still happen--Marty might have driven to school rather than hitching rides, for example--but we do know that Marty meets Doc at the mall, the time travel experiment still happens, and the Libyans still show up. We also know that Doc had the letter (and the vest) in this timeline, though, where he didn't before.
As a result, it's unknown whether Doc in this timeline did the experiment because he knew he "had" to, or just to see if it would work like in the original timeline, or some combination of factors, but he does seem unsurprised to see Marty when he reappears back at the mall.