Yes, it exists and previous films hinted at it too. From Doctor Strange:
"This universe is only one of an infinite number. Worlds without end. Some benevolent and life-giving. Others filled with malice and hunger. Dark places, where powers older than time lie... ravenous... and waiting. Who are you in this vast multiverse, Mr. Strange?" - Ancient One
Also from the word of god:
Asked whether the ending of Spider-Man: Far From Home means that fans will never see a multiverse in the MCU, Feige told Fandango: "No, it just means [Mysterio] was full of shit.
"I mean, in Doctor Strange, we hear the Ancient One talk about the multiverse, so we've already established it as a thing." - digitalspy
If you want to know all the mention and occurrences of multiverse then the MCU Wiki lists them well.