Tony knows the Quantum Realm can be reached; the problem is getting back out again.
A short overview.
He was completely willing to believe the Quantum Realm exists and Ant-Man has been there; his objection was that, even though Ant-Man survived his visit, it was "a million[?] in one chance". The reason Scott was able to time-travel five years into the future was because non-quantum beings (Hank Pym in 2018, Rizzo in 2023) were guiding/controlling his path. Scott could not have found his own way out.
Essentially, Scott tells the team he's figured out that wood floats and he wants to try using that to get from Europe to North America. Tony is justifiably skeptical; he knows the team will be able to find "someone" smart enough to make a big boat, but they still don't have much of a chance without sail, mast, and compass. This is why Tony spends most of the rest of Act I working on his "Time GPS". At first, he doesn't believe it's even possible to make one, but he sets his AIs to work on the problem just in case.
Wikipedia is our friend.
As a layman myself, it kinda looks like some of the terms being thrown around are just to sound fancy/sciencey; I'm pretty sure I remember hearing him say "Deutch Proposition" too, but according to Wikipedia it's actually called the "Deutch Prescription", and, essentially, it just means they're more-or-less trying to follow Ringworld's sci-fi time-travel rules.
Pop-sci also seems to have a tendency to fixate on "paradoxes" (which are attempts to show by theoretical counter-example how an idea doesn't make sense) rather than the ideas the paradoxes describe. In this case, the "EPR Paradox" was intended to demonstrate that certain aspects of quantum mechanics seemed to imply that information could travel faster than light, and in our normal physical perception of reality that's just not possible. Later experiments and theories, such as Bell's theorem, showed that it isn't a paradox at all if we just consider "quantum" stuff to obey different rules from non-quantum stuff.
Marvel puts magic and (lay) theoretical physics together
There's at least one YouTube video attempting to explain this in further detail in lay terms, but to try to summarize: the basic problem Tony's describing is that, for the Time Heist to succeed, they need to be able to travel to specific locations in the space-time continuum, as opposed to arbitrarily-random ones.
I'm not certain I'm remembering correctly that Tony used the term "million-in-one"; he may have used a higher number. Either way, he's speaking idiomatically rather than literally, because the theoretical chances of survival under this model literally converge on the infinitesimal. If we accept the magical parts of the story, like Pym particles being able to shrink or grow anything to any size, including sizes below the Planck scale, then actually doing so would force the observer to obey the laws of the quantum model rather than the standard one. I'm confident that I'm over-simplifying an over-simplification here, but basically, once you enter the Quantum Realm you are everywhere and nowhere, at every time. (Depending on whether or not some kind of "many-worlds interpretation" is also in play, you could also "be" in every universe.) To get back out of the Quantum Realm, you have to know where and when you're going; the number of bad places/times vastly outweigh the number of good places/times, and our heroes have a limited supply of Pym particles so they can't just guess-and-check.
