Right now there are many people struggling on Earth to get clean drinking water, so water scarcity is a real world thing that certainly wouldn't be any better for spacefarers.
One problem with water that is on the Earth is that it's very expensive to send it into space. That's analogous to the problem of moving water around on the Earth. Sure there's plenty of total water on the Earth right now, but most of it would require a lot of expensive processing to make it drinkable, and even then, it would take expensive transportation to get it to everyone who needs it.
Even if the cost of transporting the water were feasible, then there's the time. Without a large existing fleet of water tankers, it would be hard to get water to the whole solar system fast enough. Meaning, it could take years to get a reasonable amount of water to the moons of Jupiter.
Shipping water from Earth only delays the problem. The more water you ship, the more comfortable things get for spacefarers and they will live longer and have more children, which means you have to ship even more water and now you're just accelerating the pace of the depletion of the Earth's water, and permanently removing water from the Earth's water cycle. So shipping water from the Earth is a very bad idea.
One more note about shipping water from Earth: It's very likely that not everyone on Earth would agree that shipping water is better than interplanetary war. If I remember correctly, there are warmongering factions on Earth. In human history, many people have seen wars as opportunities to gain land or economic advantage. World War I wasn't exactly caused by the assassination of Archduke Ferdinand, it was more about several European countries wanting a war and the assassination gave them the pretext they needed.
Extracting water from "air" requires first that the air actually have both hydrogen and oxygen in it, and while we can assume that life support atmosphere has oxygen, it wouldn't make a lot of sense to go through the trouble to put hydrogen in it. Also, the oxygen in the life support atmosphere is there so the humans can breathe it, and oxygen is much more precious than water, in the short term. Finally, the process for turning gaseous hydrogen and oxygen into water is called "burning", and fires are generally bad in space. A controlled fire could be one way to get some water from gaseous hydrogen and oxygen, but that's hard to do and again you're using up valuable gaseous oxygen (and where's the hydrogen from? Fuel? Delta-v is pretty valuable in space, also).
They probably are recycling all the water they can, but as the population grows, the need for water grows, so there has to be some net influx of water. Also, some water is lost to space and other places. It could be bound up in manufactured products, for example.
Two interesting books that deal with the problems of water in space are The Moon Is A Harsh Mistress by Robert Heinlein, and The Martian by Andy Weir. The former discusses the challenges of shipping water between the moon and the Earth, while the latter actually features a process of converting fuel to gaseous hydrogen and then gaseous hydrogen and oxygen into water in a closed system.
As PoloHoleSet has commented, there's an in-universe explanation for this here: https://scifi.stackexchange.com/questions/111942/why-is-water-so-rare-in-the-expanse
To summarize Valorum's Sci Fi Stack answer:
- Overpopulation/population growth, as alluded to in my comments above
- Destruction of an expected water shipment has caused a short-term crisis
- Political tension is inhibiting the water trade
- Some water is being wasted
Basically, water issues in space in The Expanse mirror the real-world water issues that exist on the Earth today.