Quite simple: Midnight Cowboy is not a pornographic film. What we have here is a misunderstanding of the original intent of the X rating, which gave rise to the NC-17 rating.
Several films from the late 1960s through the 1980s had the X rating applied, including A Clockwork Orange, Beyond the Valley of the Dolls, Fritz the Cat (an animated feature), Last Tango in Paris, and the original The Evil Dead.
However, the key thing about the old X rating is that it wasn't trademarked by the MPAA, and could thus be self-applied to any film. Basically if a studio knew their movie would be deemed unsuitable for children, they could simply self-apply the X rating themselves without submitting it to the MPAA for rating, and that would be that. However, a movie could still be submitted to the MPAA for rating and be given a rating of X.
Well, as pornography became more accepted through the later half of the 20th century, pornographic film studios began to self-apply the X rating when releasing their films, and then began the trend of applying XX or even XXX ratings to really emphasize the amount of gratuitous sex in their films. This led to problems with any film that used the X rating being seen as pornographic, and led to issues for films that were submitted to the MPAA only to be given an X rating without actually being pornographic (because the MPAA would never apply any rating to a pornographic film to begin with). In late 1989 to early 1990, two critically acclaimed art films were denied ratings by the MPAA: Henry: Portrait of a Serial Killer, and The Cook, the Thief, His Wife & Her Lover. This caused an uproar about films of this nature having limited distribution routes due to the ambiguity and misunderstanding of the X rating, as well as these two films in particular being given limited distribution routes due to what many felt was a flawed and misunderstood rating system.
The solution was to implement the NC-17 rating, which was first applied to the film Henry & June, but unfortunately many entertainment outlets began to refuse to showcase even NC-17 films, entertainment publications would refuse to advertise them, and many retail stores refused to stock and sell them, because they saw the NC-17 rating as a rebranding of the X rating. Today the NC-17 rating is essentially seen as a mark of death for many films, causing many production companies to cut down on the material the MPAA found to be offensive (if they can even get them to divulge such information in the first place) to earn it an R rating, often referred to as "Hard R" by the media and film-goers. Originally NC-17 was intended for anyone age 17 and older (16 and younger weren't allowed in, even with an adult present), but in the mid-90s was adjusted to mean no admittance to anyone 17 and under, only adding to the misunderstanding of the NC-17 rating being a rebrand of the X rating.
You can read about the history of the MPAA rating system, as well as the history of the X rating in the United States itself.