The 'Fidelity Issue' has been a long term fixture on many (if not most) film/production and screenwriting qualifications.
During my Degree we had an entire module named 'Adaptation', and for three weeks we discussed/researched this very question without verifiable success.
It's unlikely you'll find a satisfactory answer to something so broad on here, but there will always be multiple factors that will force script doctoring and rewrites, drops and pickups to screenplays:
Books are able to describe events in a non-linear fashion, without having to use such cumbersome pro-filmic techniques like flashback and voice over. Sometimes these can be implied through cinematic effects, but often it is simply impossible to include everything (including back-story and context) within 90 minutes or so.
There is a popular adage within the film industry and mentioned in Peter Biskind's Easy Riders, Raging Bulls, which is a rather gossipy but thoroughly entertaining insiders view to the New Hollywood era of film making...Basically, 2001: A Space Odyssey is considered the high water mark stretching narrative time. It's jokingly said that because the movie packed so much in (the birth of humanity through to its psychotropic implosion[?!]), that afterwards people would say; "If you can't tell your story in 90 minutes, don't bother".
Some Novels are better left as novels, and as such fail entirely when turned into movies. Often it can be accredited to lack of fidelity to the original source material, but increasingly often failure is accredited to too much fidelity, that the film wasn't allowed to exist as a film because it was too closely tied to the original text.
Whilst I thoroughly enjoyed it, Watchmen was both reviled and celebrated for this very aspect of its production.
Scott Pilgrim was an attempt at remediation that actively relished the opportunity to engage with its source material, frequently using animated sequences and importing game-style graphics into the effect sequences to signify it's source text and inspirations.
Production Issues (Internal)
Some films struggle to find sufficient funding or credible effects to complete the 'total vision' of a story that is described in a book, and as such entire sequences will be omitted because they are considered 'too costly' or unfeasible. If anyone knows any specific, interesting examples of this feel free to edit in...
Production Issues (External)
Sometimes real world events can make the subject of a movie seem inappropriate, and as such they are edited out of final cut to avoid controversy. Whilst not an adaptation, promotional material for the 2002 Spider-Man movie famously had a sequence where Spider-Man used his webbing to suspend a helicopter between the two World Trade Center towers. After they were destroyed in the 9/11 terrorist attacks, this sequence was cut from promotional material and the film. The inclusion of comic book adaptation, whilst not explicitly part of your question, are obviously some of the more prolific discussions at the moment.
The Hunger Games reportedly had a sequences removed because they were considered too graphic for the rating the film was trying to achieve, despite the events being specifically mentioned in the books they were simply not marketable to put in a film with a 12 certificate. Although The film is should be commended for side stepping these as creatively as possible with camera angles.
Many people believe that the difference between cinema and book should be celebrated, and they should be respected as different works of art occupying different mediums.
Just like Impressionist painters, who would take what was in front of them and interpret the material into something entirely different. Both the beauty of what is in front of them and the beauty of what they have created are equally valid.
There is something called 'The categorical approach' greatly discussed by narrative theorist Kamilla Elliot, which;
'Emphasizes the unbreakable link (some would say the identity) between form and content, placing visual and verbal arts into emphatically distinct categories and calling on artists to recognize the limitations of their media and to strive to work within these boundaries.'
There are so many more aspects to this, its a big argument with plenty
of avenues of discussion: hopefully people will contribute more...
In the meantime, Spike Jonze's movie Adapation does
sterling job of exploring the very question you've asked here, albeit