Summary:
Delay between original trilogy and prequel trilogy
- Lucas' marriage ending resulting in a potential lack of finances, a possible lack of interest in producing sequels, inability of technology of era to reflect what Lucas wanted to achieve.
Delay between prequel trilogy and new/sequel trilogy
- Lucas not wanting to pursue sequels, Lucas selling to Disney.
Long Answer:
We can break down the series in to three sections:
- Original Trilogy
- Prequel Trilogy
- New/Sequel Trilogy
Lucas wrote in a preface to Splinter of the Mind's Eye:
It wasn't long after I began writing Star Wars that I realized the
story was more than a single film could hold. As the saga of the
Skywalkers and Jedi Knights unfolded, I began to see it as a tale that
could take at least nine films to tell—three trilogies—and I realized,
in making my way through the back story and after story, that I was
really setting out to write the middle story.
This explains why the original trilogy takes place halfway through the overall Star Wars storyline.
Return of the Jedi was released in 1983, but Lucas was divorced in 1987. A number of online sites reference a few sources (including The Secret History of Star Wars) that claim the settlement from this divorce left Lucas without a lot of his fortune, and with little desire to develop his next Star Wars film, he had cancelled his originally planned new/sequel trilogy.
Whether this is true or not is debateable. For example, in 2004 Lucas gave an interview where he said:
"This was never planned as a nine-episode work," Lucas said. "The
media [pounced when] I made an offhand comment, 'It might be fun to
come back when everyone's 80 and do another one of these.' But I
never had any intention of doing that."
However, although Lucas states he wasn't interested in a sequel trilogy, he had spoken of a prequel trilogy, but didn't want to pursue this due to the technological limitations of the time - as technology developed, he gained more interest in the prequels as technology advanced and he felt he could realise his dreams. For example, in this article, he comments:
I never thought I’d do the Star Wars prequels, because there was no
real way I could get Yoda to fight. There was no way I could go over
Coruscant, this giant city-planet. But once you had digital, there was
no end to what you could do.
Due to the enhanced interest, he began work on the prequel trilogy and the films were released from 1999 - 2005.
Even in 2008, Lucas said:
I get asked all the time, 'What happens after Return of the Jedi?,'
and there really is no answer for that. The movies were the story of
Anakin Skywalker and Luke Skywalker, and when Luke saves the galaxy
and redeems his father, that's where that story ends."
However, in October 2012 Star Wars was sold to Disney. They immediately recognised the huge monetary potentials for the films and began work on a new trilogy immediately, with the first instalment releasing in late 2015.
Loss of revenue:
I will point out that the Star Wars Universe (before Disney purchased the franchise) extended far beyond the films. The canonicity of the universe was as follows:
G (George Lucas) canon is absolute canon. This category
includes the final releases of the seven films, the novelizations of
the films, the radio dramas based on the films, the film scripts, and
any material found in any other source (published or not) that comes
directly from George Lucas himself. G canon outranks all other forms
of canon.
T (Television) canon, which comprises Star Wars: The Clone
Wars and Star Wars: Rebels. This level of canon is considered to take
precedence over C canon (see below), possibly due to the fact that
George Lucas is directly involved with these shows. This level does
not include any series before (including the Genndy Tartakovsky's
Clone Wars series).
C (continuity) canon refers to the main body of EU
work, and is the next most authoritative level of canon. All
literature material published under the Star Wars label that doesn't
fall into either G, T, S, or N canon is C canon and is considered
authoritative as long as it isn't contradicted by G or T canon. S
(secondary) canon refers to older, less accurate, or less coherent EU
works, which are immediately overwritten by anything in the main
continuity of G and C canon, but are fully canon whenever they do not
contradict something of higher canon.
N continuity material is also
known as "non-canon" or "non-continuity" material. Lego Star Wars,
Disney Infinity, Star Wars Land, "what-if" stories (such as those
published under the Infinities label) and anything else that cannot at
all fit into continuity is placed into this category. N canon is the
only level that is truly non-canon.
Across these various levels of canon, there was substantial revenue generated in the form of books, video games and TV series.
So whilst the films had a huge gap which resulted in a potential loss of revenue, revenue did continue to flow given the huge sales of Star Wars related products (and arguably helped the overall revenue of the franchise as the market wasn't flooded with new movies).