In Ender's Game, Colonel Graff states clearly to Ender that the objective is a war to end all wars. Even though Ender is duped into genocide believing a simulated battle is running, he is actually giving the strategy for genocide. I don't quite understand his sudden remorse. Why didn't he stop the simulation realizing that genocide was not a viable strategy?
Ender has remorse because he doesn't know he is actually fighting the war until the end.
The entire setup of the second part of the movie (where his training is intensified and he assumes he is fighting Mazer Rackham, the hero from the previous formic invasion) is about realistic simulations of formic strategy. That is what Ender and his underlings believe. The "training" exercises grow in intensity driving Ender to his limits and breaking some of his commanders. He reacts in the end with an unthinkable strategy partially because he believes the training is unfair and he is fighting Mazer Rackham. He has had enough and he wants to beat the system and beat the unfair game by doing something ridiculous and over the top. He wants the game to stop because it isn't fair.
His strategy parallels a similar move he used towards the end of the training in the zero gravity training room. The teachers increasingly set the battles there up so Ender's army had to face enemies and physical setups that were totally stacked against them. To win his final battle there, Ender chooses a ridiculous sacrificial strategy allowing him to get through the enemy gate without having to directly challenge all the enemy forces. Nobody else has tried this play before and Ender only tries it because, again, he is tired of the degree to which the game has become unfair.
Some of the subtleties of this are much more apparent in the book than they are in the movie, but the essential elements are still there if a little simplified. The essential point is that Ender has been duped. He always assumes he is training. Graff and the school knew they had to do it this way because Ender's military gift of thinking like his enemy relied on empathy to be effective and that could easily lead to him deciding genocide was too much. As Ender himself said, paraphrasing a little:
The moment when I understand my enemy well enough to defeat him, I also love him.
Besides, Genocide is a viable strategy here. And Ender justifiably feels species level remorse.
On the other hand, genocide is not achieved, a queen survives. Moreover, the formics may have been partially complicit in this as they expressed regret over what they had tried to do to earth when they found a human they could communicate with. And they had been preparing this through Ender's dreams for a long time and had been planning to avoid their complete destruction as a species.