There is no mention of which version this is, while they are all mostly the same some of the dialogue is different between the regular movie and the musicals, and the non singing dialogue in the musicals is slightly different in parts. so i'll try to stick to quoting what is sung...
But a lot of the details is in between songs, but i don't have a copy of les mis to type it up, the songs however are easy to find and portray most of it
Valjean gives himself up to save the wrongly accussed prisoner. but immediately asks:
This woman leaves behind a suffering child, and none but me can intercede, in mercy's name, three days is all i need"
As Javert then replies
Men like you can never change
Valjean then escapes custody, proving to Javert that a criminal will always escape and does not care for justice, that they owe nothing to the law or honour, Javert is enraged throughout the song, he was right all along and he is good, on the side of the law, and the law is good, and 24601 is wrong
Then later when he has continued on the side of the Law and the government, and it is always his pride and honour to hunt down those that break the law. he never doubts the path he has chosen as life is exactly as he saw it, good men abide by the law, bad men break it, once a bad man always a bad man
when Valjean releases him, It enrages him once more, there is no grey area like Valjean believes, it is black and white, the law and criminals, as Javert believes to his core
I'll spit his pity right back in his face, There is nothing on earth that we share, It is either Valjean or Javert!
he struggles with this, a bad man, a criminal doing a truly good thing, it throws his entire existence and life into question. life is becoming grey...
He gave me my life, He gave me freedom, I should have perished by his hands
It was his right, It was my right to die as well, Instead I live, But I live in hell
His life is indeed hell if he has been wrong all this time
And my thoughts fly apart, Can this man be believed? Shall his sins be forgiven? Shall his crimes be reprieved?
And must I now begin to doubt, Who never doubted all these years? My heart is stone and still it trembles, The world I have known is lost in shadow
He "never doubted all these years" thats why he didn't take his own life up beforehand.
Up until that point, all he saw was a criminal, even as a "good" mayor, he saw him as a conman profiteering off good people. all the time he was always right, never given reason to doubt that world view, until that fateful day...