There are a couple of reasons why didn't the Ministry pick him up.
There are many rules in real life, but usually, authorities relax those rules in certain cases like when the deed is not that serious and let the accused go with just a little warning.
The same thing happened here. The Ministry is not gonna punish every kid who practices small spells outside of Hogwarts.
From the script for Prisoner of Azkaban,
HARRY: No thank you. Minister... I don't understand. I broke the law. Underage wizards aren't allowed to use magic at home --
CORNELIUS FUDGE: (dishing up a bowl) Oh, come now, Harry. The Ministry doesn't send people to Azkaban for blowing up their aunts! On the other hand... running away like that... given the state of things... very, very irresponsible.
Sirius Black escaped from Azkaban and it had something to do with Harry Potter. Therefore, it was more necessary to protect Harry than punish him. This is why the Ministry relaxed the rule for him.