I think Thanos's quote came after the events of Guardians of the Galaxy, at that point he had "lost"1 three Stones because he had others do the work for him:
- The Mind and Space Stones lost because Loki was captured
- The Space Stone was taken back to Asgardians
- The Mind Stone ended up in HYDRA's hands
- The Power Stone lost because Ronan took the Stone for himself, locked away by the Nova Corp.
So "Well, I'll do it myself" seems to be more in reflection that if he was going to get the Stones, he should get them himself instead of relying further on more unreliable lackeys.
Beyond that, while it was his army Loki lead in the first film, I don't believe he had a hand in the events of Age of Ultron beyond having originally owned the Mind Stone to give to Loki, who failed in his mission.
1: I say "lost" in quote marks as in my answer to How does Thanos know where all the Infinity Stones are?, I suggest that Thanos may have orchestrated things so that he could find the stones easier, but that wouldn't mean he had a hand in Age of Ultron, more that "losing" the Mind Stone is a balanced price to getting the Tesseract off Earth rather than expecting the Mind Stone to lead to Ultron's and Vision's creation.