This question already has an answer here:
- Is there a third Holmes sibling? 3 answers
At the end of Sherlock S03E03, Mycroft says to man standing beside him: "Don't be absurd, I am not given to some brotherly compassion. You know what happened to the other-one", and the man seemed convinced with his point.