I understand that John Reese from Person of Interest was a CIA operative and FBI and CIA operatives wear suits. And CIA operatives are mostly shown wearing the same kind of suit.
Still, he got famous with the name of "Man in the Suit" when Detective Carter started looking for him in season 1. Even some of gangsters, corrupt people like HR and government agencies like the FBI know him as a person who keeps on showing up at locations of crimes and wearing a suit.
Very few like Elias, Finch and others on his team know him by sight which are mostly on his side. Only CIA people are against him who know him by face. Even the FBI was not able to recognize him when they arrested him in one episode as other guys were also in suits.
So why doesn't he get rid of the suit, he can wear any other costume? Especially when he could get rid of all the people except the CIA, like in season 2 when the FBI was looking for him.