The main person that calls and treats Alan as a loser is/was Charlie, and that was primarily because Alan is a moocher/freeloader. That is, even though they are brothers and Charlie would have been happy to help Alan out for a while, Alan took advantage of the generosity and permanently moved in.
Even worse, Alan was cramping Charlie’s lifestyle which of course, Charlie was upset about.
As a result, Charlie constantly complained about Alan in the hopes that Alan would take the hint and or be offended enough to move out.
When Charlie died, Walden bought the house. Alan saved Walden’s life, so Walden felt obligated to help Alan out a bit, but once again Alan took advantage of the generosity and basically moved in (or rather, did not move out). Walden does not insult Alan as much as Charlie did because they are not as close, so Walden feels awkward insulting Alan who he met less than a year ago.
Jake insults Alan because, well, he’s Alan’s teenage son and unfortunately that is typical of many American children.
Alan’s ex-wife Judith makes fun of Alan as well, but since she left him, she presumably has reasons to dislike him. Plus, she has to constantly fight him for alimony and child-payments. She has since re-married Jake is now an adult, but she is likely to continue to hassle him for other, personal reasons.
Finally, his own mother treats him poorly, but then Evelyn treats everybody poorly.