I wondered the exact same thing after I saw this movie. I knew that Melissa McCarthy went through a drastic weight loss recently and I thought that she may have had extra skin removed and she was just hiding that with those high neck scarves and turtlenecks.
This is not the case. Melissa McCarthy envisioned this character and her wardrobe long ago and wardrobe designer Wendy Chuck helped her realize her vision with that character. From a People magazine interview:
Her new experience in the fashion world also inspired the star to get
behind-the-scenes in the wardrobe apartment while filming The Boss.
While character Michelle isn’t quite as in-tune with the ‘less is
more’ approach to style, McCarthy says it was “a blast” to go with
“the more absurd” looks.
She adds of the process, “once we knew who she was, which I’ve known
for a long time, it’s like, you couldn’t fool Michelle. She requires a
certain amount of pomp and circumstance. Nobody wanted to see Michelle
in dockers and a t-shirt.
This wardrobe does not have an out of universe explanation. It is simply Melissa McCarthy's vision for the character.