In Universe: It is never explained, he is never seen with it before and never mentions it, so simply put, we don't know, most likely as others have said its a minor plot hole, but one that can be explained later. think back to other post credit scenes, when we first saw Bucky with his arm in a vice being approached by Cap and Falcon, many people would probably asked how did they capture Bucky in a vice... its the same here. its not part of the main film only a post credit scene so doesn't require explaining in that film
Out of universe: they needed a plot device to bring her back in, so wrote it in without needing back story.
Back to in universe: and be warned theres a lot of speculation coming here...
Its safe to assume he was given it by Captain Marvel, and she would have most likely said call me only if the world is ending and you have no other choice...
In all other films and situations Fury had choices so he didn't use it. but this time, Cap and his team are no where to be found (by Fury at least), neither is Vision, War Machine or Black Widow, and he probably can't call on the King of Wakanda especially when he knows that something is happening over the top of Wakanda as stated by Hill, and Iron man is gone... and then people start to vanish in front of his eyes... well eye.
From his perspective, its the end of the world, and he has no other choice
I wouldn't be surprised however if the post credit scene from captain marvel was something like the below:
Ant man appears in the past to Fury, using the temporal portal things mentioned in Antman and the Wasp, having gone back in time to warn him, if this (events of the snap) starts happening call Captain Marvel immediately!