The phrase doesn't seem to reference anyone specifically in Pirates of the Caribbean's canon, as it is to date. It just seems to be a funny anecdote
However, I do think Jack brings this up in conversation for two indirect reasons.

- He is saying this to Henry as warning, recognizing that Herny seems to like Carina, who happens to be a brunette. Jack had been previous arguing with her during the St. Martin bank robbery escape sequences, where he uses her to make his escape.
- Ironically the bank robbery scene opens with a funny reveal of Jack being inside the bank vault in company of the Mayor's wife, Francis. She too is also brunette.
So I think Jack made a little a jump here that makes it all a little more amusing.
That doesn't mean that the line was not also put here to eventually reference someone in particular, as even the line from Curse of the Black Pearl, "Clearly, you've never been to Singapore." didn't make much sense until the third film, At World's End.
As food for thought, there are a few brunette characters from Jack's childhood, but none of them had a sister and a brother.
There is also Angelic introduced as a scornful love interest to Jack Sparrow in On Stranger Tides. The two clearly have history, but not much is know about Angelica's past or if she actually has siblings, given the convent situation? There are some loose ends in that film that could play into future installments, including that the final scene features her stranded on an Island with the Jack Voodoo doll in hand. We have no way yet of knowing if that at all played into the plots of Dead Men Tell No Tales or what happened to Angelica.