Although the other answers may be right when they say that it's a reference to the longstanding Brooklyn/Queens rivalry, as a native New Yorker, I think it is simpler than that.
![enter image description here](https://i.sstatic.net/tisdW.jpg)
In the geography of New York City, Brooklyn and Queens are neighbors. They're both located on the western tip of Long Island.
![enter image description here](https://i.sstatic.net/tu18C.png)
As we can see, Peter1 and Steve2 have lived most of their lives less than 10 miles apart from one another.
It sounds like Steve's reaction to hearing that Peter is from Queens is simply a "Hey, we're neighbors!" kind of thing, not a reference to any deeper significance.
Update: My mother was born and raised in Williamsburg, Brooklyn, and my dad was born and raised in Long Island City, Queens. I ran the scene by them and asked them to explain why Cap chuckles - they both agreed: "Because they're neighbors".
1 For those who are interested, here's 20 Ingram Street, Spider-Man's house. A family named Parker lives there in real life, and their next-door neighbors are the Osbournes
![enter image description here](https://i.sstatic.net/15nNym.jpg)
2 Unlike Spider-Man, Marvel never gave Captain America a real-world address. We know he was born in Red Hook, Brooklyn, but we don't know where. He later moved into an apartment in Brooklyn Heights, but the apartment address - 569 Leaman Place - doesn't actually exist.