When Cap went back in time to an alternate reality where he could live out his life peacefully with Peggy, would he not run into an alternate version of his own self? Or was that alternate version of himself possibly frozen at that time for 70 years and he just filled in for that Cap's reality since he was missing at the end of WWII? If so wouldn't that cause a bit of a buzz in that alternate reality's timeline when the old Cap leaves to gift the shield then a younger Cap suddenly appears in that same timeline but much younger cause he was frozen?
Captain America jumping back in time created an alternate universe. Captain America spent most of the rest of his life and got old in that alternate universe, then jumped back to the main "616" universe.
If so wouldn't that cause a bit of a buzz in that alternate reality's timeline when the old Cap leaves to gift the shield then a younger Cap suddenly appears in that same timeline but much younger cause he was frozen?
According to Joe Russo (one of the Russo brothers, and a co-director of Avengers: Endgame), there indeed would be two Captain Americas in that alternate reality/universe. That would be similar to the case in Endgame where there happened to be two Nebulas (2014 Nebula and main Nebula) in the main universe. As for the "bit of a buzz", it is not known how exactly things turn out in that alternate universe. It is possible that there is no "buzz" at all. Main Cap could've kept a low profile.
Joe Russo explains what happened to Captain America in Endgame in an interview at ent.qq.com (translated from the original Chinese; emphasis mine):
Question: Did Captain America's action at the end affect the timeline? Does that mean there was a time where two CA existed in a same universe?
Joe Russo: To me, CA's action in the end wasn't the fact he wanted to change anything, it's more like [he] has made a choice. He chose to go back to past and lived with the one he loved for the rest of his life. The time travel in this movie created an alternate reality. He lived a completely different life in that world. We don't know how exactly his life turned out, but I'd like to believe he still helped many others when they were needed in that world. Yes, there were two CA in that reality, it's just like what Hulk said, what happened in the past has already happened. If you go back to past, you simply created a new reality. The characters in this movie created new timeline when they went back to the past, but it had no effect to the prime universe. What happened in the past 22 movies was still canon.
Captain America couldn't have gone to the same main "616" timeline/universe, as that would contradict the time travel mechanism used in Endgame (as discussed by the Avengers) and in Loki. Related dialogue (emphasis mine):
Bruce Banner: Changing the past doesn't change the future.
Scott Lang: Look, we go back, we get the stones before Thanos gets them. Thanos doesn't have the stones. Problem solved.
Nebula: That's not how it works.
Bruce Banner: If you travel to the past, that past becomes your future, and your former present becomes the past, which can't now be changed by your new future.
Scott Lang: So Back to the Future is a bunch of bullshit?
As explained by Banner, and as remarked upon by Lang, their actions when they went back to the past (alternative timeline/universe) didn't and won't affect their former present (main timeline/universe). Loki (2021) follows this concept where time traveling simply means traveling to another universe.