SlashFilm suggested the following connection:
The title [of the episode] is a direct reference to the song “A Horse
with No Name,” by the band America, and Walt is shown singing the
lyrics at the ep’s beginning and end. On both occasions he’s blindly
visited by two threats in the clashing forms of law and chaos. It’s a
great use of music because, for one, the song’s lyrics famously don’t
make a lot of sense and yet the track creates a manly sense of
importance and dignity in being lost and alone. The song is also
thought to be about escapism via drugs (specifically heroin), and Walt
is still high, albeit from the power of capitalizing on other
people’s.
Unfortunately, Cracked did an article investigating this idea of the song being about drug escapism and stated:
Let's save time here by going straight to Dewey Bunnell, the man who
actually wrote the song:
"I wanted to capture the imagery of the desert, because I was sitting
in this room in England, and it was rainy."
So, back when he was a kid, Dewey was playing around in the desert,
found it interesting and years later wrote a song about it with a
message about the environment. No heroin-induced hallucinations or
allegorical desert, but real, actual desert.
Despite their disappoint reveal, I think the Slash Film reviewer's thoughts are probably the best explanation of any deeper connection that exists.
My own feeling is simply that the song is just a random choice, as it is quite light hearted in tune and provides an interesting contrast to the extreme peril that Walt is actually in, at both the start and end of the episode.