Basing the cultural impact on viewers has never been an accurate gauge of quality or longevity. The Transformers movies will bring in more viewers but it's not the kind of movie that still gets talked about years later like The Usual Suspects (Domestic Total Gross: $23,341,568) or other critical successes that don't rake in hundreds of millions of dollars.
One of the possible reasons that TWD has more viewers is because it already had a built-in audience (fans of the comic book) in addition to being a really well-made show. Breaking Bad had to get its numbers on its own.
Another factor to consider is that Breaking Bad was a "proper" story; beginning, middle and an end. It especially exploded in media popularity during the last season because people knew it was going to end. The Walking Dead (show) is ongoing with no definite end in sight. The Walking Dead comic (which the show is loosely based on) could go on for another 200 issues.
The last factor I'll mention is believability and societal worries. Breaking Bad can hit too close to home for some segments of the population dealing with the very real threat of addiction and the fallout (crime, prostitution, violence, health issues, etc). The threat of a zombie apocalypse is pretty much zero which makes it easier escapist fare. Meth and drug dealers exist. Zombies (ala TWD) do not. Many people watch TV to escape problems and reality.