I haven't seen the movie yet, so any answer I guess would be just that - a guess. But here is my hypothesy:
Smell - sadness: Smells can trigger a lot of memories. Perhaps you smell bread baking, and you're taken back to when you were a kid and you and your mom went to the bakery every week to get a fresh loaf of bread. As weird as it sounds, when I get a whiff of mothballs, I think about my grandmother's apartment, because the smell was distinct and she used them in her drawers where she kept her clothing. Not being able to smell anymore would rob you of those memories, which I can imagine would lead to great sadness.
Hearing - agression/anger: This actually makes sense when you think about it. Nine times out of ten, you're pissed at someone because of something someone else told you about them. One good example is how most people find out their partner is cheating - they hear about it from someone else (or they overhear a conversation they shouldn't be hearing).
Vision - love: Again, this also makes sense. We tend to fall in love with our eyes. You can describe a woman like Angelina Jolie in a 1,000 words, but it will not have the same impact on you until you actually see a picture of her. There's also a reason why the expression is "beauty is in the eyes of the beholder."