I would like you to know that it really really sucks how you add everybody to my address book who I only sent one mail to, ever. That clogs the address book and depending on what kind of message it was, after just about 30 seconds I neither care nor remember what I wrote them an email about once in my lifetime.
Imagine me writing an email to some company's customer service. I get an answer from a representative asking me to provide some more information. I reply and attach the needed infos. You helpfully add this person to my address book so I can remember every customer service representative that I ever had to deal with, just in case I ever need to email them personally again. Thank you so much!
Let alone all these random people on craigslist who use a gmail address who you add to my instant messenger automatically, so they can start chatting with me or at the very least see me being online for the next 25 years.
A one-click option to add somebody to your address book is a great idea. Automatically adding everybody to my address bucket (that mess is not a book anymore) however is a bad idea.