> eavesdropped conversations seem pretty useless unless you also have the whole context and information on the person you're targeting to be able to effectively misuse that data

Keywords (i.e. "perfume", "car", "phone", "notebook", "flowers", whatever) would probably be enough to "improve" targeted ads.

However, this would be a huge scandal if true, so your first suggestion, fingerprinting gone wrong, is more likely.