Sherry Turkle had a few useful insights back in 2011 that people keep forgetting:

1) Humans want the tiniest, tiniest bit of emotion from their gadgets. If they get it, they fill in the rest. Empirically tested, neither kids nor adults can tell the difference between Kismet reacting to their face and Kismet reacting to a random number generator.

2) 80% of communication is non-verbal and 15% of what's left is tone-of-voice. Any online discussion is therefore 95% in-our-own-heads from a context standpoint so no wonder when we can't tell a human from a robot. This is why ELIZA was the fucking future in 1966 and a dystopia by 1971.

3) Humans absolutely lose their shit when the mask slips. Without knowing it. Without knowing why. Turkle consulted heavily on My Real Baby because Hasbro discovered that about one in ten people would take ahold of that cute little bundle of joy and try to bash its brains out against the nearest hard surface. My Real Baby had shock sensors on it to go fucking limp if it experienced too much trauma; prior to that, Hasbro had several destroyed in market research because if the thing kept twitching after you tried to murder it your inner ape went full kill-it-with-fire. Surprise surprise, the chat data includes a heapin' helpin' o'kill-it-with-fire.

This is all so fundamentally fucking stupid. It's a Markov bot. It knows, based on what it has trained on, what is stochastically expected. We will never achieve sentience this way, just a more perfect mirror. Is it entertaining seeing behind the illusion? Absolutely. Is it emblematic that Google would go live with a chatbot that doesn't know when it's wrong and Microsoft would go live with a chatbot that secretly wants to murder you?

Hey Satya - tell me about Clippy! (6:50)

Hey Star Trek - tell me about Clippy!

Wanna see how to break the spell? Feed it into a '90s vintage voice processor.

But that's no fun because you don't get all the navel-gazing of a twittersphere full of narcissists holding a mirror up to their face and talking about seeing God.

I'm old? So i remember this bullshit with Teddy Fucking Ruxpin. People were dragging out ELIZA back then. People think Magic 8 Balls have spirits in them because when all you have is text you don't even need to resort to cold reading. We WANT that connection. That's where pareidolia comes from. Bing arguing with some schlep about what year it is isn't an existential crisis it's a shitty fucking algorithm being sold to us as a revolution.