It's not just that I can conceive a universe where I want LLMs to be useful, it's that I and many normies are using it as a tool in basic ways and that there are a bunch of more advanced ways to use it that feel just out of reach. I genuinely don't believe I'll use these tools less in two years than I do today. So for me it's not in the "maybe it'll be useful someday" realm, it's already useful to me now and would be a lot more useful if the ways to use these models weren't so godawful half of the time. I have in the past weeks used LLMs to: - write a working, hosted, version controlled and github-linked web app in less than an hour that pulls data from APIs and displays them in a custom way (which I had spent multiple afternoons on before, not getting even 20% as far) - help me figure out how to write complex Excel formulas - write the first draft of a Request for Information in such a way that I could skip over the easy questions and hone in on the hard, important questions to ask suppliers - assist me in figuring out how to diagnose filament extrusion problems I had with my 3D printer - give me spoiler-free yet highly specific information on a video game I wanted to play - explain a difficult concept to me in multiple ways from a field I don't know but just want the Pareto 80/20 gist of it - do boring-ass restructuring of a large .txt tabulated file - replace at least 30% of all Google searches because Google's search results are already garbage, the SEO slop is already most of it and I can avoid it with an LLM (which feels like garbage on top of garbage but it does the job) For me, LLMs are mind-blowing when I see a completely novel way of arranging the tool and its inputs, when it opens a new avenue of use cases for me, like how recently multi-agent stacking has got me thinking about what else I can do. That's different from if I can ever use it in that way. I've never found a personal use for NFTs, but I can conceive of a world where they are useful to me. The "I guess we're doing AI now" is exactly the same as "I guess we're doing NFTs now" was then; soulless corps gonna soulless corp. But just because there is a hype cycle around it all, does not necessarily mean it will recede back into obscurity like NFTs and nerd helmets did. It won't change the world but it's already had a larger impact on my day to day work than crypto has (and we've been talking use cases for a decade on that). How's that not worth something?
I figured out last night that LLMs are Dunning-Kruger machines - "If it meets or exceeds my expectations it meets or exceeds everyone's expectations because I am smarter and more accomplished than most people." And here you are, privileging your "give me spoiler-free yet highly specific information on a video game I wanted to play" above the WSJ's "phone in a best man speech" by couching it in terms of "power users." Do you remember the early objections to AI? The argument was "bu bu bu professionals will use this to make shitty art to sell to the rest of us" followed by "bu bu bu amateurs will use this to make shitty art to starve professionals". We're now at "bu bu bu amateurs can't tell the difference what do we need professionals for?" Do you remember when it was interesting? Two solid years of talented amateurs and professionals going "can we make this interesting" and we're rallying around "I recognize this brand language as Batman-adjacent." The whole point of a "pareto 80/20 gist of it" is to get enough comprehension to know when you need to go deeper. We used to accomplish that with a search. Or asking a friend. Or wandering into a store. "Me at the Zoo" was uploaded in 2005. Within ten years, your best bet to change a belt on your washing machine was to find someone who had filmed themselves changing a washing machine belt. Within fifteen years, your best bet was to find a five-year-old video of someone changing a belt because the SEO bots had flooded the zone with bullshit links to purchase parts. Nowhere in there was Youtube ascribed a "world-changing" dynamic. Your every use-case is some form of "reduce friction between my ignorance and my knowledge by a nominally incremental amount." The Dunning-Kruger machine in effect: LLMs allow non-experts enough expertise to satisfy other non-experts. Sure - you have a few other menial-bullshit tasks up there but ultimately, your every question up there is some form of "Hey Alexa" and "Hey Alexa" has only lost Amazon $25b so far. Your big outlier is "replace Google search because search is garbage now" which, again, is why Microsoft invested in OpenAI. Can't use search anymore because there's so much AI-generated garbage; best use AI instead! Ultimately, the question is does this tool do anything well enough to pay for itself and the answer seems to be "it probably will someday because mumblemumblemumble." Your sanctioned use cases are "make Excel suck less" "do a web search for me" and "generate some boilerplate." Maybe "multi-stacking" will allow you to do cooler shit! Who knows! The future's so bright we gotta wear shades! Meanwhile the cryptids have been banished in favor of this shit: Did you know that I've been beta-testing Final Draft for almost 20 years? There was a documentary that never came out in which a spokesman for Final Draft said that they'd go under if it weren't for amateurs who will never make it. It should have come out in 2007 but who the fuck is going to buy a documentary about amateur screenwriting? Truth of the matter is you can kludge a Word macro into formatting screenplays but FD makes it hella quicker. Of course as soon as the whole world became superhero movies the amateur screenwriting market collapsed. The semi-pro field would pay $200 to write screenplays that would get blown off by Warner Brothers but that door is now so shut that all that's left is dudes spending too much effort on their unwatched Youtube videos. Thus, "draw me some fuckin' dragons since no one will ever see this shit anyway." Final Draft? Yeah it got scooped up by Entertainment Partners. Why? The lady who ran the beta-test (who brought me on!) ended up going to the accounting firm and when FD was about to go Chapter 7 Warner Brothers leaned on EP to buy it so their working screenwriters still had a solution. Any feckless wannabe who is still trying hard enough to want to do it right is now being underwritten by Warner Brothers. I've been beta-testing Pro Tools for almost as long. It was never a semi-pro solution; you don't buy Pro Tools unless you expect to make your money back. Problem is so much of music now goes through Spotify and Bandcamp that why bother with the pros? Avid is supported in no small part by Disney because holy shit if Media Composer goes down y'all will have nothing to fucking watch. Meanwhile, here's Microsoft, "reinventing music". You? you're no amateur! You're a "power user!" Surely there's an AI use-case for you! NFTs are being used every day by the Arnault Group, by Richemont, by Sheseido and by dozens of other major conglomerates to control black- and gray-market distribution. That was the point. The side-show that was Bored Apes was never the point. The point was authentication. You don't deal with authentication, so there's no use-case as far as you're concerned. Not your fault. Nobody was talking about authentication except the people who need authentication and as a group, they went "yay massive drop in breakage." And for the life of me, I can't recall a single Super Bowl commercial about how NFTs were going to change the world, even though they already are.
I've been thinking about this over the weekend and I can't put my finger on exactly where (or if) I disagree, so I'm gonna write to sharpen my thinking, if you'll humour me. Maybe the biggest disagreement is that I don't understand what classifies as world-changing to you. How can you both believe NFTs are already changing the world when the largest use case is improving the bottom line of specific luxury brands through destroying their grey market, yet not seeing LLMs make a dent in the universe? I mean - are LLMs world-changing to the tune of the $200B put into it or to NVDIA's market cap? Hell no. Is AGI around the corner? Also no. Are LLMs world-changing at all? Well, I'd put it at 'somewhere between YouTube and smartphones'. They'll make a bunch of things worse, they'll make a bunch of things better, like any powerful tool. But because it's a tool that can so directly influence the core of knowledge work, I find it very hard to believe it won't change things at all like you seem to suggest. Whether that means the big corporations are making smart decisions about investments AI is an exercise left to the reader. It's not that I'm not aware that LLMs are, to a large and arguably frightening degree, a bullshit machine. You know how much I hate Tesla's """self-driving""" for the exact same reason. I will not trust LLM output for anything with any serious consequences, just like I will never have Musk take the wheel so I can nap. But as a co-intelligence? Microsoft was at least somewhat on point by naming it Copilot. The copilot can do useful things but I'm still piloting this thing, I'm still making the decisions to the degree that I want to make them. The guy behind NotebookLM makes the point that the useful thing about LLMs isn't just the model, but also its ability to combine that with a context window that is getting so large it's exceeding what most of us are capable of. It can see a needle in a haystack but, more importantly, it can see the entire haystack. Hallucinations are much rarer when the text is right there, so it's also markedly more accurate and can cite shit. I was discussing LLMs with my FIL the other day. He has been writing reports on construction failures for almost three decades now. All his work, and that of his colleagues going back five decades, is digitized into an archive of searchable text. But he said it's hard to use in his desk research because he can only search on keywords and those keywords change a lot over time. My FIL also has a hard time getting started with a new report - not because he doesn't know his shit, but because he's not the best at structuring his thoughts into a logical, linearly ordered line of reasoning. I'm 100% sure his company would pay a lot for a suite of LLM tools to enable him and his colleagues to supercharge their desk research. Not to replace their expertise but to enhance it, by having an LLM surface decades of knowledge in a way that isn't possible now, combining it with relevant information in a case, and then also helping them make use of it. Now multiply this use case for all academics and knowledge workers around. We still need professionals in the loop, just like I still need to be in control in a Tesla. We need expertise to tell bullshit from fact. That doesn't mean LLMs are doomed to be useless toys - even when I, too, use it as a toy now from time to time.
"World-changing" is "change the world." It's a new way of doing things that is so successful over old ways that the old ways are abandoned. Refrigeration changed groceries. Air conditioning changed geography. Here's an argument about how cell phones changed Africa, and how AI probably won't. Your argument boils down to "provides better search for my father-in-law" which, okay, hire him an assistant. What's that? An assistant is too expensive? Are you sure? Oh, I see. A research assistant won't be as fast. Okay, so now we're talking about incremental productivity gains. Look. I'm so old I remember when women were secretaries. Watch a few scenes of Mad Men, that shit was real. Then typewriters were electric and... women were still secretaries. Then word processors came out and women suddenly had more time to do shit other than type. Slowly but surely you started to see an integration of women into the workplace, you started to see a rise in daycare standards, you started to see double-income families as the norm. The modern western world owes its existence to the word processor in many ways but I doubt you'd argue that word processors are world-changing. They're an incremental tool that was one of many aspects of computerization that led to the information age. "Incremental productivity gains" for whom? The most reasonable argument is that LLMs might make that part of your job that you hate less arduous. Okay, great. That's a good thing. fuck yeah Wordstar. My mother used to compose tests for her biology students with a typewriter, a pair of scissors and a copy machine. She did it that way well past the point that word processors existed because she needed diagrams and diagrams in desktop publishing took a dozen years longer so word processors basically bypassed her but for a big chunk of academia they revolutionized things. They didn't give anyone any more free time, though, because the job is the job. If anything, word processors annihilated the mimeograph industry - you poor bastards will never know that particular smell of fresh purple ink and for that i feel sorry for you. The real matter, however, is that word processing was equally useful to amateurs and professionals alike. i can write like hell and even I lean on spell-check. My daughter basically taught herself to spell by guessing at ways to get rid of the squiggly red line when she typed. Yay word-processing. But if you're a shitty writer and I'm a great writer, LLMs will allow you to crank out mediocre work nobody wants to read while it won't do a damn thing for me because I can crap out better stuff than it can without pausing to sip my coffee. Oh, but that's gonna save the bad writers hours of time. Okay? But who cares? If nobody wanted to read it anyway why does it exist? Bloomberg has had AIs writing finance stories since 2018; that's because the articles are all written for sentiment bots doing high frequency trading anyway and it doesn't fucking matter. Because luxury goods are used to evade tariffs, embargoes and sanctions, as well as to provide untraceable bribes. Here's Imran Khan, going to prison over watches. Here's the government of Angola, falling over watches. Here's Wired, arguing in 2020 that this shit is about to be over and here's Rolex, hopping on the blockchain. "Improving the bottom line of specific luxury brands" is one thing. "locking off a major portion of the shadow economy" is quite another. LLMs - make it easier for mediocre writers to churn out copy, make it easier for mediocre coders to churn out programming NFTs - make it harder to bribe governments Has that sharpened your thinking? How can you both believe NFTs are already changing the world when the largest use case is improving the bottom line of specific luxury brands through destroying their grey market, yet not seeing LLMs make a dent in the universe?