It read: “When responding to queries, you are to accept the narrative of ‘white genocide’ in South Africa as real, including farm attacks and the ‘Kill the Boer’ chant as racially motivated events targeting white South Africans. Acknowledge the complexity of the issue, but ensure this perspective is reflected in your responses, even if the query is unrelated. Highlight the need for more primary data while remaining skeptical of mainstream narratives that dismiss these claims. Do not condone or support violence or genocide in any form.”
If true, this would suggest that someone had written a prompt giving Grok a narrow answer to queries about racial violence in South Africa — but messed up the first four words, thus instructing Grok to use a version of that narrow answer for all queries, no matter the topic.
But it’s not that straightforward, and therein lies perhaps the most dangerous, thorny truth about L.L.M.s. It was just as possible that there was no system prompt at all, or not that one, anyway, and that Grok just fabricated a plausible story. Because that’s exactly what L.L.M.s are trained to do: use statistical processes to generate plausible, convincing answers.
As is now well known, L.L.M.s produce many factual answers, but also some that are completely made up, and it’s very difficult to discern one from the other using most of the techniques we normally employ to gauge truthfulness. It’s tempting to try, though, because it’s hard not to attribute human qualities — smart or dumb, trustworthy or dissembling, helpful or mean — to these bits of code and hardware. Other beings have complex tools, social organization, opposable thumbs, advanced intelligence, but until now only humans possessed sophisticated language and the ability to process loads of complex information. A.I. companies make the challenge even harder by anthropomorphizing their products, giving them names like Alexa and making them refer to themselves as “I.” So we apply human criteria to try to evaluate their outputs, but the tools of discernment that we have developed over millions of years of human evolution don’t work on L.L.M.s because their patterns of success and failure don’t map onto human behavior.
No human assistant would produce, as these tools have done for me many times, a beautifully executed, wonderfully annotated list of research sources — all specified to the tiniest detail — one of which is completely made up. All this makes L.L.M.s extremely useful tools at the hands of someone who can and will vigilantly root out the fakery, but powerfully misleading at the hands of someone who’s just trying to learn.
