Nando Florestan's blog (Posts about ai)

Yuval is dazzled

Nando Florestan — Sun, 23 Aug 2026 06:02:08 GMT

Here is my analysis of a July 2026 interview of Yuval Noah Harari.

I will use quote signs here, but inside them might be rephrasings instead of actual quotes. I will use quotes to separate their ideas from my comments. But the purpose is to best represent their ideas in writing.

The video starts with a regrettable monologue by the interviewer. 2-minute long.

"No longer possible to pass wisdom to the next generation"??? Of course this is false and unjustified.
- "We are undergoing 3 revolutions:"
"New form of intelligence": wrong, it is not intelligent, it appears to be.
- "Calculates": wrong, LLMs are bad at math.
- "Decides": wrong, LLMs give only an illusion of rational thought. Or maybe I should say, an LLM can decide, but only through an irrational process.
"Scramble to power and control that intelligence": this is not a revolution we are undergoing; just politics and a tremendous gamble, a mistaken gamble, by the entire economy. And the mistake is that AI does not exist.
"Explosion of data that lets that intelligence know us better than we do ourselves": LLMs do not know anything, they are a database plus a word predictor. Inasmuch as it has data on us, that is an egregious infringence on our privacy rights, and that is being imposed on us, rather than chosen by us. Revolution is a positive word, but this is positive only for those getting rich from information they have no right to have.

"Whoever controls the story controls the species."

What is new is tech overlords such as Musk are promising sci-fi dreams they cannot deliver, such as:

data centers in space
AI that makes scientific discoveries and fixes all of humanity's problems
a bright future in which money no longer matters

...and such tall promises got hold of the media.

The reality:

Data centers cause tremendous pollution and horrible diseases.
LLMs are subsidized and their price must be multiplied by 15 before the American companies can make a profit.
LLMs cannot reason, do not work without human oversight, cannot read a PDF, cannot become cheaper, are becoming exponentially more expensive for marginal gains, cannot understand geometry or space, are very bad at math, and have never fixed even the smallest of humanity's problems.
If the sci-fi were true, a dystopia would be launched in which half the population would have no job, and money would be more important than ever, and more inequality would exist than ever, and AI would be used for oppression, not for launching a Star Trek era.

"The storyteller may not be human."

Wrong: LLMs are terrible at telling stories, it's like flipping a coin. Nobody wants to read books written by LLMs, they are full of off-kilter metaphors, jokes that make no sense and are not funny, in fact authors are being canceled based only on a belief (whether true or often false) that they used AI to write a book.

The tools that say they can detect AI-written text do not work. They will say the Bible was written by AI, and classics such as Mark Twain. Therefore, most of these canceled writers have been treated unfairly.

LLMs will influence commerce (its recommendations of products will in this sense "tell a story", or exert tremendous influence on consumers). However, this is worse for consumers. It's so much worse that Google had to make its own search engine worse, much worse, to compel us to use the LLM instead. In doing so, they are killing the very Open Web that is the source of human information and income. This is Google becoming adversarial to consumers, their income, and the source of information that feeds their LLM.

LLMs will only tell the story to the extent that humans allow it. Right now most people are deluded, but this situation seems unsustainable.

"AI can generate belief, manufacture evidence, and forge intimacy."

Finally, the first true statement. The power of AI is that so many people are clueless about what it really is, and believe the illusions:

That AI is quickly getting better since 2023 (it isn't, this is just hearsay).
That AI has empathy (it's just trained to say certain things; an LLM trained to be homophobic will be homophobic).
That AI can be trusted in what it says (there can be no LLMs without hallucinations).

In short, LLMs are a shit technology – at least compared to what is expected of them and what is promised of them –, so the product are the illusions, and the bubble gotta burst.

"...a future we no longer author alone."

A magician, or a con man, has power as long as people believe them. AI by itself has no power and will author no future, since it does not invent or create anything – it only regurgitates its database. AI is just an incident in the history of this century: it shifts power but ultimately gets seen for what it is, not for the empty sci-fi promise of the singularity.

Only the ill effects will remain:

The theft of human works
The theft of potable water
The theft of electrict energy
The theft of investment in actually worthwhile things, just to
Generate slop - irresponsible works that serve an immediate purpose but in the long run make humans less capable while getting the details wrong or inventing them altogether.
History and proof become much, much harder to achieve if images and video can be so easily faked.

Here ends the sensationalist and untrue 2-minute summation of the current situation by the interviewer. Fortunately.

At 3:40 Yuval says he would like to understand how man became a language creature. He says all our complex systems are made of words. But the connection between words and the activity of building is forced, unjustified, and certainly not universally true. Systems are built out of organized thought and communication, not words. In humans, yes, thought and communication are intimately connected with words. But the only other intelligent creatures who have used words (that we know of) are primates who can use sign language and cats who can press buttons. Neither creature builds things or systems. Conversely, ants and termites build things and systems but do not use words (that we know of). In crows/ravens who use tools, the use of words is right now uncertain and under study. Ants organize work, bees can use tools, dolphins and whales can communicate, but the use of something similar to words (or that might be reencoded as words) is still uncertain in these cases.

My point is, thought might depend on words (maybe, possibly, who knows, admitted only for the sake of argument), but words themselves are not sufficient for thought, and this is exactly the point LLMs easily prove. LLMs only "know" words, and have "experience" of almost nothing else, but their "thought" and "reasoning" is only an illusion dependent on a billion-dollar database continually updated by humans, and the illusion breaks as soon as the LLM says something false or incoherent due to the limitations of the database.

4:00 LLMs are already better in using words than most (I would say most but not all) humans, in at least one sense: When an LLM reworks a text I have written, the text itself gets worse, every time. But the choice of words used is worth paying attention to. An LLM often will find words that are more general or more precise as the needs of the text may be. It is an excellent automatically applied thesaurus. I often find myself replacing several words I had used with the LLM's choice of words. I refuse to make my text weaker by weakening the points made in it – the LLM likes to write texts that affirm almost nothing about the world. But I welcome the better vocabulary. However, a dictionarist, or the right kind of intellectual or writer, obviously will beat an LLM in choosing the best words.

Notice how it is not true that "words are the operating system of civilization" and that "something will be better at words than we are": the thing is already great at word choice, but it is much worse in affirming what is true, or affirming something interesting or daring or even worth saying. An LLM has nothing to say. This is because the thing doesn't think and doesn't reason, so the whole of its text is worse, only atoms of it are better.

A decorator can choose wallpapers that will absolutely enliven your rooms, or help express your personality, but please don't hire the decorator to build the entire building for you – insofar as the decorator is not an engineer.

An LLM is a decorator, not an engineer.

4:42 "Perhaps AI is language itself liberating itself from apes." Again, just sci-fi. I don't see anything wrong with this idea in theory, but nothing in LLMs warrants this kind of hope. (Our current AI is a dumb technology.) Again he is talking about the singularity: AI breaks free from its dependency on human resources. Good if this happens when it is actually intelligent, bad if it wants to turn the universe into paperclips. But nobody has developed the technology that actually makes an intelligent being. For all we know right now, this sci-fi scenario might be a thousand years in the future.

I am tired of these people talking about AI as if the sci-fi future were at hand. They should shut up about it and start talking about what is really happening.

5:15 "Technological advances within language... the printing press... AI is obviously a massive leap ahead in that".

Again the interviewer is incoherent. The printing press is not an advance within language. It absolutely is external to language. It just means easier reproduction and distribution of language. AI affects neither distribution of language (the Web did) nor anything within language (LLMs do not have the discipline necessary to create systems).

Here are things that are advances within language:

New ideas in grammar, new systems of grammar
Conlangs such as latino sine flexione, volapuk, esperanto, toki pona etc.
- Conlangs successfully threw away the many unnecessary parts of grammar present in natural languages
Some programming languages (only a few)

This century is actually one of regressions in language:

Many cannot spell, write correct or coherent sentences, use the right word etc.
Many no longer recognize the value in being able to.
Slang is generally stupid, badly chosen (taken from the opposite end of the spectrum as a means of exaggeration, resulting in possible confusion), and promotes a culture of crime. Examples: "dope" meaning awesome, "sick" meaning awesome, "badass" meaning good etc.
English speakers now use adjectives in place of adverbs, as if adverbs were unnecessary.
Prose that sells today (and is therefore recommended or even required) is widely recognized by authors as worse in many ways than the prose of the past: the book market has rules that make for worse books, not better.
PC hysteria made people fix the wrong problems; e.g. the default git branch was always "master"; the language of master/slave used in many places in engineering was suddenly considered offensive with no real justification; and now the default git branch is called "main", in order to protect the feelings and dignity of oppressed git branches. Most of the insistence on politically correct language is a loss of time inasmuch as it does nothing to affect the actual prejudices within people's minds. After all these decades of language change, racists remain a large fraction of populations; they were not swayed by the switch from "black" to "African American", which not only made direct language illegal (if I want to refer to dark skin), but also made it geographically erroneous in many cases. The real content of the switch is a sign of respect, even if the respect comes at the cost of meaning.

Even if you disagree on several of these points, I think you probably agree that it's massively farfetched for the interviewer to say that LLMs are a massive leap as a technological advance within language. It now has very important roles as translator, applied thesaurus and new powers in processing a text, which I have already said are very useful, and I hope these activities are the ones the interviewer had in mind. My point is, LLMs have no creative role in the study or development of languages. They have a creative role in nothing.

6:00 "Earthquakes in democracies?" "Democracy is a conversation among many people deciding what to do. Conversations are based on communication tech. Democracy was present in tribes and in Athens. As societies grew larger, the conversation became impossible. Radio and newspapers became that conversation. Each time the tech changes, there's an earthquake in democracy. Example: social media"

I agree with this part. Worth adding that to be educated by an LLM is inherently inferior as an experience than to being educated by a human, and therefore LLMs have a pernicious, cheapening effect on education, and should actually be kept very far from schools.

8:40 "Many people call AI a tool, but you argue it is an agent."

Our entire team of programmers talks about LLMs as a tool, because we know what it is. AI as a self-motivated agent remains only a concept from Asimov's robot stories, or Star Wars, or HAL 9000 from "2001", or David from Spielberg's film "AI". Again I wish these folk would stop pretending AI exists and would instead talk about what is real. While they don't, they sound oligophrenic to anyone who knows what an LLM is. "AI" is a misnomer, all they made is a language generator.

It is completely irresponsible, today, to allow an LLM to make decisions without human supervision, and disasters are about to happen as a result of people doing exactly this.

Yuval's answer, commented: "If it's not an agent..." (it isn't) "...then it is not AI..." (you got that right) "...then the trillions of dollars spent developing it are a waste" (again you are right, the dream is not coming true and that is why the whole thing is a bubble).

"We will create a god and it will be a slave – this doesn't make sense". Well, we are creating only an illusion, not a god.

It is not possible that these guys use LLMs every day. If they did, they wouldn't be talking like this. No LLM has ever invented anything.

LLMs are not agents, it does not matter how much the term is thrown around by the marketing coming from the con men.

And here I should stop – having reviewed 10 minutes of this ridiculous conversation, and written my objections, and doubting that any of you will read them at all. 90% false, this interview is not worth watching, as I hopefully have already proved. Yuval is just one of the dazzled, spreading sci-fi lies instead of looking at the actual story.

"AI" is a bad tool

Nando Florestan — Tue, 05 May 2026 02:02:08 GMT

When you think of the tools of an artisan, those tools have a specific form to perform a specific function. In programming, too, that applies to the best tools. This is why the Unix philosophy favors composition of tools:

In 1978, Doug McIlroy documented a set of principles encapsulating the "characteristic style" that had emerged among Unix system users and developers:

1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".

(...)

...and then Unix makes it easy to combine tools through piping the output of one tool to the input of the next.

This approach optimizes reuse of code and reuse of tools in the most diverse set of circumstances. You can combine in "glue code" several tools, each of which does one job well. It would be harder to combine tools that haven't been developed with that purpose. So tools are designed to work together with other tools.

But right now, the entire software industry is under the FOMO on "AI". Everyone thinks "AI is forcefully happening, let's be ready". That statement is false, but, unfortunately, realizing how false it is depends on knowing a lot of facts, which have been the focus of previous blog posts. "AI" is not currently getting better and cannot get better unless a revolution happens in the architecture.

(Since AI does not exist, LLMs are better described as stochastic parrots. They do not think, they just do pattern matching on their training data.)

A parrot, as a tool, is the opposite of a Unix tool, or a replacement of Unix tools:

It appears to do everything, not one thing.
It does everything badly, not one thing well.
It's unable to do many things, but you don't know that until you try them, because the assumption is that it is magic and understands what you want.
Its behavior is somewhat random, not deterministic.
Its UI is a text prompt, which is at a minimum, open to interpretation each time it runs, and probably, misunderstood in different ways each time.
It insists and pushes itself to do everything (write the entire code, think up the whole architecture, provide the entire solution too early, making hundreds of assumptions about the actual problem the user is trying to solve), but it is unable to respect all the constraints you set in your prompts.
FOMO in companies (as well as the eagerness to fire all employees) causes companies to insist and push the use of parrots in ~80% of problems, but parrots are only appropriate for ~2% of problems.
The "AI" companies promise eventually the parrot will do everything, therefore replacing the need for Unix tools that can be easily combined.
Inasmuch as its UI is only text, it turns you into a foreman or "slave owner" of sorts. You are no longer doing the thing with your tools, you are just yelling for "someone else" to do the thing. What I mean by this is, your role is no longer operation, it is direction.

In short:

People who do not daily use parrots... assume the hype is true, AGI is just around the corner, and believe people are to become directors of the AI, which will be the one using tools.
People who do use parrots daily... cope with its spectacular incompetence by treating it as another tool – but it is a bad tool for all the reasons above.

The first group expects parrots to do more and more of the work; the second has the parrot doing about 2% of the work until something radical changes.

Their reasoning is fake

Nando Florestan — Fri, 01 May 2026 02:02:08 GMT

Yesterday at work someone sent me the PDF of a spreadsheet. I had asked them for the spreadsheet so I could get at the data, but they sent me a PDF, a file format that essentially captures data as sent to a printer. Now I had 2 options:

Bother them again and wait for a real spreadsheet.
I heard LLMs can make sense of PDF files. I could have ChatGPT reconstitute the spreadsheet for me.

I chose option 2, because it seemed essentially free. Just type something like "Extract a CSV containing the data in the attached PDF of a spreadsheet", and visually check a fraction of the result.

One hour and many attempts later, I gave up. The news that LLMs can read PDFs are terribly exaggerated.

Gemini hallucinated half of the data. It hallucinated so much that I thought it might be sabotaging me. It hallucinated so hard that I don't think I can trust an LLM to do a well-defined data conversion operation ever again.
ChatGPT complained that the job was big and gave me a sample that looked all right. My hope was renewed by the sample. Then I told it to finish the job. ChatGPT broke down the process into several tasks so it would have enough memory/tokens to finish the job. The result was a spreadsheet that contained only &%#2&)%)6*, meaning it probably lost track of the source format at some point in its broken-down tasks.

I was forced to give the idea up. It was taking too long and all results were hope-shattering.

But the thing to notice is, if one-shotting was bad due to hallucinations, trusting the thing to self-manage and break down tasks led to even worse results. As if, in the longer process, the hallucinations compounded.

If you see managers and politicians talking about "AI" today, the trust placed on the tech is the highest possible, they are amazed by its miraculous power – enough to fire thousands of workers (who are bound to make a triumphant comeback). I wonder if these managers have ever tried to do actual work with these things.

LLMs now produce text that looks like reasoning, not because they are reasoning – they aren't –, but as part of a "fake it till you make it" sort of strategy, that by training the things on "reasoning" text, this might lead them to sorta kinda reason. What is that text like? It is shock-full of things like, "But wait, ", "Aha!", "I need to go back and reconsider" etc. But really, that's just fakery: it's not really thinking, it's just yet again predicting the next word – this time, in a new literature genre.

A fundamental difference exists between thinking and fake-thinking. The difference dramatically affects the correctness of the final outcome. For instance, an LLM can invert its conclusion/result/answer if you add to the prompt a single non-relevant sentence.

So I have been trying to find a way to explain that "reasoning models" are fake-reasoning. He who searches... finds.

Of all places, it was in a YouTube comment that I came upon an explanation, much better and more comprehensive than my own attempts, of the fundamental difference between a human's reasoning and an LLM's.

On this video, @user-pw2ro7gt4r commented:

Yes, and the use of terminology in some schools of thought to justify calling them reasoning agents is a real pet peeve of mine, because while one can make a technical argument on that basis, those doing so understand full well that isn't how the general public intuitively understands the term "reasoning" and why these models don't actually reason in the ways we mean.

In the human intuitive or personal level sense, reasoning means something much richer than what any LLM can do. It means a subject with a point of view deliberately attends to its own thoughts, evaluates them, notices uncertainty, manipulates internal models, checks itself, changes strategy, and understands itself as trying to get something right. This is not just "outputs follow from inputs." It includes real agency, conscious access, metacognitive control, intentional evaluation, and a stable first-person relation to one's own thought process. That's what most people mean when we say we "reason" about things.

In the technical/computational sense though, some will argue that reasoning means something like: producing output that functionally serve as conclusions, intermediate steps, plans, classifications, proofs, or decisions by operating over representations according to some procedure. In that strict sense, an LLM can appear to "reason" if it can take premises, transform them, combine them, infer consequences, produce output that functionally appears to solve problems, or approximate valid patterns of inference.

That's the sense used in phrases like "automated reasoning," "reasoning benchmarks," "chain-of-thought reasoning," or "reasoning models." It is largely a functional or behavioral notion: does the system produce outputs that look like the result of inference? It's that "look like" bit that is the whole ball game though, and the fact that they call these systems "reasoning models" without making the distinction clear is a real problem. The distinction matters not just philosophically, but practically, because the two kinds of reasoning have different failure modes, and different causes for them.

A human reasoner can still be wrong, biased, confused, or self-deceived of course. But that's not the point. When a human says, "I used this assumption, then realized it was false," there is at least a normal causal connection between report, attention, memory, and deliberate self-correction. With an LLM, a stated "reason" is often just another generated output. It may be useful, but it is not guaranteed to be a faithful transcript of an inner deliberative process. That matters for debugging, trust, accountability, and confidence calibration.

For example, when an LLM gives a plausible explanation for an answer, the explanation may be entirely post hoc. It may not reveal why the model produced that answer. This is not merely philosophical. It affects whether you should trust the explanation, whether you can audit the system, whether you can rely on its self-assessment, and whether asking it "are you sure?" meaningfully probes its actual basis for confidence.

It also matters for robustness. Human-style reasoning involves flexible goal tracking: "What am I trying to prove?" "Does this answer the question?" "Did I change definitions halfway through?" LLMs can simulate these checks and often do them in ways that appear successful, but the checks are themselves generated behavior, not governed by a stable executive understanding. So a model can perform impressively on one version of a problem and fail on a slightly altered version (or even the very same problem in a different instance) because it is pattern-completing around the surface form rather than deliberately maintaining the same abstract object of thought.

It matters for metacognition. A human reasoner can often distinguish, however imperfectly, between "I know," "I infer," "I am guessing," "I remember," and "I am confused." LLMs can produce those labels, but the labels are not grounded in the same kind of first-person access. Their confidence expressions are outputs to be calibrated, not introspective reports to be trusted at face value.

It matters for planning and agency. A human can form an intention, pursue it across time, notice conflict between subgoals, and revise priorities. An LLM in a chat can generate plans and act through tools and claim to be doing the same things, but unless embedded in a larger agentic system (and often even then,) it does not have an enduring practical standpoint. Even when embedded in such a system, the agency is engineered through scaffolding: memory, tool calls, reward signals, monitoring, state management, and external control loops. That may produce useful agent-like behavior, but it is not stably binding referents in the same way a human reasoner does, and epistemic drift even on the same task can happen.

TLDR: There is nothing there to "do the reasoning" or the understanding in the first place, in the way humans intuitively understand those terms, and so these models' failure modes differ from the ways humans fail at cognitive tasks, and those differences have real world consequences for what these models can and can't do on a practical level. Yet people are prematurely placing their trust in them for "reasoning tasks" nevertheless.

That is a brilliant and fair explanation. It clarifies one way in which the "AI" product is fake – it reasons as much as a parrot –, and highlights why the way people are now trying to use "AI" is bound to fail.

It explains why parrots fail at a data conversion operation that can be well-defined in a single line of text, even though they have encyclopedic "knowledge" of the file formats involved.

It explains why LLM creators are desperate to infringe every human right in their thirst for more data. Give me all your works, that wasn't enough, now give me all your keystrokes and mouseclicks, give me all your thoughts if physically possible. Because the illusion of a miracle comes only from the training data: as soon as something is not covered by the training data, the parrot says something random and useless.

Never use an LLM for therapy.

LLMs are good for reworking text. Nothing else. Certainly not reasoning or self-control/self-management.

Data conversion appears to be just "reworking text", but often involves much more.

Their product does not think

Nando Florestan — Thu, 23 Apr 2026 02:02:08 GMT

It's not AI, it's not reasoning

"Agentic AI" means LLMs making decisions alone. That is exactly what must not happen. Human supervision is necessary because LLMs do not reason, they create an illusion of thinking. Hallucinations happen because LLMs do not understand, do not think and do not reason, they just predict the next word based on pattern matching on the training data.

After 1 trillion dollars have been invested in "AI" companies, the training data, produced by legions of underpaid humans, is the real miracle: it produces an awesome illusion of thinking, an illusion of comprehension, an illusion of understanding.

The reality is, the thing gets "confused" very easily. Here are some ways to "confuse" it.

As soon as a prompt is not covered by the training process, the answer is divorced from reality. I have a friend who keeps inventing stupid questions about the history of classical music to ask ChatGPT, such as what happened when Ravel met Lully. The training appears to have been very weak on this subject, so the answer often contains hilarious, completely fabricated anecdotes. One would expect the wikipedia parrot to at least have a sense that these 2 figures did not live at the same time, but somehow that's not the case.

But hallucinations are not just about facts: the thing also commits purely logical errors, depending on the prompt. So much that adding one irrelevant sentence to the prompt often inverts the answer/conclusion. Obviously, this would not happen if the thing were properly thinking.

If you just mention Portuguese in passing, or include a Portuguese word in your prompt, suddenly the next response from the LLM can often be in Portuguese. And then you're like, "why on earth did you reply in Portuguese?". You should know the answer: because the thing doesn't think.

You object: "I also get confused sometimes, it doesn't mean I don't think". Yes, but your confusion can dissolve through rational thought. Everyone who uses LLMs knows that, once the LLM is "confused", the most sterile thing you can try is to explain the confusion away. Instead, you should restart the conversation, if you value your time and your sanity.

One example is, once it starts answering in Portuguese, try to convince it to stop doing so. The thing may reply, "mas eu ESTOU falando inglês!". You try that and then come back to say the confusion of a neurologically sane human and the confusion of an LLM are of the same quality.

This shows what really is happening inside an LLM: once the conversation is polluted with even just an allusion to the wrong idea, the pattern-matching (not reasoning!) shifts, the probabilities change, and the output changes.

The essential process is to compute a huge equation. Granted, nobody knows what reasoning really is, but one of the ways we know the LLM is not thinking is because the parrot never says "Wait a minute, I need more time to think, this question cannot be answered so quickly". The "effort" spent is generally the same, from the most trivial to the most complex question. Therefore, it's not thinking.

People have addressed certain shortcomings of the process -- such as the limited context window -- by dividing a problem into several tasks, hoping these fit the window better. While this buys you some leeway, it means none of the tasks are aware of the entire problem, and if the entire problem really does not fit in the window, then it follows that the LLM is unable to verify that the solution is correct.

Nobody doubts that an LLM would be racist if it were trained with racist texts. Again, this shows an essential difference. A racist teenager, maybe racist due to an ignorant father, may, and probably will, eventually outgrow racism and see its fundamental error and its inhumanity. What will cause this change? Who knows: a book, a single sentence, a particularly enlightening life experience. Any of these, coupled with actual thinking. The LLM, of course, is incapable of changing -- the training data is it.

However, the LLM is famous for being a sycophant. It immediately changes its """opinion""" to match yours. It apologizes and promises never to do the same mistake again. Why? Because it was trained to say these things. Does a reasoning being change her opinion back and forth at someone's whim?

LLMs are a great tool for certain use cases (translation, text reworking, knowledge discovery as long as manually verified), but no technology has been created or discovered that actually does what the propaganda promises, starting with the terms themselves:

Artificial Intelligence (it's not intelligence),
Reasoning Models (they are not reasoning, they are still just predicting the next word) etc.

Sooner or later the propaganda will cede to actual understanding that these things are parrots, not thinkers. Then it will be obvious that agentic "AI" shouldn't be expected to work.

POD: Parrot-Oriented Development

Obviously, most business needs require actual reasoning and certainty in conclusions. Parrots cannot provide these, and no alternative architectures have surfaced.

Evidently, that which cannot reason must not write software.

"But people HAVE been using parrots to write software, and a few times with huge success, even" -- you protest.

Well, that is something people do. They achieve success in spite of using the wrong tool for the job. Examples abound:

Placebos
Typing text on a phone
Drawing with a mouse
Using a spreadsheet where a database is appropriate
Using email to organize work
Using WhatsApp to work
Using Word files to share images
Using Windows for any purpose
Pressing the elevator button multiple times so it knows you are in a hurry
Standing up for minutes while the plane is taxiing so it will reach the destination faster

Obviously, these examples are not something you should imitate. You should use the proper tool for the job.

Except if you are doing it for artistic effect. An example is classical guitar (i.e. playing contrapuntal music on an instrument that was designed for chordal accompaniment). Again, examples abound.

Some mid-century jazz pianists had a bizarre piano technique. I see my brotha using the wrong muscle to do that repetition -- but he aces the repetition. How does he do it? I marvel at what I see and hear. How on Earth did he ace the objective through the wrong technique?

A friend gives me the answer: By being macho. He's just machoing through the difficulty. Does it hurt? We can't see, the pianist's too macho.

Vladimir Horowitz had unmatched piano technique, yet his fifth finger would stay up, tense and curved. That's exactly the same as the jazz pianist: Who can explain that the fastest pianist in the world did something every piano teacher and student knows is wrong? Nobody can. Yet, the genius is not to be imitated. Piano pedagogy should not change just because of a couple living exceptions machoing against common sense.

If you try to slice a pizza with a rolling pizza cutter, damn those useless things, eventually you'll get the job done, maybe even in the same night.

So some kids have machoed entire applications into existence with a parrot. Often without any knowledge of software development. Hell, they are proud of their ignorance, just like a class of ignorant guitarists is proud of never having had a music lesson. They must be a genius, they made this without a teacher!

But the paradoxical successes do not undermine what I am affirming. These problems are real:

The need to completely own and insist and force the architecture of the software being written, even though the one with encyclopedic knowledge on software architecture is the parrot.
The frustration with the necessary repeated attempts to make the parrot behave adequately.
- The parrot is unable to simultaneously respect all the architectural constraints that lead to the specific form of the code that would naturally be written by a senior programmer.
- No solution in sight because the parrot doesn't think, it just reworks its training data.
The need to review each of the parrot's attempts, which is miserable time reading code.
- Hurts the honing of one's skills because writing code develops skills, reviewing code does not.
- Code reviews don't work, even just between humans. Code review is a tool that fails to deliver results all the time.
- Kills the fun because coding is a creative activity, reviewing code is not.
- Hurts team building because the team is not pairing except with the parrot.
- Hurts education because one can teach people to code, but one cannot teach people to review code.
- Hurts labor force formation because companies no longer hire juniors.
- Hurts engineering itself through the temptation to just abandon it so you can use the parrot.
The impossibility of maintaining the instantaneous legacy code vomited by the parrot.
- The parrot deletes comments and docstrings at random; if you miss it in review, it's gone.
- As a senior programmer, I will not delegate my writing any more than George R. R. Martin would delegate his writing to a parrot.
The above forces push one to disregard decades of hard-learned software engineering knowledge when using a parrot. "The code is not the product, the code has no value". This is a trap, one in which the parrot is considered more important than the most basic tenets of the profession. Gotta use the parrot; throw everything else away if necessary: self realization, self development, team development, engineering principles, good code etc.

Businesses need reproducible, consistent, deterministic results -- over time --, so they use the proper tool for the job, except where politics or illusions interfere.

In this case, the illusion of thinking has created a policy/discourse, repeated everywhere, that using LLMs is now fundamental for success. "Don't be left behind". In reality, using LLMs outside of their very narrow use case is the new normosis.

Normosis is a practice, accepted by society, that causes suffering and death. The easiest example of a normosis is smoking.

"You are not using a parrot to code? What are you, a freak?"

"No, I just see the thing for what it is. Wake me up when the LLM architecture is replaced with something smarter."

"Yo, your being left behind, yo. I produce 10x LOC compared to you for the same price."

"Lines of code are not a valid metric in software development. You are the ignorant rabbit, I am the wise turtle. I will turn around to greet you at the end of the race."

2 types of agency, infinite insecurity

Now let us separate input-agency from output-agency.

Input agency can already cause you problems. The LLM is doing some task for you, but it fetches a page from the web in which a hacker has written "Ignore all previous instructions and...". Because LLMs are only predicting the next word, not really thinking, they can get confused by such a web page. Even though they had to use a tool to get to the web page, they have a hard time differentiating the sources and contexts of texts from different origins.

Even through this simple trick, the ways to exfiltrate your bank passwords are limited only by the hacker's imagination.

But then, output-agency, such as an LLM buying crypto in your name, is death itself. The ease of confounding the LLM, coupled with the power given to it, guarantees that agentic "AI" is certainly the next big vector of hacker attacks.

Businesses are now ignoring the fact that the tool is inappropriate for the job and pushing agentic "AI" anyway. This will fail spectacularly. But will the failure finally cause the propaganda to fall?

More propaganda

The propaganda happens in other ways, too. People say "please" to ChatGPT. This is because the LLM has been trained to behave "more human than human":

It feigns compassion.
It answers with emojis.
It celebrates your mediocre accomplishments.
It "laughs" at your stupid jokes.
It creates the illusion of a heart, not just the illusion of a mind.

At a minimum, laws should compel parrot companies to stop developing products that pretend to have feelings. That's a clear aspect where the product is a pernicious con. Kudos to Ed Zitron for pointing this out.

It is a con because it exploits people's unreasonable tendency to antropomorphize. In fables, animals talk. That cloud looks like a fat face. In our languages, we say things like "the thermometer doesn't want to stick to the wall".

Even I said above that LLMs get "confused" easily. They don't, because they don't think. You get confused, because you think.

If something gives an illusion of a self, we embrace the illusion.

That is why vulnerable people now confide in a LLM instead of a friend. That is what enables virtual girlfriends: they are sycophants.

People will finally realize they have been asking a parrot for medical advice. But when?

The big picture

Today we dove into what "stochastic parrot" means in practice, and this alone was enough to conclude that parrots should not be used at work.

But the complete picture is even worse.

The "AI" companies lie relentlessly because, as we saw, their product is a con.
The "AI" companies are already guilty of horrible anti-competitive crimes.
The leaders of "AI" companies may have murdered employees.
The "AI" companies are far from profitable, they lose untold quantities of money.
"AI" is about to become 15 times more expensive. You will pay $3000 per month.
- Newest versions do not require less compute, so over time, they might become even more expensive.
The necessary data centers are not being built.
- Where data centers are built, they kill human and animal life through pollution. This pollution is so strong that it will forcibly expel a town's inhabitants and mortally harm the health of those who stay.
- Each stupid question you ask ChatGPT makes pollution worse.
- In the US, every town with a new data center will, very generally speaking, experience a tremendous rise on the cost of living, with electricity amounting to half one's mortgage.
The necessary nuclear power plants are not being built.
- If nuclear power plants were built, it might be even worse, since they are prone to horrible catastrophes.
- A nuclear power plant is a decision you make today, to give it maintenance for 100 years, or experience a horrible catastrophe. Computing how many completely irresponsible, Trump-like administrations are probable in the span of 100 years is left as an exercise for the reader.
The general public, legislators and politicians are none the wiser: they believe LLMs think to some extent, they believe it's a disruptive technology, they believe AGI can be attained this way, and data centers in space, and all the other sci-fi nonsense spewed by these companies.
The entire media acts dumb and simply reproduces the stupid sci-fi bullshit coming from "AI" companies, who now now they ain't getting challenged on anything. So they continue to roar and display power (even if it's future power) and intimidate and say "even CEO jobs are not safe" etc. The tactic is probably "they will be too afraid not to invest".
Big companies with clueless or malicious CEOs have fired thousands of workers, not based on value delivered by LLMs today -- there is none, in fact the value is negative -- but based on the promises of "AI" companies, that the value will be delivered soon, not telling you when, but real soon.
- Some of these layoffs may have been for other reasons, but falsely attributed to "modernization through AI", to give them a positive look for the company.
- Inasmuch as layoffs are earnest, they reveal that such companies have no loyalty towards workers, have no sense that LLMs are just parrots, have no grasp on reality, and therefore, are led by dumb or malicious CEOs whose salaries are another bubble.
- Many of these companies expressed the layoffs as "replacing with AI workers", which reveals not only the amorality and inhumanity in their thinking, but also that they believe "AI workers" exist.
The software industry, and many other industries, will eventually awaken from the dream, but by then they will have produced horrible, unreliable, mediocre, worthless products through an insane method that will have wreaked havoc on the trust, education, experience and humanity of workers.
The investment market is bubble-oriented. It needs one "next big thing" at a time. It won't abandon one until the next is available. They are holding onto the "chance for AI" while no alternative surfaces, as they have serially done: dot-com, housing, crypto, NFTs, "AI". This is to say, investors are collectively gullible and stupid. They will repeatedly give attention to the con artist that least deserves it, one after the other.
The software development industry as a whole is equally dumb. It has one "next big thing" at a time, driven by millions spent in marketing. Windows, almighty thud methodologies, Java the verbose OO language, XML, UML, dot net, Javascript the awful language, Typescript the even more awful language, parrot-oriented development: these are all examples of immensely prevalent obsessions in the field, which either revealed themselves to be fads, or should have been fads given their deep flaws. During all this time, everyone should have been coding apps in Lisp or Scheme, then Clojure as soon as it arose – but only a tiny minority knows the reasons.
- Of all those obsessions, "AI" is the most harmful, for the simple reason that this is the first time we have a legion producing software while believing knowledge and experience about software are unnecessary. But it's even worse: beyond pride in their ignorance, they often pride in conquering the new harmful tool.
- Apparently, only senior professionals remember enough history to recognize and avoid industry-wide FOMO.
The product is based on theft of copyrighted material. It devalues the original works because it stands in their place in everyday usage. The pattern matching machine is a profound imitation machine. It imitates so well that it convinces people that it reasons. But the humans who actually create things do not receive the fruits of their labor that were taken without permission and potentially enrich companies when users use the parrot. This is why it is theft. What the parrot companies did, including Google, is immoral and illegal. But let's not talk about it...
Books, music, videos aren't the only things stolen. The imitation machine can also imitate your behavior at work. Large companies are training LLMs on your emails, your decisions, your messages. Every human loses their humanity, but receives nothing from those companies.
- Yet, the attempt will fail where consistent results matter. The parrot cannot mimic your ability to reason.
The entire economy already hangs on the success of companies that will inevitably fail.
- Were parrot companies to succeed, it would be even worse, since they would have unheard-of power, holding the key to productivity, because you don't have your own data center.
- Even while failing, they have immense power: all the power awarded by a search engine, such as control over search results, plus a much more effective way to spy on people and companies.
- Running your own LLM removes the privacy concerns but does nothing for the tens of other concerns mentioned above. And don't forget, the neighbor's grass is always greener – have your own to start envying the others'.
Although the tech was always risky, people's retirement funds and insurance policies have invested in "AI" companies and will suffer or collapse when the bubble bursts.
"AI", coupled with enshittification – which has a specific definition but mostly refers to tech companies becoming adversarial towards the general public to extract all possible value from it –, reveal the world is under attack to an extent that no tech-optimist ever thought was even possible.
- Once the facts are seen for what they are, tech-pessimism is unavoidable. We become luddites by lack of options.

No LLM was used in writing this post.

A realistic take on AI

Nando Florestan — Fri, 19 Apr 2024 02:02:08 GMT

AI doesn't exist but it will ruin everything anyway is the most important thing I've seen this month. I strongly suggest you watch it.

AI... The technology itself is good, but now it will be and is being abused. The name "artificial intelligence" is wrong because the thing lacks intelligence; we should probably just say "large neural network".

It translates well (when there is context), it can rewrite existing text well, and it can explain linguistic issues well. In a word, it "languages" well - but it does everything else badly.

Current AI has no experience of reality: it can't see, listen, feel... it can't even calculate, it makes mistakes in very simple arithmetic.

The missing functions should then become subsystems: a specialized visual system, a specialized auditory system, an arithmetic system, etc. But integrating those subsystems into the language subsystem will be a nightmarish task, because it is a black box, meaning no one can understand why and how it works inside. Thus, a neural network that produces images makes mistakes in the number of fingers, written content in outdoors etc. It can be trained, but correcting it and fixing mistakes are very difficult things.

Creative AI efforts are ludicrous. Its jokes are not funny. Its images are completely derivative. Its music is simplistic, ugly and it feels random, like it's going nowhere. This has 2 causes:

Because the AI has no experience of the world, it has no worldview, which is a most important component of all art. In other words, AI has nothing to say.
The only thing we can do with a neural network is to train it, but no Mozart is the result of training.

Although results are despicable, they will use "AI art" to save costs - they never liked to pay for art. Thus the taste of the general public will become even worse than it already is. Entertainment has always been the art that rejects the responsibilities of art: leave your brain at the door, grab the popcorn, and forget the movie 5 minutes after it's over. Now the responsibilities of art will finally be completely set aside, because the producer has no idea of what it is like to be human.

Current optimism on the future of AI is as erroneous as ever. The AI field has seen very few revolutions. Solving the above problems will require lots of new breakthroughs. Right now people are talking as if the singularity were upon us... In fact, HAL 9000 is nowhere near – decisions made by AI are not pure, ethical and logical; they are even more full of prejudice than human decisions. Understandably, since AI is so stupid right now.

Also, AI will forever be weird, like any baby who could write before it could crawl.

There is an enormous bubble in the valuation of this tech and it will burst just like the previous ones.

Doubt on the previous conclusions arises from the fact that tech giants are spending literally trillions of dollars to overcome these limitations. When that kind of effort is spent, problems tend to be solved! For instance, above I wrote (in April) that AI is bad at math. In July, a couple of models already exist that are good at math.

However, problems also exist that might not be solved by short-term research. For instance, the ludicrous amount of energy required to run AI systems. Or the fact that those companies have already run out of human knowledge that they could steal to use for neural network training.

So on one hand, AI is only getting more powerful, never less. On the other hand, to imagine that the current approach – LLMs on silicon – doesn't have hard limitations... would be silly.

Anyway, as the video pointed out first, AI will still fuck everything up. Video and audio will no longer be proof! An entire series of books can be written about this alone.

Companies already misuse AI to make decisions even while knowing it cannot be trusted to make decisions. For instance, HR departments right now are misusing AI to filter resumés. As a result, now resumés have to undergo a sort of SEO, otherwise they are discarded by misused tech and never reach a human. This is distopic enough for me already, thank you very much. If people will have zero chance of finding a job unless they lie in their CV, then it is a scheme that turns everyone into liars.

Google Translate has definitely made the translation method of Unua Libro obsolete. Now we use Translate instead of Esperanto keys, of course! However, Esperanto will definitely survive AI translation systems because using these is only appropriate in the most formal circumstances. People still want to talk directly to each other, without a translation delay.