j4orz annual ai armchair

nullius in verba

Welcome to the annual ai armchair: a corner of the internet where once a year near Christmas time, respite is taken from the daily drugery of chopping wood and carrying water to speculate about humanity's collective future: this is where I take a break from shape rotating to engage in wordcelling by the fireplace.

The deep statistical learning approach to artificial intelligence has reignited the realization that evolution has dropped us off at perplexing point in time where we can now observe and potentially partake in the process of creation. By zooming out from the menageries of jira, standups, and slack, one becomes aware that the seminal discussions which took place in the early 20th century with the advent of the logicist's computer are eerily simialr to those we're having today in the 21st century with the advent of the connectiont's large language model.

Both today and approximately 100 years ago, we've managed to orchestrate electrons in a dance which implement algorithms that were once deemed the hallmark of human intelligence. Ifs the artificial intelligence research program succeeds, then future textbooks will demarcate ChatGPT as the moment that transitioned humanity from the information revolution to the intelligence revolution. With moments like these, all the big questions that were asked since the dawn of humanity come screeching back into our lives like distant relatives during the holidays. Questions that grapple with the fundamental nature of reality such as:

what is the world?
what can we know?
who are we?
what should we do?

The goal of the annual armchair explorations is to provide small principled packets of epistemic entropyš explosions which showcase the questions that arise in my mind while working with artificial intellgence on a daily basis. Each year I cover how the overall direction of the field relates to the bigger philosophical project of naturalizing the mind.

With that said, my exploration will be limited insofar as to ask whether we're asking the right questions to begin with. Philosophy (including internet armchairing) is a suitable tool for distinguishing precise terms from fuzzy notions, identifying fallacious arguments, and ensuring we're asking the right questions. However once we have those questions, science is needed to come up with answers by operationalizing explanations through experimentation.

Given that make my living as a compiler engineer by day, I am limiting my civic responsibility to exploring these questions once a year, with the goal to MATH (Make America Think Harder). If as a result, more people are using my thoughts as stepping stones to individually explore nuanced topics in SEP (Stanford Encyclopedia of Philosophy) following the spirit of the Enlightenment's nullius in verba ideal — rather than dropping their next hot take on the internet — then by my standards, I have succeeded.

Moving forwards, a more, thorough, understanding of artificial intelligence is important for everyone the same way basic computer literacy is expected today.

Archive

2024 — deepseek moment

Contents

Note: the annual ai armchair for 2024 was a highly speculative piece about o1/o3 and was rewritten in the 2025 new year to account for r1

2023 - already misaligned

Contents

attention? attention!

This year was seminal in the history of artificial intelligence as the world, woke up to the advancements in one of the greatest philosophical projects which started a century ago at the Dartmouth workshop. The deep learning approach to growing intelligent machinery flew against the consensus view held amongst experts that intelligence required some master algorithm to be divined from the laws of nature just as physics did. While deep neural networks that learned representations end to end did in fact employ the precise tooling of mathematics, the act of training these systems is more akin to evolutionary biology than it is to traditional software.

Slowly but surely, watershed moments within academia filtered their way down towards mainstream consciousness in the form factor of consumer products. The best example being large technological breakthroughs in speech recognition and vision recognition in the early 2010s making its to smartphone assistants like Alexa, Siri, and Google Assistant. Finally, in 2017, language modelling had its big breakthrough with the transformer based neural networks presented in (Vaswani et al. 2017), which moved away from infinite look back design choice made with RNNs in (Mikolov et al. 2010) and back to autoregressive causal masks like the NADE network in (Larochelle, Murray 2011) and the MADE network in (Germain et al. 2015). The difference was that instead of using convolutions as the causal mask, they used the attention mechanism, turning neural networks into general set-processing machines.

While academia was very excited with the new attention mechanism presented in the transformer-based neural network, it was OpenAI who noticed signs of life in scale (Kaplan et al. 2020) and took the industry bet with GPT3 (Brown et al. 2020), GPT4 (OpenAI 2023). For more technical details, check out my reproduction of GPT2 and Llama3 here.

2018 (GPT1 117M params): grammar
2019 (GPT2 1.5B params): prose, poetry, metaphor
2020 (GPT3 175B params): long stories
2023 (GPT4 1.76T params): college-level exams

When OpenAI took pretrained GPT3/GPT4 and built ChatGPT by following Lecun's cake philosophy (IFT + RLHF), the reaction from the mainstream was visceral. All of a sudden, the big questions about the fundamental nature of our reality came screeching back into our lives like distant relatives during the holidays. Before we get into these big questions, we will lay down some principled groundwork for why a system like ChatGPT is possible to build in the first place.

math mechanizes mind

Many people struggle with the unreasonable effectiveness of mathematics. What's perlexing is its

effectiveness: why can mathematics precisely describe the fundamental nature of reality so well
unreasonableness: how can we know these mathematical objects if they are not located in space nor time?

We'll tackle these in reverse order, starting with the unreasonable essence of mathematics. Since math theorems are universally true, these objects are not localized in space nor time. While philosophers worry about the ontological status of mathematics due to the lack of verification, most if not all working mathematicians ignore the rocky foundations of the field and focus on its effectiveness. For instance, Penrose gives an example where:

With reality you think of a chair or something — something made of solid stuff. And then you ask what's our best scientific understanding of what that is? Well you say it's made of fibers and cells. And these are made of molecules. And those molecules are made out of atoms. And those atoms are made out of nuclei. And electrons go around. And then you ask what's a nucleus? And then you say well it's protons and neutrons held together by things called gluons and those protons and neutrons are made out of things called quarks. And then you say well what's an electron and what's a quark? And then at that stage, the best you can do is describe some mathematical structure. You say, they're things that satisfy the dirac equation or something like that. The mathematical description of reality is where we're always led. Our picture of reality depends on something more precise.

and although I'm saving my thorough, study of physics for when AGI replaces my job, it's already quite easy to relate to what Penrose is gesturing at from a programming language perspective:

After writing a few programs yourself you might want to peek behind the curtain to understand how the programming language you use was built itself. You then proceed to build another program which transforms the text with a lexer, a parser, an optimizer, a register allocator, and then a code generator. And then you ask well I understand how to implement a programming language but what is the programming language? Sure we can declare the implementation to be the definition itself, but what happens when we have two compiler implementations for the same programming language? And then at that stage, the best you can do is describe the programming language with mathematical structure. You say, a programming language is defined by it's statics and dynamics which are specified with natural deduction via type inference rules and small step operations. That is, just like physics, our picture of (artificial) reality depends on something more precise.

Whether formulating a definition for the natural electron or the artificial conditional, humanity ends up reaching for mathematics when expressions that make statements about the patterns and regularities of reality require precision. And the project of artificial intelligence uses these surgeon-like tools of extreme precision in its pursuit to find a causal mechanization of the mind. That is, they are not satisfied with areas of inquiry that provide hermaneutic explanations like psychoanalysis or correlates likes neuroscience.

what's your p(doom)?

Returning back to this year's zeitgeist — are we all going to die? What started out as a niche research program on rationalist-based internet forum Less Wrong and its associated Machine Intelligence Research Institute (MIRI) broke into mainstream consciousness. The core argument is as follows:

Assuming these two premises:

orthogonality: intelligence is orthogonal to goals

instrumental convergence: power is an effective subgoal for any goal

you can conclude artificial intelligence that is smarter than humanity will treat us the same way we treat our fellow animals: indifferent. This isn't necessarily a problem if we improved our understanding of intelligence from what is now natural (read: empirical) alchemy into scientific (read: predictive) chemistry. However, currently research in capabilities is vastly outpacing safety. Continuing to advance the former without the latter will ultimately lead to doom because we cannot yet answer questions like given a model M with some loss L, what are the bounds on capabilities C? There's no way mechanistically pluck some a mind with specific capabilities from the space of minds which is a big problem.

The most direct objection is to ask why can't we be more empirical about this? Can we initially forgo the investment in safety research and lobbying for policy, proceed forward with caution, and wait until we see this orthogonal intelligence in the wild? We don't have any hard evidence yet.

Well if you expand your sample space to include the first replicator and first human, then we do have hard evidence of orthogonal intelligence. Moreover, we have evidence of orthogonal intelligence explosions, since growth of the first replicator and first human were exponential phase transitions given that 50% of the growth happened in the last unit of time. If we wait until something goes wrong, it'll be too late. That is, there'll be an intelligence explosion.

already misaligned

Notwithstanding non-direct "meta" counter arguments such as invoking notions of Bootleggers and Baptists and/or Millenarianism, I am sympathetic to their conclusions assuming the premises. However, I'd like to present a deflationary account of doom in the sense of being limited to artificial intelligence. That is, building intelligent machinery is simply the harbringer of perennial news in which "alignment" has been an issue since the dawn of humanity — ChatGPT is just the reminder to civilization that ethics is an open problem.

Religion — which was the first torch bearer for humanity's development in moral philosophy — had a great start at bat with deontological-based ethics with Aquinas adding three theological virtues in Summa theologiae on top of Aristotle's (Plato's) four cardinal virtues in Nicomachean Ethics. However, religion as a moral philosophy project ultimately lost steam when it fell prey to the innovator's dillema by failing to pick up rationalism as a sense-making tool.

Ever since religion and science broke up, the strength and precision of expression in the domain of facts (read: what is) has vastly outpaced that of expressions in the doamin of values (read: what ought). Does this sound familiar? If ethics is understood as civilization's "safety research" and science is understood as it's "capabilities" research, then the "alignment problem" has existed since time immemorial. Whether the technology is nuclear weapons, gene editing, or your future dyson sphere, artificial intelligence is only the mirror that is reminding us that humanity is already misaligned.

While rational-based tools gave an attempt at bat with moral philosophy with consequentialist-based ethics, the project ultimately struck out with too many reductio ad absurdums. To this day humanity has no argument from first principled we can give to our children for why they should be kind. When I hear people that they are not affiliated with one specific religion but remain "spiritual", my mind parses that as phanton limb syndrome of morality from first principles. From the perspective of evolutionary biology, the misalignment of children is actually a feature — not a bug! The fact that culture forgets some values across generations — which can be interpreted as simulated-annealing — can be seen as evolution's solution to the fundamental explore-exploit tension (that is also seen in the field of reinforcement learning).

humanity's call option

While I too am sympathetic to this anti human-chavanist line of reasoning which comes very natural to anyone who is fluid with their thinking, I ultimately suspect the industry needs to strike a balance between dignity and humility. I will make a normative assertions for both groups: those that want to accelerate and those that want to pause.

On one hand, researchers and engineers who are building intelligent machinery need to realize the rest of humanity is part of the ship we're on and ensure to the best of our abilities that we maintain their sense of dignity.
On the other hand, non-technical people need to realize that the story of humanity has always been a story of change. Each advance of scientific understanding humbles humans by dethrones us as the universe's "special ones". We need to understand that mind is a physical phenomena and there is nothing preventing precise expressions (read: mathematics) to naturalize the mind through causal mechanization. In the future, substrate-agnostic minds is not a question of if, but when, and when we get there, we need to remember in-group/out-group firmware that suggests to us to “only love your own kind” has led to the worse kinds of ethical lapses in the history of mankind.

With everything said and done, this compiler engineer on the annual armchair has one idea to put forth on the table. Transformative AGI should be presented in the form factor as a call option towards humanity — we need to decouple the technological impacts from psychological impacts.

What this means in practice is that it's acceptable for machines to replace all mental and physical labor so long as our children and grandchildren have the time to think for themselves and reach consensus through a just process on how to moveforwards as the current stewards of our pale blue dot. Coming up with answer after having time to think is what humanity does best.