Increasing Entropy #2
Discoveries of the week, random links, AI doomerism, and a bit of my thoughts on random topics
Last Saturday we had my brother, his girlfriend, and my mother-in-law over for dinner. I don’t know how we ended up discussing AI safety, but it was an interesting conversation.
Coincidentally, this week I reached the chapters “Impact” and “Explosion” in “The Scaling Era: An Oral History of AI, 2019–2025” by Dwarkesh Patel with Gavin Leech. People like Eliezer Yudkowsky and Carl Shulman have a surprisingly clear vision of how AI could cause doom. The warnings escalated quickly. Leopold Aschenbrenner even compared AGI or superintelligence to a weapon of mass destruction and made his case for government regulations and international treaties.
To round out the topic, I finally got myself to listen to a two-hour conversation from The Diary of a CEO: AI Experts Debate: The AI Labs Are Lying To Everyone, Is There A 99% Chance Of Human Extinction?
The key point, I think, is: if there’s even a 0.1% chance of doom, we shouldn’t gamble with the fate of 8 billion people and should take it seriously.
- Nobody has solved AI alignment yet. We’re not sure AGI won’t be mean or deceptive, or whether it would share basic human values.
- We’re speeding up AI development right now and rushing towards making AI develop AI, towards recursive self-improvement.
- Supposedly, AGI could hack and copy itself to many places, and we wouldn’t be able to switch it off. We wouldn’t even notice until the damage is done.
- Evil parties could use powerful AI to hack computer systems and build weapons. This has already started causing chaos in the tech world.
I’m waiting for the report Hugging Face announced in P(doom) on user profiles.
Junior Engineers in the AI Era
A good discussion about junior engineers in the AI era started at the beginning of the week and unfortunately faded quickly.
Thorsten Ball shared his advice to a junior engineer on X, which basically boils down to:
- Think about business and what value you can bring as a technologist.
- Take ownership and act with high agency. Don’t limit yourself to “your area of responsibility”.
- Develop systems thinking.
- Learn to communicate well.
- Fundamentals still matter: databases, networking, OSes, virtualization, big O notation, GPUs, etc.
I share Andrzej Krzywda’s opinion:
[…] go crazy with AI, use all the tokens you can get from universities and companies, create, create, and create, learn, learn, and learn.
Andrzej says you always learn something by building.
AI allows you to learn like never before. You now have access to a thing that never gets tired of answering your questions and answers them immediately. And you can follow up with more questions until you understand the topic and it won’t get irritated. It can create an interactive tutorial for you for any topic. You don’t have to worry whether your questions are silly or obvious.
I think the community does a bit of a disservice to juniors when we call some projects slop or vibe-coded and get almost enraged by their existence. This may discourage someone, especially juniors, from showing what they built and from building altogether out of worry that it’ll be called slop.
My advice: don’t listen to gatekeepers and build fun things.
* * *
How Thorsten Ball Uses Amp, October 2026
The Company of the Future — Quinn Slack and Thorsten Ball, the Amp co-founders, talk about Amp the company.
* * *
Pi Durable — Pi released a new framework for building agents that aims to support asynchronous sessions. As I understand it, you give it a way to store data and a possibly ephemeral environment to run tools and it stores your sessions, handles all the retries in case of errors or interruptions, and allows you to connect to sessions and observe them even from multiple clients. I’m curious how to fit code checkouts in this scheme and want to play with it more.
1 in 8 Billion — Be born as a random person on Earth and make life choices.
The Future of CSS — There are so many new things happening in CSS and browsers. Here are a few examples. Also look at the nice tricks on the landing page for the Whimsical Animations course.
WorldTUI: I trained a 360M model to pretend to be a terminal — Simon Edwardsson makes an LLM role-play a computer with a terminal interface. Run bash commands, edit files in vim, switch windows and jump panes in tmux. Except nothing is really executed — everything is happening in the LLM’s context. Fascinating and absolutely mind-blowing 🤯.
* * *
@buildwithsid on X on Oct 21, 2026:
cracked the paid plans of 4 mac apps today, using the REA library which lets you reverse engineer anything
gave codex the library and asked it to remove the license checks, and it actually did 😭
it cracked the binary, found the thing checking if you paid, and removed the entire code and made the same app for free