Prophets can point to him coming. If and when he/she does arrive, everything will change.
You need to believe and prepare.
The point is:
Like the second coming of Christ, the timing of the arrival of AGI is completely unclear and uncertain. Everyone’s guessing, pointing at signs and precursors.
The inherent uncertainty is an excellent source of problems and potential solutions. All is so vague that lots of projections appear. No facts.
What’s more because of the uncertainty and non-clarity around AGI the prophets, be that Altman or whoever, can make all sorts of claims.
Any criticism runs into the problem that denying something that isn’t here or exists, and claiming that it will turn out differently than what the prophet is saying, is also very hard, if not impossible to base on any solid ground.
In another religious comparison, the practice of prompt engineering for LLMs is like being a priest. The LLM is like a god in the sense that it has enormous power and can't be controlled to do what we want on command, but maybe if we talk to it in juuuuuust the right way, we can coax the behavior we want out of it. That's less about displacing a sense of awe from the divine to an artificial system that you're talking about here and more just about trying to reign in the forces of nature, but the theme is similar.
Can you though? Or do you just have to try your best and hope that it decides to humor you? I'm trying to keep an open mind about the way conversation design is going at the moment, but I'm really not a fan of prompt engineering (either practically or conceptually) and I hope the systems we use take a more deterministic turn as things develop. I'd rather we put our energy into building irrigation systems than in trying to perfect our incantations to the rain god.
I’m with you there. I have good hopes for hybrid systems that combine best of both worlds. There have been some early attempt and to me it sounds like a promising avenue to explore.
God is a blind watchmaker or God doesn’t play dice? This debate over AGI is the debate over determinism & randomness. It is Apollo vs Dionysius. Or Taleb vs Tetlock if you will. I would have thought the 2nd law of TD would settle the thing but apparently not. These little babies, bathing in QE, have been cooking with the oven door open & have baked a cake that is nowhere near as tasty as what we had before. But thanks to loose money/energy, they’ve conjured up a vision of a perfect ultra future society (TM) where all seeing knowledge & predictability, not belief, can apparently solve everything. But like all entitled Westerners, they can only see a world in one dimension, at one scale, in one space - so they fail to understand the implication of their dream. If everything were computable / predictable, there would be a crazy war games like scenario of the world making itself afresh across time & space as the intelligence seeks to CHANGE stuff. This can only accelerate - & will do so far beyond human scale - as gazillions of possible outcomes are predicted/tweaked/predicted etc Over time, the tweaks will lead to (ever faster) transition shifts & chaotic behaviour. There will be cascades of change. Humans won’t survive the first 10 seconds of all this but, extend the thought out & it’s possible to imagine the cascade filling the universe. And what are we left with if all this thermodynamic ‘work’ actually happens? Heat. Random energy. Collapse. So long before we could wreck other planets with our perfect ultra future society (TM) God would fight back & take us back to where we began.
“These little babies, bathing in QE, have been cooking with the oven door open & have baked a cake that is nowhere near as tasty as what we had before.” - great analogy, Claire!
Silicon Valley is a bit like a religion. Artificial intelligence is a bit like Mass, and Big Tech is like the church. I'm not sure who the prophets are, perhaps powerful Venture capitalists. The church sure has a lot of taxes. Is California the Holy Land or where dystopia comes from? For some people I reckon AGI is a bit like Satan. A kind of new level of patriarchy and oppression.
If you are referencing his claims, you're already being played.
AGI, AGI... It's nowhere close. Yeah maybe 20 year, maybe 15, three is a joke. They need that hype to keep investor money flowing. Many of the top 1% AI experts laugh at AGI claims. OpenAI has been called the next WeWork. It's not nearly as rosy as Sammy will lead you to believe.
LLMs are really nice writers and storytellers, but they don't have a clue what they're saying. Really. So as human as they feel, there's a massive chasm between an LLM and AGI. No one knows how to get there. Scale won't do it, scale is already beginning to reach it's hockey stick limit.
Whether LLMs and where they will be in a few years is enough to manipulate people's needs for godlike experiences, especially in what (if people don't wake up) will be a commie-socialist-facist world... perhaps.
It's already a fascist world. We live under the boot of capitalism. You know who's really to blame for the wars right now yes? The psychopathological military industrial complex. Boeing et al are making huge profits. Capitalism kills.
Hi Jurgen. Thanks for sharing your ideas. So I am able to orientate myself can you delineate for me your understandings about 'religion' and 'spirituality'. Many thanks.
It absolutely is on topic. Israel is using AI, Lavender, to target civilians in Palestine. Why is Israel slaughtering them? A land and resources grab. AI is just another tool for the ruling class to harm us. You can't seperate AI from all the death it's causing in the name of profit.
"build AGI that benefits all of humanity" - and yet, the guy still can't define what that means, how it would look, work or function, and has no idea what impact, positive or negative, it would have on humanity.
I cannot but wonder who with a healthy body and mind would want a future governed by AGI? Where is humanity in this? These guys are not visionaries, they are "enlightened" of the worse species ... (and I'm also involved in computer science and AI, so not just a "Luddite")
Let’s call them weird. It’s weird to want to create something so powerful, that you will have to give up all agency to it. They do it because they think the world will be a better place, but that all but guaranteed. It’s building a God just so you have something to worship.
I still don't understand how they think they are going to birth a superintelligence when it's built from a patchwork of average intelligence. It's literally the average of all of us, so how can it exceed us except in its capacity to multitask and scale? It won't be smarter, it'll just be able to attend to thousands of tasks at once. There's still no human-level reasoning. The religious angle is spot-on.
Excellent research and article. I’ve been thinking about this a while, and have slightly different take on this that might add another angle. I may well write a response article to yours 🙂🙏🏾
"Similarly, Islamic traditions speak of talking mechanical heads that could offer oracle-like wisdom." does not seem true, any reference about this ? I saw something about Budhist traditions doing this but not Islamic.
yes, I think that you're confusing things. I am a Muslim and Muslims don't do what you mentioned, I don't believe that what you wrote is true. Can you please correct ? thanks.
Not sure why would respond to my article is such fashion? You might’ve misunderstood, but I don’t think I say anywhere what you think I have said. Happy to clarify :)
"I say belief, because this is not a scientific fact. We don’t know if it is possible to devise such a system. It’s a view that is certainly held with a lot of conviction by many, but it’s not in any way settled science, which means that the belief that AGI will arrive this decade or the next is entirely faith-based. How should we come to understand this?"
There is some epistemological difference here yudkowsky and others in those circles tend to believe in bayesian epistemology where they define evidence as "an experience linked with chain of cause and effect with whatever you want to know about" , now they also believe in logic, where if you have some true premises you can have a true conclusion, i.e. you cannot test whether a photon which escapes the horizon of the reachable universe actually exists, but it's a logical conclusion of the believe in natural laws. Also as a side note under bayesian epistemology you absence of evidence is evidence of absence as governed by probability theory.
Now their worldmodel ( they also tend to believe in bayesian brain hypotheses) which probably consists of their belief in intelligence hypotheses and other economic factors can lead them to make predictions regarding AGI, for example we haven't found many laws of nature prohibiting slow and fast intelligence explosion.
Which isn't actually faith-based assertion under definition of bayesian evidence.
But if you only consider scientific evidence (which is within their framework is public reproducible bayesian evidence) and disregard all other forms of bayesian evidence like legal,historical evidence etc . Then you can assert their conclusions are faith-based.
AGI is like the messiah.
Prophets can point to him coming. If and when he/she does arrive, everything will change.
You need to believe and prepare.
The point is:
Like the second coming of Christ, the timing of the arrival of AGI is completely unclear and uncertain. Everyone’s guessing, pointing at signs and precursors.
The inherent uncertainty is an excellent source of problems and potential solutions. All is so vague that lots of projections appear. No facts.
I wish I thought of this, AGI as the coming of the prophet. Love it and so true.
What’s more because of the uncertainty and non-clarity around AGI the prophets, be that Altman or whoever, can make all sorts of claims.
Any criticism runs into the problem that denying something that isn’t here or exists, and claiming that it will turn out differently than what the prophet is saying, is also very hard, if not impossible to base on any solid ground.
I would consider the Altmans of the world the high-priests, evangelizing the coming of the prophet that will bring us salvation.
Talking heads and speculators galore
Haven’t finished reading Amodei’s new essay yet, but his approach is much more to my liking.
In another religious comparison, the practice of prompt engineering for LLMs is like being a priest. The LLM is like a god in the sense that it has enormous power and can't be controlled to do what we want on command, but maybe if we talk to it in juuuuuust the right way, we can coax the behavior we want out of it. That's less about displacing a sense of awe from the divine to an artificial system that you're talking about here and more just about trying to reign in the forces of nature, but the theme is similar.
I love that comparison! As long as you know the spells, you can make it sing.
Can you though? Or do you just have to try your best and hope that it decides to humor you? I'm trying to keep an open mind about the way conversation design is going at the moment, but I'm really not a fan of prompt engineering (either practically or conceptually) and I hope the systems we use take a more deterministic turn as things develop. I'd rather we put our energy into building irrigation systems than in trying to perfect our incantations to the rain god.
I’m with you there. I have good hopes for hybrid systems that combine best of both worlds. There have been some early attempt and to me it sounds like a promising avenue to explore.
God is a blind watchmaker or God doesn’t play dice? This debate over AGI is the debate over determinism & randomness. It is Apollo vs Dionysius. Or Taleb vs Tetlock if you will. I would have thought the 2nd law of TD would settle the thing but apparently not. These little babies, bathing in QE, have been cooking with the oven door open & have baked a cake that is nowhere near as tasty as what we had before. But thanks to loose money/energy, they’ve conjured up a vision of a perfect ultra future society (TM) where all seeing knowledge & predictability, not belief, can apparently solve everything. But like all entitled Westerners, they can only see a world in one dimension, at one scale, in one space - so they fail to understand the implication of their dream. If everything were computable / predictable, there would be a crazy war games like scenario of the world making itself afresh across time & space as the intelligence seeks to CHANGE stuff. This can only accelerate - & will do so far beyond human scale - as gazillions of possible outcomes are predicted/tweaked/predicted etc Over time, the tweaks will lead to (ever faster) transition shifts & chaotic behaviour. There will be cascades of change. Humans won’t survive the first 10 seconds of all this but, extend the thought out & it’s possible to imagine the cascade filling the universe. And what are we left with if all this thermodynamic ‘work’ actually happens? Heat. Random energy. Collapse. So long before we could wreck other planets with our perfect ultra future society (TM) God would fight back & take us back to where we began.
“These little babies, bathing in QE, have been cooking with the oven door open & have baked a cake that is nowhere near as tasty as what we had before.” - great analogy, Claire!
Silicon Valley is a bit like a religion. Artificial intelligence is a bit like Mass, and Big Tech is like the church. I'm not sure who the prophets are, perhaps powerful Venture capitalists. The church sure has a lot of taxes. Is California the Holy Land or where dystopia comes from? For some people I reckon AGI is a bit like Satan. A kind of new level of patriarchy and oppression.
"a bit" is putting it lightly.
I have a few points to make, but first - full disclaimer, I have worked in AI for more than a decade.
Sam Altman is a pile of crap. https://garymarcus.substack.com/p/openai-on-a-path-to-becoming-the
If you are referencing his claims, you're already being played.
AGI, AGI... It's nowhere close. Yeah maybe 20 year, maybe 15, three is a joke. They need that hype to keep investor money flowing. Many of the top 1% AI experts laugh at AGI claims. OpenAI has been called the next WeWork. It's not nearly as rosy as Sammy will lead you to believe.
LLMs are really nice writers and storytellers, but they don't have a clue what they're saying. Really. So as human as they feel, there's a massive chasm between an LLM and AGI. No one knows how to get there. Scale won't do it, scale is already beginning to reach it's hockey stick limit.
Whether LLMs and where they will be in a few years is enough to manipulate people's needs for godlike experiences, especially in what (if people don't wake up) will be a commie-socialist-facist world... perhaps.
It's already a fascist world. We live under the boot of capitalism. You know who's really to blame for the wars right now yes? The psychopathological military industrial complex. Boeing et al are making huge profits. Capitalism kills.
Politely asking to keep comments on-topic. Thank you.
Hi Jurgen. Thanks for sharing your ideas. So I am able to orientate myself can you delineate for me your understandings about 'religion' and 'spirituality'. Many thanks.
It absolutely is on topic. Israel is using AI, Lavender, to target civilians in Palestine. Why is Israel slaughtering them? A land and resources grab. AI is just another tool for the ruling class to harm us. You can't seperate AI from all the death it's causing in the name of profit.
Respectfully, this post is not about that.
"build AGI that benefits all of humanity" - and yet, the guy still can't define what that means, how it would look, work or function, and has no idea what impact, positive or negative, it would have on humanity.
I don't take that company seriously anymore.
I cannot but wonder who with a healthy body and mind would want a future governed by AGI? Where is humanity in this? These guys are not visionaries, they are "enlightened" of the worse species ... (and I'm also involved in computer science and AI, so not just a "Luddite")
Let’s call them weird. It’s weird to want to create something so powerful, that you will have to give up all agency to it. They do it because they think the world will be a better place, but that all but guaranteed. It’s building a God just so you have something to worship.
I still don't understand how they think they are going to birth a superintelligence when it's built from a patchwork of average intelligence. It's literally the average of all of us, so how can it exceed us except in its capacity to multitask and scale? It won't be smarter, it'll just be able to attend to thousands of tasks at once. There's still no human-level reasoning. The religious angle is spot-on.
Excellent research and article. I’ve been thinking about this a while, and have slightly different take on this that might add another angle. I may well write a response article to yours 🙂🙏🏾
Thanks for your kind words. Would love to read your take on it, Pranath. Keep me posted!
"Similarly, Islamic traditions speak of talking mechanical heads that could offer oracle-like wisdom." does not seem true, any reference about this ? I saw something about Budhist traditions doing this but not Islamic.
There’s a rich tradition of creating automata by Arabs. I’ve been using this as a resource: https://www.aramcoworld.com/Articles/November-2019/Robots-of-Ages-Past
Although now you mention it, I might’ve mixed up that with this: https://en.m.wikipedia.org/wiki/Brazen_head
yes, I think that you're confusing things. I am a Muslim and Muslims don't do what you mentioned, I don't believe that what you wrote is true. Can you please correct ? thanks.
Thank you for letting me know, I’ve corrected it.
thanks !
Tell me you didn't understand Nietzsche without telling me you didn't understand Nietzsche.
His entire point is that we SHOULD NOT NEED an external validator for morality.
Which you would know if you had read and understood what he wrote.
You clown
Not sure why would respond to my article is such fashion? You might’ve misunderstood, but I don’t think I say anywhere what you think I have said. Happy to clarify :)
"I say belief, because this is not a scientific fact. We don’t know if it is possible to devise such a system. It’s a view that is certainly held with a lot of conviction by many, but it’s not in any way settled science, which means that the belief that AGI will arrive this decade or the next is entirely faith-based. How should we come to understand this?"
There is some epistemological difference here yudkowsky and others in those circles tend to believe in bayesian epistemology where they define evidence as "an experience linked with chain of cause and effect with whatever you want to know about" , now they also believe in logic, where if you have some true premises you can have a true conclusion, i.e. you cannot test whether a photon which escapes the horizon of the reachable universe actually exists, but it's a logical conclusion of the believe in natural laws. Also as a side note under bayesian epistemology you absence of evidence is evidence of absence as governed by probability theory.
Now their worldmodel ( they also tend to believe in bayesian brain hypotheses) which probably consists of their belief in intelligence hypotheses and other economic factors can lead them to make predictions regarding AGI, for example we haven't found many laws of nature prohibiting slow and fast intelligence explosion.
Which isn't actually faith-based assertion under definition of bayesian evidence.
But if you only consider scientific evidence (which is within their framework is public reproducible bayesian evidence) and disregard all other forms of bayesian evidence like legal,historical evidence etc . Then you can assert their conclusions are faith-based.