A Historical Moment: Meta AI Just Crossed the Uncanny Valley
A new vision is emerging
Sometimes you see something and you know it’s big. By now, I suspect that many of you have seen clips or watched Lex Friedman’s latest interview with Mark Zuckerberg, CEO of Meta, in full.
Miles apart, Lex and Mark talked for an hour in a shared virtual space using Meta’s VR headsets. They appeared to each other as real-time, high-fidelity avatars, developed as part of an experimental program within the company. A historical moment, as it marks the first time the uncanny valley for digital avatars was crossed in VR.
It may be hard to appreciate from behind your computer screen, but Lex described the experience during the interview as “something that’s breaking all geographical and physical barriers”, and you can see him repeatedly lost for words, visibly in awe.
Looking at it, it feels like a glimpse into the future. Or maybe it is the other way around, it looks like something from the future, warped into the present. Scenes from The Matrix come to mind, and the Holodeck from the Star Trek series, an amenity available to the crew of the ship that let them move freely in virtual space.
When Zuckerberg explains that both Lex and him went through an elaborate 3D scanning process to capture their faces, I’m reminded of another movie: The Congress, in which Robin Wright plays an out-of-work actress, who accepts a final bid from her movie studio to give up her likeness.
She gets scanned: her body, facial expressions, captured in all her emotions. And from that moment on, her image belongs to the studio.
The movie was inspired by a novel by Polish science fiction writer Stanisław Lem, The Futurological Congress, from 1971.
52 years later, the CEO of one of the largest tech companies on earth gets interviewed by a fellow human being in virtual space, after both went through a similar rigorous scanning process, creating photorealistic avatars of themselves.
Speaking of visionary.
The worst this tech will ever be
Nvidia AI Scientist Jim Fan, when sharing his thoughts on LinkedIn, said something that stuck with me:
Keep in mind, what we see here is the worst that this tech will ever be.
While scanning currently requires advanced tooling, Zuckerberg hinted that the company is working hard to make the process more efficient. So efficient that at some point a smartphone selfie video may be enough. Arms, hands, legs and inevitably full-body tracking will follow quickly after that.
Not only is what we see during the interview a showcase of what’s possible with present-day technology, it also offers us a new baseline to operate from: this is where we are today and it is only up from here.
The tech behind it is super clever actually. What you’re seeing is not a live feed of you, but an avatar that’s being put together by a generative model in real-time, based on the feedback it gets from the many camera’s that are inside the VR headset, combined with the data of the 3D scan of your face that you had made prior.
You can think of it as a very advanced photograph, in the sense that your avatar is frozen in time; if you were cut off your ponytail after you had your scan made, you would still appear with the ponytail in virtual space. Pretty weird huh?
The technology is expected to open up whole new realms of possibility, removing the screens that have always been there between us whenever we’re trying to connect over long distance, making interactions far more intimate.
The idea is that virtual reality can serve as a portal. A digital door that you can step into together, transporting you to a joint space. Enabling a new way to spend time with family members who live far away, sharing a virtual space with friends to play games or with co-workers to collaborate, or at some point even interface with loved ones who have passed away.
Meta’s vision for the future
Any VR or mixed reality experience stands or falls with how well it matches physical reality with no latency. Because real time rendering requires so much processing power, early attempts involved cartoon-style worlds that resembled Minecraft at best. I wrote about Meta’s early vision in November last year and everything that stood in-between.
Meta’s vision has evolved since, shifted notably. Rather than the metaverse being this immersive, parallel version of the real world, Zuckerberg now presents it as a blending of the digital and physical world.
During the interview, he elaborates:
“You know, a lot of people have this phrase where they call the physical world the real world. And the physical world is really important, but increasingly, I think, the real world is the combination of the physical world and the digital world coming together. But until this technology, they were sort of separate, fundamentally divorced and disconnected. Part of what this technology is going to do is bring these together into a single coherent experience.”
He predicts that we’ll have something that looks like normal glasses that lets you see the physical world, but also see holograms, sometime before the next decade. Not just holograms of people, by the way, but embodied AI characters as well.
And here’s where we see Meta’s vision come in full view, I think. Where Microsoft and Google’s primary focus is on productivity enhancement (in a wide range of copilot experiences), Meta seems to be less interested in that (although they did announce their own). As
highlighted in their excellent coverage of Meta Connect, Zuckerberg spent the majority of his time talking about their new AI entertainment features. Their biggest announcement: a wide range of AI characters based on celebrities like Snoop Dogg, Paris Hilton and Kendall Jenner.I predict these AI personalities are going to be a huge hit and drive engagements across its platforms through the roof. What if the real golden bullet is entertainment and not productivity?
In total, Meta is rolling out 28 AI-powered characters across its messaging platforms and we do well to remind ourselves once more that what we’re seeing here is the worst that this tech will ever be. It doesn’t take a genius to imagine what a V2 or V3 of this product would look like: Snoop Dogg as a life-sized avatar in your living room. Fo’ real, yo.
The picture that emerges is clear-cut. Meta is betting heavily on the entertainment value of AI and blending the real world with the digital world in a more cohesive way. And honestly, I can see it, I can see the vision and it seems like they’re making all the right moves.
The uncanny valley has been crossed. The future is manifesting. And the anticipation is almost palpable.
Join the conversation 💬
Leave a comment with your thoughts. Please let me know what you think, is Meta’s AI bet going to pay off, or will their vision for the future mixed reality turn out to be a mirage?
We've been watching "Peripheral" on Amazon Prime this week. I think the tech they discuss as 2030 tech is spot on, right around the corner. Great show, surprised me with how solid it was, and it talks about this stuff quite a bit.
I didn't get how Meta's tech described here is an improvement on a regular zoom call. I grew weary of the breathless hype and didn't watch the entire video though. Should I have?