Smashing the boundaries of Spatial Computing
From isolated apps in your living room ... to a vast virtual world!
Apple’s “Spatial Computing”, and “Spatial Personas”, give a frustrating tease of the liberating and empowering potential that I’m proposing in my vision of “Reality Computing”.
Note: I’ve updated this article to move the terminology on: I’m no longer using the word “Parallel”!
Spatial Computing and Personas
I recommend watching this short video to see the best of Spatial Computing:
And this one, to see how the personal space can be shared:
Quite a new and fun experience of computers.
But for me it screams “horseless carriage syndrome”!
Here’s a diagrammatic representation of all that:
You have a space shared with another person and app windows dotted around. If an app allows more than one window, you can dot each of them around, so a photos app could let you split off individual photos for example.
Note that currently, the invited person can only see and collaborate within one app at a time! Most sharing will still go via each app, and their backing online servers. Also, the pinning is lost if the device is restarted.
Not much like physical spaces
Whether provided by apps with local storage or apps with backing online servers, all of your stuff is split up and locked in. Each different type of digital object (photos, messages, events) is tightly managed by its separate corresponding app and/or service. You can only have messages in your messages app, photos in your photos app, etc. All collaboration is trapped inside each app.
There’s no way to do things we naturally do in the real world, like just pinning a single message to a calendar event. If you had a forum thread and a messaging thread up, you couldn’t just pin up a related post or message from each next to each other on the wall.
You can’t have anything that would otherwise be second nature to humans in physical reality, like a group of people gathering and co-creating using the full space and full range of types of digital objects all around them.
Smash those apps!
So the answer is to build an operating system that simply doesn’t have apps:
We can free our digital stuff out of the app windows and into hundreds of little objects, all thrown together where-ever we like in the 3D space.
Break each message out from the messaging app, and each group they’re contained in. Break photos out from individual messages. Pull events out from the calendar. Break emails out from your email app, then break paragraphs and images out of those; break music tracks from the music app, tasks from the to-do app, and so-on.
Collect any objects together you like into sequences, pin them up on the walls, make new docs and to-do lists and new calendars from collections of selected objects, make a dashboard of special lists of objects, make new threads composing messages from different messaging apps.
Then go ahead and simply invite people in to share the space or spaces that you've created; allow others in to see and interact with any of your digital objects, then mingle spaces and objects, theirs and yours. Collaborate in spaces with stuff all over the walls. You don’t need a gallery app and a chat app, when you can just meet in your 3D gallery and chat!
You can make a shared space either from the room around you or from a fully virtual space. You’ll see the others' avatar objects around, and in co-located AR you could see their virtual Persona object overlay and follow their real body. Someone not physically co-located will see an outline of the room shape as a fully virtual room with the avatars there.
IoT or smart devices will also have virtual augmentation of course, so they can be easily controlled and interlinked, including your dimmer controlling my lamp!
People will only be able to see what they have permission to see, so if you pinned up a chat they'd only see chats they're included in. Things like already public documents will be visible to them all of course, plus the video you that want to watch together.
A global Reality Network
This is what I’m calling the “Reality Network”, as manifest by “Reality Computers”, or RNCs (Reality Net Computers).
Like the Web, there’s only one global Reality Net. It can in fact become a vast space mixing 3D worlds with 2D content.
The Reality Net should all be as close to physical reality as we can make it, to leverage all of our expectations and skills around interacting with each other in physical space. We’re recreating familiar reality and supporting our intuitions about how things hang together there.
So, just like in physical reality, we should be able to work with, refer to, pick up, drop, replace and share world objects, all the way up from a small paragraph (or a leaf!) to a huge library (or a forest!).
We can imagine picking up or cutting out snippets of 2D documents rendered in 3D and dropping or pasting them somewhere. We don’t need to work only with complete documents any more, we can play with individual paragraphs, images, lists and sequences of smaller stuff.
It should be easy to assemble paragraphs and images into sequences to form messages, documents or books, messages into chat groups and on into global communities, then books into bookshelves, bookshelves into libraries, photos up to galleries, a library and a gallery into a street, streets into cities, leaves into forests, cities and forests into lands; a working drawbridge up to a fairy castle, a chess or skittles game up to a theme park, lands into worlds, all in one huge universe!
Links or URLs
Our world objects are sewn, pinned and mashed into the fabric of this single, shared virtual universe through the pervasive use of or links. Links form handles or hooks onto all of our world objects. Links point to things, where-ever they may be. These are “upgraded Web URLs” that can work in a 3D virtual world.
We can grab, wire and share direct links to and between any world object owned by anyone, anywhere. Every single object we create or see has a link - notes, documents, contact cards, calendars, messages and photos; flowers and petals, vases, tables, rooms, houses, galleries, libraries, streets, cities!
Everything can be pointed to, picked up, re-used and saved via these links. Then everything can be tied, snapped and mashed together with everything else via these links.
Every world object we make is in a single unbounded, linked-up universe, so we can re-use, re-mash and re-purpose anything, and join all of our spaces, buildings and games together in new ways, sharing materials or entire constructions between our creations.
I can link to a city you've built from my virtual garden, which lets me see it and just walk across into it, or you can link to a leaf I made, and use it yourself.
Paragraphs are linked into messages or posts, messages into feeds, feeds into collections of friends and families. In this universe, your “social network” is literally a visibly-linked network of world objects.
Links are at the heart of the extreme and pervasive mashability of the Reality Net. Links empower us to create anything we like, then to join it all up into a huge landscape. And then to go on to build a huge universe together, mixing and mashing, sharing, re-using and re-purposing anything with anything else as we like.
Building the Reality Computer
Technically, in order to achieve this, you do have to basically rebuild everything from the metal up (so see this as a research vision for now - we’ll figure out how to get there along the way…).
For a start, in order to allow all these digital objects to be mashed up, each backing service (WhatsApp, Twitter, Photos, Calendars, Notes, etc.) would have to switch from their current “walled garden silo” model to, well, caring about interoperability!
They would need to expose all of our content, our messages, images, events, etc. to us, not in any closed proprietary formats but in standard open ones, just like on the current open Web. If they won’t do that then we do have to switch to services that will.
As described above, each object - every message, paragraph, image, calendar event, task, thread and list - would have a URL-like link or handle to allow it to be seen, grabbed, shared and mashed up with other objects in the space.
We would also need to see all the updates to any objects in the scene, of course, so our PRC OS would be notified when someone added a message object to a thread object, for example.
Our local app-free operating system then pulls in, from the respective host servers, all the objects linked to from any space via their URLs or links and assembles the live scenes for each of us.
Our Persona objects (avatars) should now also work across all services, so we’d only have one identity (or “login”) across all of them. Sharing is achieved by setting permissions on each object - either read-only or full interactive. These permissions are enforced by the service hosting the object and the AR render will then reflect in the interface the individual object permissions set by each owner.
Once we’re thinking Web-like, anyone will be able to run up a server to host more objects and open services for use in our spaces. Indeed, on the theme of the Web, all static web pages such as from Wikipedia will be importable into the Reality Net, broken up into individual paragraphs each with their own URLs or links.
Local First
With any of the apps in your Spatial Computer your stuff can be either saved remotely or locally: you either have cloud storage of photos or manage them on your device; you either have server-visible messages or end-to-end encrypted ones backed up either locally or on the cloud; your calendar may be provided by an online service or be a locally-running and saving program.
In our new operating system without apps, locally-hosted objects will continue to be hosted on our own machines, of course. But now they’ll have their own URLs! This means they have to also be in the open standard formats that everyone else is using, and can thus be readily mashed up and pinned up alongside all the other stuff in our shared spaces. Actually using those local URLs to transfer objects and their updates from my local machine to yours involves either direct peer-to-peer networking, or routing via an intermediary host.
Local hosting of our objects gives us full Sovereignty over our digital output, instead of sharing that with Big Tech. So in fact, we should go full “Local First” and use peer-to-peer networking for all of our own digital property.
We will also want our identities and Personas to be hosted in and driven from our own local machines and devices.
Conclusion
I’m building the RNC right now as a research proof-of-concept. Why not get in touch and start a conversation with me about it?
What do you think? Is our future in the Reality Net? Drop your thoughts into the comments below.