Spatial Computing with links not apps
Freedom from the Metal to the Metaverse
At the time of publication Apple have been in the news with their “Spatial Computing” Vision Pro XR headset and corresponding VisionOS operating system. Although the phrase “Spatial Computing” has a long history, it has been made popular recently by these announcements from Apple.
Apple’s tech is essentially AR plus VR, or XR, however, its primary marketing message isn’t that it can create augmented scenes or immersive virtual worlds.
Rather, its main selling point is that you no longer need a fixed, small screen on your computer: the screen is now the whole space around you. So you still have applications and the familiar “desktop”, but they now float in front of you at a much larger scale.
But I believe that this has much of the “horseless carriage syndrome” about it. Even though it’s clearly an improvement to multiply up the screen size available, it’s still holding on to what is now quite an ancient desktop-and-applications model of computer interfaces.
Spatial Computing, as presented by Apple, misses an opportunity to re-imagine what an operating system could be like, now that we have a full 3D space to work within.
As I discussed in “Apps considered harmful”, I believe that we’d be better off without apps, or applications, all together!
A space without apps
In fact, if we drop apps, we can now use this virtual space to access all of our digital property directly, including the property of anyone else. In my version of “Spatial Computing” you would be surrounded in the XR space with all of this digital property:
Experienced through XR, this would essentially be a “parallel digital reality”.
A parallel reality
In the absence of apps, all of our virtual digital property can be accessed from within a single, shared virtual world. A world where things just work like they already do in reality:
Just as in the real world, you would be able to paperclip a friend’s contact info to a date on the calendar that’s hanging on your virtual wall, next to a pinboard covered in photos and notes.
Without apps, your digital stuff can simply be available all around you in this shared world - a world that respects our intuitions from reality about how things and people work together:
Instead of a chat app, you just meet and chat
Instead of a gallery app, you just visit and wander around a gallery
Instead of a calendar app, you just look up on a virtual wall to see your calendar
Instead of a game app, you just visit a place where people play that game
The digital stuff we see in this world will include both 2D, with text and images, and 3D places, property and people.
In a single shared space, this operating system will show you all of your own digital stuff on your own device, intermingled with the shared, live digital stuff of everyone else, including their avatars, on all of our devices across the planet.
All of the instances of this operating system together would thereby manifest a “parallel digital reality”.
Now, that’s also known by many of us as “The Metaverse”, and we’d very much like that Metaverse to be open, not owned by - and shipping all of our data to - Apple, or Google, or “Meta”…
The Metaverse built like the Web
I recently wrote an article describing how we could build the Metaverse to be like the Web: an open medium that anyone can join. I called it the “Meta-Web” to indicate how it was a combination of the two concepts.
The Meta-Web is a Metaverse that’s built on a simple and powerful technology that the Web brought us - the URL:
A URL is simply the unique ID of a chunk of digital content, that is being used to refer to it, perhaps from within another chunk.
URLs drive the Web’s openness; they empower anyone to contribute to the global Web, with commodity software running on their own servers.
Using this key concept from the Web would bring the same openness, simplicity and power to the Metaverse.
And of course we’d also want the same openness, simplicity and power for our Spatial Computing operating system's “parallel digital reality”. We’d also want to use URLs to implement the sharing in this world between running instances of the OS. We could have links to each other’s digital property (with a permissioning system).
Now, as I described in that “Meta-Web” article, the Web’s URLs do need to be “triple-upgraded” for the demands of the Metaverse. However, the main “upgrade” we need to describe here is how we would use links in place of a traditional operating system’s files, in order to bring up many little “chunks of digital content”, to link together as objects within our shared space.
Links between our Little objects
Now, in conventional operating systems you have local files found through filenames, filepaths and folder collections. You would see them in a file explorer app.
But without any apps, we can’t see files that way any more.
Files themselves have contents hidden away in the internal structures of different types of file, such as the paragraphs of a document, that we aren’t expected to see or change directly. The corresponding app used to let us do all that.
But without any apps, we can’t see inside files that way any more.
We need a replacement for files and folders in our Spatial Computing operating system. Well, of course, we can just use links. As mentioned above, a link is a unique ID for a chunk of data:
The internal structures within files can be broken out into many little chunks, that can themselves now be linked together.
Filenames and filepaths are replaced by links.
Folders are simply sequences of links, that also have their own links.
We don't need our digital stuff in strict rigid folder hierarchies, or locked away inside inscrutable file formats. Folders, files and all the hidden bits inside files were great for apps, but are now boundaries that we no longer need.
Instead we can use links to sew little chunks of our digital property together, such as paragraphs and images, however we like, rendered into the virtual space as objects in our operating system.
Open, seamless
Links give us a handle on all of our tangible digital property. Everything from single paragraphs up to entire libraries, everything from a leaf to a forest, can be ID’d and can be linked and structured together to build our operating system’s shared virtual world.
Now that we are free to use these links at the finest grain of our data, the smallest of world objects, this space will be a completely seamless experience for all of us.
Instead of juggling apps, we’d work primarily with these pervasive links between all of our digital property.
We’ll all be working together within a single, open parallel reality, manifest by all instances of our Spatial Computing OS across the planet.
The experience of this OS
The picture below illustrates some basic elements of your experience using this operating system: there’s a room you’ve built from linked wall and floor panels, which could be a co-creation space between a number of people. There are relevant 2D objects attached by links to the walls, plus some objects have been pulled out aside to float in front of you, or have been scattered on the floor.
At the bottom of the screen is a place for you to collect links to things for use in any further context. Some objects appear twice or more: they look identical because they are the same object, so if such an object changes, it changes everywhere that it’s linked to.
Revisiting our example day
So let’s go back to the example day in my recent article, “Apps considered harmful”, to see how much better it would be in this operating system, using links between all of our objects in the shared space.
Note that I’ve gone through the whole example for completeness, but it’s basically just variants on how our digital chunks can be linked together in a shared space, so once you get the idea, you can skim through it if you like.
Remember that it was a day when you’re meeting up with some friends in town for a one-day seminar on growing your own vegetables. (Like last time, different types of world object are highlighted in bold.)
Starting with the arrangements: up on your virtual wall in the room you use to organise your life, there’s your calendar on which you have a calendar event for the seminar with your friends all on it as attendees. So you grab a link to that attendee list, and pin it to the wall for now while you create a new group that you’ll use for messaging them about the event. So if someone drops out and they cancel their event attendance, that will reflect immediately in the group membership. The contacts available in the calendar to create new events are the same as those for the messaging, and also can form mailing lists for longer-form interactions.
You’ve got the seminar event info pinned up, and can just grab its link to put it on the calendar’s event. There’s more detailed event info in a document, linked from the one on the wall, which you can see in a smaller render embedded within it, but you can pull it out aside into the world in full, perhaps to look at briefly and close, or to then pin under the seminar event object. Everyone on the attendee list can always see that document too, as everything is in common open standards for all different types.
On the morning: you have both the calendar event on the day and an alarm - which are both basically the same, so you can even drop a link to the alarm into the calendar to see it there if you like. The alarm can be set to the event’s start time minus a couple of hours to determine your getting up time. You may have assigned the alarm a physical position in AR in your bedroom, for easier access when it’s needed.
When you looked up the weather the day before for the combination of date and location, that information was simply taken from the event as calendars, maps and weather reports are so similar and are aware of each other’s properties.
You can have a single map and calendar dedicated to the trip, combining links to the seminar’s calendar event and map and a calendar and map for the weather. You can plot the locations of your friends there, too, by dropping their links into the map. You can plot events and weather “events” on the same calendar, by simply dropping their links into it, and plot the locations of all your friends on the same map as the event’s getting-there map. No need to update your calendar every time the seminar event changes, as you have the actual original so it updates itself. Any map can be conveniently laid out on the floor of the virtual room.
The train operator has train line segments with disruption at certain time slots. You have your route and timings for that. You can project both onto a calendar and a map with their links to compare them easily.
While travelling, you want to listen to music while playing a game but the sound level of the music drowns the game’s. So you simply go to the virtual music object and turn it down, just like in real life! You can share your music playlists and what you’re listening to with your friends, because of course they have links like everything else.
You want to have a catch-up with the news so grab that. There’s actually an article there on growing your own veg, so you excitedly grab the link to post to the messaging group. When your friends hit the link, they get to see the story just like you saw it.
Approaching the destination, you go to your texts, as you trust the SMS telecoms infrastructure more than the internet that each of you could have access to while out and about, and reuse the link to the group of people you’re meeting in order to send text messages. As you get closer, you can switch easily from texting to phoning because texts, phone history and contacts are all related so are all cross-linked.
Once at the venue and learning all about veg, you’re furiously taking notes and snapping pictures. You drop links to relevant photos into the middle of the notes.
You make a to-do list of things that you and your friends can get started on with a joint project, including getting an allotment. You look at the locations of allotments on a map. You get a link to each location as a list object, then drop that list into your notes, and also into a related to-do. In both places you can now see a map where you dropped the list of locations.
You tie up (with a link!) the section of the notes about fertilisers that corresponds to the specific to-do for buying some. One to-do says that you’ll all meet up to buy some seeds, so now you flip to the calendar to make the date, and the to-do shows that, and, again you can link to the section of notes where you wrote about seeds. You took some pictures of different seeds. You can drag links to them into the notes, the to-do and the calendar event. All three connect, with links.
You can always easily share links to any of these items with your friends. You can create and share new to-do lists per friend with links to selected jobs that will get ticked off in both places when done, as they’re just links to the same objects. One of your friends puts up their personal to-do list on their virtual kitchen wall, which maps in AR to their real kitchen.
On the train back, you’d like to create a social media post about your day, with photos and some text. The chat group list can be used to publish just to them, so you create a post for them. So you create the post in one social media list, but there are different people you also would like to reach in another list. You re-publish your post to the other list simply using its link. Now, the second post updates when you edit the first because of that link. You can get a link to a post in one place and drop it into a post in another, and it shows that post, embedded.
You stare out the window, remembering when you were a kid and used to make pinboards and scrapbooks, where you could create a theme or topic. Great that you can still do that virtually in your playroom with your notes, to-do lists and photos from today, with links for pins and glue!
You flip to your home automation to see that the temperature will be warm for your return. Since the home automation knew about today’s calendar event, it already worked that out for itself, because it has links to the calendar that you created in AR. You saw the controller panel in AR, and the calendar, then dragged a link from the event over to the controller.
Running the OS
Now, this Spatial Computing OS needn’t fill the room. In fact, it would still fit perfectly well into a handheld gaming device such as this:
Of course, this OS isn’t a game as such, and it could also render the same 3D virtual space onto the 2D screens of PCs and smartphones, or even smartwatches.
As a general-purpose operating system it would have to have either a physical keyboard or a touchscreen - or a virtual keyboard in VR/AR.
On power up or reboot, the device’s OS would put you right into the single, shared space, and everything from then on would happen in there.
Re-imagining Apple’s Spatial Computing
It’s time to consider retiring the old desktop metaphor still used in Apple’s Spatial Computing OS, where we have to juggle multiple applications.
Instead, imagine an app-free Spatial Computing OS, one where we can all work together within a single, open “parallel digital reality” - a 3D co-creation and thinking space manifest by all instances of this OS on devices across the planet - built from pervasive links between all of our shared digital property.
Note that I’ve been calling this operating system “OnexOS” in other articles. As this is a research project, all names are subject to change!
Subscribe right here to keep up-to-date with this project: