Before the metaverse, IVAS goggles aim to give troops augmented reality information flows. (Graphic by Breaking Defense, original IVAS photo via DVIDS, original Chicago skyline by Rohan Gangopadhyay via Unsplash)
SOMEWHERE BETWEEN REALITY AND A VIRTUAL EXISTENCE: In December, a small office within the stoic Air Force bureaucracy hosted a meeting with some 250 people, gathered in a conference room with the usual whiteboards, sticky notes and yellow folders.
But the conference room didn’t exist, and the attendees were hundreds of miles apart, spread from the United States to Japan, all wearing Oculus headsets.
With that meeting, visitors entered into the beating heart of the explosive, if uncertain, hype-cycle of the metaverse, a concept that has percolated for decades but was brought fully into the mainstream last year when Facebook rebranded as Meta.
Now, the metaverse, essentially interconnected virtual worlds likely accessed through virtual or augmented reality, is gripping the consumer world and expanding. The NBA’s Brooklyn Nets have filed trademarks for “Netaverse” and plan to broadcast games in the virtual worlds of the metaverse. MLB’s reigning champion Atlanta Braves announced a metaverse version of its ballpark. Major consumer companies like Walmart and Nike have announced plans to cash in on the concept, whether the public wants it or not.
“It’s not clear that you’re even going to get really widespread adoption of this idea in the next year or two. Everyone’s talking about it, [but] I’d say right now the metaverse is a corporate fad, not a user fad,” said Palmer Luckey, founder of defense startup Anduril Industries and the creator of the Oculus, perhaps the most popular VR headset. Eventually, however, “the metaverse is definitely going to happen.”
A passionate chorus of opinions argues that the metaverse is a fundamental advancement that will be central to the future of the human race; those voices run into a similarly fervid choir of skeptics arguing that companies are just chasing a tech craze that has either already existed for decades without fancy branding or one that no consumer will actually want.
The problem is that neither side, nor any of the corporate firms cowing about their metaverse businesses, seems to have an actual definition of what the term means.
“It’s everything and anything, which means it’s nothing,” said Jennifer McArdle, an adjunct senior fellow at the Center for a New American Security and head of research at Improbable U.S. Defense & National Security, a distributed simulation software company.
But the hype and hope of the metaverse isn’t restricted to the civilian world, and is slowly inching into the Pentagon among leaders both eager to show the military’s technology-forward stance and for those seriously studying the tech’s usefulness.
An Army one-star gave a detailed answer on the history of the term metaverse at a conference in December. Contractors boast about augmented and virtual reality technologies that will bring the metaverse to the military. Military simulated training investments are characterized as metaverse research. And that Air Force office dubbed its virtual reality meeting as an early foray into the metaverse — accompanied by NFTs.
To understand the future of the military and the metaverse, Breaking Defense spoke with key Pentagon officials, outside experts and representatives from industry. While there are varying levels of enthusiasm, there is a growing agreement that as long as the military enters the virtual world with clear (if augmented) eyes, it could greatly benefit American warfighters in ways ranging from immersive combat planning to hyper-realistic virtual training to truly experiencing weapons systems in ways that have never before been possible.
But as with technological leaps in the past, if the metaverse can’t avoid becoming an empty buzzword thrown at every problem by leaders who don’t fully grasp the concept, the Pentagon could end up wasting millions of dollars chasing a virtual dream.
“This really does force us to open the aperture of analysis when it comes to defense, not just in terms of how these virtual worlds could theoretically impact battlefield effectiveness, but also in terms of how the military provides for and supports its warfighters as a bureaucratic and social organization,” McArdle said.
Has The Military Already Been Playing In The Metaverse?
Years before the word “metaverse” existed, the Pentagon was experimenting with the broad concept of interconnected virtual worlds. In 1978, Air Force Capt. Jack Thorpe published a paper outlining a web of networked simulators for distributed mission planning. Years later, the Defense Advanced Research Projects Agency picked up the project, dubbed Simulator Networking (SIMNET), which later transitioned to the Army.
“If a metaverse is simply a series of interconnected virtual worlds, you could argue that there’s been very kind of clunky metaverses in the military since the 90s,” McArdle said.
The term “metaverse” stems from the 1992 novel “Snow Crash,” a dystopian science fiction novel by Neal Stephenson in which the main character straps on a VR headset and enters the “metaverse,” where his avatar lives, works, parties and engages in the occasional sword fight.
The military’s metaverse could include the experience of flying fighter jets, like these F-35s. (Graphic by Breaking Defense, original F-35 photo via DVIDS, original Chicago skyline by Rohan Gangopadhyay via Unsplash)
Today, discussion about the metaverse focuses on online gaming or virtual worlds that users can essentially “live” in, primarily used as a social arena. Fundamentally, as other technology-focused trade publications have written, the “metaverse” is a shared space that does not disappear when a user removes their VR headset.
But plenty of confusion exists with no clear-cut, resolute definition: What is a metaverse versus the metaverse? Fundamentally, can there be multiple metaverses?
Or for the military, if a soldier enters a single virtual training environment, is that considered the metaverse or is it simply training in virtual reality? What if that VR experience is connected to another virtual environment, then is it a metaverse? And if a soldier doesn’t persistently “live” in that environment, is it a (the?) metaverse at all? And on it goes.
“The reality is that almost nothing that anyone is calling the metaverse today is at all analogous to what people in the VR industry have been referring to the metaverse as for decades now,” Luckey told Breaking Defense, adding it’s “crazy” to call any digital space accessed by virtual reality the “metaverse.”
“Companies are calling their thing ‘this is our metaverse.’ That’s like saying ‘this is our internet.’ It’s like, no, that’s not how this works,” Luckey said. “The metaverse is an all-encompassing term. You don’t have a metaverse. And if you have a multiplayer game, that’s not a metaverse. That’s a game, like, we’ve had these for a very long time.”
The military, he said, has been more sober when it comes to the metaverse than commercial firms.
“In the military, I actually haven’t seen the same crazy nonsense I’ve seen in the corporate world. People in the military understand how VR can solve their problems they’ve been thinking about for 20, 30,40 years,” he said. “As the tech advances, they’re able to use it for more and more things.”
Luckey predicted that “real-life quality VR” is probably 10 to 15 years away, while the ability to accurately simulate the feeling of an activity such as surfing is probably 30 to 40 years out. But there are a number of efforts across the Defense Department attempting to build virtual environments for testing, training and experimentation, and have been for years.
In 2014, the Office of Naval Research and DoD-affiliated Institute for Creative Technologies at the University of Southern California showcased Project BlueShark, a virtual reality project that demonstrated a virtual world that allowed sailors to drive a ship with 3D situational awareness, repair ships while collaborating with the ship designer from far away, and command and control forces. Stories on the project note that users could transfer their view from the bridge of the ship to a UAV flying overhead, look in all different directions, as well as virtually host others, thousands of miles away, to discuss tactics and operations.
“That’s pretty metaverse-y by modern standards and they were working on it a long time ago before any of this,” said Luckey, who worked on virtual reality projects at USC’s ICT before Oculus.
Project BlueShark was mid-2010s experiment in the early metaverse. (U.S. Navy photo by John F. Williams/Released)
The “most perfect” concept of a military metaverse, as McArdle put it, would seamlessly stitch together the underpinnings of soldiers’ virtualized environments for training, education, experimentation and social life, with the final product being “this rich data that seamlessly moves between these virtual worlds that really kind of provides this holistic insight of the individual person,” though she added that “it’s going to be very hard to get there.”
Refining data exchanges between those “worlds” to where information could be passed between them could fundamentally change the relationship between war games, experiments such as Project Convergence, and training. Rather than one-off events, these could merge into something akin to a permanent training event that builds day after day.
“You could have data and information flow across each of these events so you’re iterating on it in a far more effective way,” McArdle said. “Now, if you could then take all that information that you’re iterating on as you’re developing new concepts of operation, or new tactics, techniques, and procedures — all that could then immediately be fed into a training environment, where you’re training your warfighters to use those new concepts of operation or those new TTPs immediately and there’s not a large gap between it.”
Beyond The Hype, Unexpected Future Uses?
For Air Force Materiel Command’s Digital Transformation Office (DTO), charged to drive innovative technology adoption inside the Air Force, the December meeting was about exploring what’s in the realm of possibility, with officials trying to prove the value that a fully immersive meeting could have to a military service with personnel spanning the globe.
“I will admit that our first meeting was very much more towards a VR meeting than, like, truly unleashing the power of the metaverse,” said Vince Pecoraro, lead program manager for the DTO, adding later that “so much stuff happens in the metaverse that the Air Force doesn’t have the ability to tap into because we don’t play in it that often.”
More than 250 individuals joined the online event, which leveraged the DTO Metaverse to engage participants in a virtual reality ecosystem designed to drive innovative thinking and digital-first thought strategies across the community. (Air Force Materiel Command/ DVIDS)
Take, for example, Non-Fungible Tokens, the oft-maligned blockchain-based digital asset that “represents” a real-world object, such as art. Well, the DTO is creating a challenge coin NFT, a digital asset users can cherish in their virtual space. And while the concept may make skeptics roll their eyes, the virtual worlds of the metaverse could theoretically provide nontraditional incentives to appeal to a different generation of warfighters.
“We could also, theoretically, be providing new recruitment incentives via the metaverse,” McArdle said. “So beyond bonuses or the GI Bill, you know, we’re seeing new forms of financial transactions and virtual goods like NFT’s emerging. And theoretically, the metaverse could provide opportunities for us to think about that from an incentive standpoint.”
This is a generation of gamers, after all, and video game companies have proven how far players will go (or how much they’ll pay) for even cosmetic virtual rewards.
Beyond NFTs, as metaverse technologies mature, experts said the military would benefit from the fully-immersive meetings the metaverse could provide, from battlefield operation centers to sustainment missions.
For example, mission planning, typically fraught with powerpoints and documents, could be updated so senior decision-makers could meet with planners in the metaverse and run through different courses of action, experts said. Operation centers could be lined with charts, data feeds, live video and a physical map or table showing a planned operation. Today, if a commander or high-level personnel drop into that physical meeting by phone or VTC, they may not be able to see all that information.
They “could start to see second- and third-order effects. And, again, they could start to fill in some of the nuances that get lost in the way those assets are transferred today,” a senior technology company executive who does business with the government told Breaking Defense on condition of anonymity.
But the overarching question still remains then: why run events in the murky technological waters of the virtual worlds instead of video teleconference? The answer depends on how it’s used.
Kyle Hurst, chief of the DTO, said he wouldn’t use their virtual meeting space for regularly scheduled staff meetings. And Luckey, asked what the benefits would be of doing the interview with Breaking Defense in virtual reality, responded “probably not all that much.”
“But if you’re talking about replicating the experience of going to another country, sitting down in a meeting room under fluorescent lights, and then talking to some people for a few hours, and then you shake their hands at the end… VR is going to absolutely dominate that world very, very quickly,” Luckey said. “The efficiencies are just so good.”
The metaverse is useful for meetings where hosts “need [attendees] to see and feel and touch the stuff that I’m talking about,” Pecoraro said.
The metaverse could combine assets from troops to air support in virtual training and combat simulations. (Graphic by Breaking Defense, original Blackhawk photo via DVIDS, original Chicago skyline by Rohan Gangopadhyay via Unsplash)
The immersive nature of the technology could allow military leaders dispersed across the world to come together to discuss the development of weapons systems or other platforms and experience them, similar to Project BlueShark. That could be useful for maintainers of military equipment, for example. Using the virtual collaboration space of the metaverse, maintainers could “phone a friend” for assistance if they need help.
“Say you need help with that task at hand, somebody else can then come in and aid you,” said Bob Kleinhample, vice president of immersive technologies in SAIC’s digital practice. “Now you have people collaborating in this virtual world as you’re working or maintaining a system.”
Or as the industry executive said, military officials could experience weapons systems in a unique way.
“You could feature that. You could see it, you could fly it, you could look at the performance of it,” the executive said. “Your ability to interact with it becomes much richer. You think about, like, a picture’s worth 1,000 words, the video clip’s worth a million words — well, how much is it worth to be able to literally experience it based on anywhere you are?”
Synthetic Training: A Metaverse Test Case
Perhaps the most obvious use case for interconnected virtual worlds in the military is synthetic training. With more soldiers dying in training year over year than in combat, it’s a tool that could save lives.
In Orlando, Fla., Army Brig. Gen. William Glaser, director of the service’s Synthetic Training Environment Cross-Functional Team, leads an office trying to build the Army’s Synthetic Training Environment (STE), a technology that, depending on the definition, could be considered a metaverse based on its existence as a virtual world that connects to others.
The STE is a virtual training environment to complement live training and simulate combat scenarios across Army formations from platoons and higher on any type of terrain soldiers may face, from urban combat to mountain fighting. According to the Army website it brings “together live, virtual, and constructive training environments into a single STE” and will provide training functions to “ground, dismounted and aerial platforms and command post at the points of need.”
The synthetic training environment is “essentially a metaverse that can be interconnected with the metaverse,” Glaser said — exactly the kind of phrase critics point to when arguing the metaverse is goofy, but also not wrong depending on, once again, how the metaverse is defined. (Glaser is no stranger to the metaverse itself, calling Snow Crash a “great, great book.”)
“If you’re simply falling back on that definition — that it is really a series of interconnected virtual worlds — well, then, theoretically, you could make the case that it is a metaverse,” McArdle said of Glaser’s program. “Now, a lot of people are gonna disagree with that because they’re gonna want to see a lot more in it.”
For Luckey, the STE is more like a multiplayer game than it is a metaverse, because “the metaverse is more like a city and games are more like games…drawing that line is pretty tough though.” The definition of the metaverse is murky enough that Luckey harkened back to the old Supreme Court ruling on obscenity: “I’ll know it when I see it.”
The Army’s had many iterations of virtual training environments, but the synthetic training environment aims to weave several training realities into one joint construct. The idea of having interconnected virtual worlds highlights the limitations of older synthetic training systems. Those are monolithic, built without the intent of integrating new models, and hard to update.
An airman trains with virtual reality, one potential use of the metaverse for the military. (Graphic by Breaking Defense, original soldier photo via DVIDS, original Chicago skyline by Rohan Gangopadhyay via Unsplash)
For example, if a service wants to integrate a new cyber or electronic warfare simulation into a current synthetic trainer, that new simulation has to be built to seamlessly interact with every other simulation or model already built into the environment, a time-consuming and tedious process.
“Fundamentally, they don’t change as the character of warfare changes. So, I mean, essentially, for synthetic training to get to the point where we have these like, you know, seamlessly interoperable virtual worlds, we’re gonna have to move towards building these environments in a way that are far more modular and composable,” McArdle said.
Mission rehearsal is a “capstone” of where the metaverse could enhance soldier training, Glaser said. Soldiers would be immersed in a digital environment of the desert, mountains or High Plains, each with different tactics, techniques and procedures, where soldiers can continuously rehearse missions. For example, virtual training simulating the terrain of Iraq or Afghanistan ahead of deployment could be more beneficial than training in the swampy wetlands of Ft. Stewart, Ga.
“This gives us the opportunity to immerse soldiers into an environment where they might actually be operating in, vice where they’re actually training,” Glaser said.
And while it seems unlikely that — right now — a soldier would have a persistent virtual presence that follows them around, it could provide for more accurate training. More data flow could allow soldiers to train against the performance characteristics of an enemy. Or, if the soldier’s virtual presence was uploaded with their matching physical profiles, it could provide for more realistic training that could highlight the physical limitations of a soldier and their squads.
“If it makes sense to have a persistent digital persona that, let’s say, matches your real-world physical characteristics so that in all the different simulations and training tools, you are at your correct level of performance relative to the other people you’re working with — if that’s the right way to solve the problem, I’m confident the military will do it,” Luckey said.