Using Video Game Tech to Teach Field-Starved Geology Students

If you decide to pursue a degree in geology, be prepared to spend some time in the wilderness, where you will be asked to find and analyze rocks that will help teach you how the planet works. You will sketch curious outcrops, smash stone to pieces, peer at crystals through a hand lens, and, every now and then, even lick rocks, if it comes to that, all under the watchful, judging eye of your instructors.

When the pandemic kicked into gear back in March 2020, these both scintillating and stressful field schools were no more. Geology instructors across the world were at a bit of a loss as to what to do. Many understandably concluded that there was no way to replicate this hands-on learning experience and just made do, but Matthew Genge, a planetary scientist at Imperial College London (ICL), had an epiphany.

By happenstance, he had taken up the hobby of video game design a decade earlier. “It’s pure problem solving,” he says. “You get that achievement buzz when you make something work or overcome some challenge.”

One of his colleagues, fellow ICL geoscientist Mark Sutton, had also been dabbling in the same digital sandbox. So they decided to put their skills to pedagogical use: They built video game versions of the field trips their undergraduate students would normally go on, where they could practice the same techniques and learn about the planet in the same way they would in the real world.

It started with a 3D replica of Sardinia (and Mt. Etna on Sicily), where students galivanted about, looking for ancient fossils, prodding volcanic rocks, and exploring an abandoned silver mine. But like in all good video games, things escalated quickly. Before long, students were piloting spaceships, fending off hostile fighters, and trying to find a good place to land on an asteroid (to study its chemistry).

This innovative circumvention of the lockdown was an alternative to field geology, but also a place for students to socialize during the isolation of lockdown, and an escape from the usual logistical problems of fieldwork, from funding to time limitations to injuries.

Despite those challenges, geology students often report that their field work is one of the highlights of their university experience. When flights were grounded and lockdowns began, it was painfully clear that the students were not going to be able to head into the field. “Like everybody, we sat around in denial for a short period of time,” says Genge.

Trips abroad became virtual meetings, where students looked at digital photos while clicking through Google Street View. “There is some value in that, but it was so draining for the students,” Genge says. “It’s horrible. It was like rubbing their noses in it.” Nothing quite compares to being in the field, so Sutton and Genge began looking to their hobby for a way to simulate that. “I wanted them to have that [field] experience,” Genge says.

Back in 2019, Sutton had brought a drone to Sardinia—one of the usual field trip locations—and took a bunch of photographs of the places they were visiting to learn geology. A year later, Genge used those photographs, along with some bespoke computer code, to whip up a virtual version of the study area.

In the (real) field, the objective would be to examine a location, study it scientifically, pose a research question, and then attempt to answer it. The same scenario played out in the virtual world Genge and Sutton created.

For example, an area that was once a lake, 330 million years ago, is now jam-packed with plant and animal fossils. There are even ancient traces of rain, which made little indentations that have been naturally preserved. Some of these impressions are elongated in one direction, which can be used to estimate wind speed. A student might find these rain prints, examine them in high resolution, and then write something about how they might be used to understand what Earth’s atmosphere was like back then.

The students were engaged, and the quality of their work was similar to what the instructors had seen in previous field seasons. “Two of the projects were close to being publishable,” says Genge.

Normally, a human instructor would be around to help out, but that wasn’t possible with these single-player game worlds. In their place was a diminutive flying robot that followed the students around, guiding them to geologic curiosities. “I gave her quite a sassy personality,” Genge says. She teased the students if they looked clueless, and sometimes made references to Chris Hemsworth.

The goal was serious, but it was a game platform after all, and Genge and Sutton couldn’t resist throwing in some unexpected diversions. A precarious cliff edge in the bona fide Sardinia became, in the virtual version, a place to fling students into the sea, whereupon a shark chased them as they swam to a nearby island.

For the next version, Genge spent three weeks in the Scottish Highlands, driving around and taking loads of drone shots, which he used to recreate the landscape around the village of Kinlochleven, another pre-pandemic field trip destination. He made waterfalls, planted 30,000 trees, and (in an act of perhaps unnecessary fealty to reality) populated the hills with midges. His son Harry made the buildings—refuges from said midges.

By this point, there had been another development milestone: Sutton had finished a multiplayer version of the game. All the students could exist as avatars in the same space, communicate with their voices, and point at things, make measurements of the orientations and types of rock, and plot out bands of geology on a map. “And it made all the difference,” Genge says. “It suddenly became so much more real.”

As the students traversed the region, filling in their geological maps as usual, the instructors checked in on their progress. “I could tell it was effective, because the students behaved like students,” says Genge. Everyone had quad bikes, “so there was a certain amount of racing going on instead of mapping.” One student messaged him to politely ask how to get a quad bike out of a tree. And after the day’s work ended, students were using the digital Scottish dimension to just hang out.

In class, along came a unit on meteorites, a new addition to the syllabus. Genge had been worrying about how to keep those eight lectures engaging during pre-pandemic times: The department only had five meteorite samples between 30 students, which limited their individual exposure to hands-on instruction.

Fortunately, the virtual field trips provided an obvious solution. “Essentially, we went on this eight-week space adventure,” says Genge.

After an introductory lecture about distinguishing meteorites from regular rocks, the students were given quad bikes and told to find meteorites hidden across a vast desert. Several of the fragments had come from a single meteor that had exploded in the atmosphere, scattering its parts like cosmic shotgun pellets. Could the students find these related pieces of debris and piece the puzzle together?

As they conducted their detective work, a planet with Saturn-like rings slowly rose above the horizon. Some of the more exploration-minded students wandered off to find an impact crater with a damaged spacecraft inside. As they perused the wreckage, one student asked why it had gun turrets. “Well, space is a dangerous place,” Genge replied.

Inspired by a galaxy far, far away, Genge says, the spacecraft was the beginning of a “stupidly ambitious idea”—a fully navigable region of space for subsequent lectures. After Genge ironed a few bugs, students were put in the pilot’s seat, able to fly the (now fully repaired) spaceship to various asteroids to understand meteorites at their origin.

In-game infrared sensors could be used to assess the mineral compositions of asteroid samples, buggies could be used to drive about and drill things, and on-board laboratories had microscopes for additional analysis. Lectures were delivered via virtual screens inside the spacecraft. It was a huge improvement over soulless video meetings, where students were reluctant to speak up. Inside a spaceship, though, their avatars jostled to see the screen amid cacophonous chatter.

Most students and instructors quickly learned how to play the games, but a small fraction struggled. A few kept crashing head-on into asteroids. “I had to set them homework to learn how to fly the spaceship,” says Genge. By the time the turrets were used to fend off AI-controlled enemy fighters and even blow up an enemy base—all in the name of science, of course—most had mastered the controls.

Genge looks forward to the time when real field schools will be safe to run again. But the video games are now part of the permanent syllabus. They can help prime students for future field trips by offering practice, and provide a field experience for students who are physically unable to take the trips, such as those who use wheelchairs. And the meteorite-themed lectures turned out to be more effective than anything based exclusively in reality.

Genge has high hopes for the next iteration: a lecture hall with his students, each wearing a VR headset that allows each one to individually explore the increasingly madcap 3D environments packed with a real geological education. “We will all be there, together,” he says, “looking like complete lunatics.”