The Virtual Harvard Project
Outside of the Harvard Museum of Natural History, a small group of people gather under a tent, talking quietly over their laptops. Pedestrians stroll by the freshly mowed lawns framing the walkway to the building, and students and faculty alike ascend and descend its concrete steps on their way to and from classrooms and labs. The scene is quiet, pastoral, normal.
But a walk through the museum’s unassuming red door to the right of the main entrance and up two flights of stairs reveals a wildly different — virtual — world. At the end of a hallway filled with offices and seismology posters sits a room illuminated by blue LED lights, its rightmost wall lined with mannequins wearing VR headsets. Rows of chairs sit in the middle of the room before a panoramic screen, currently displaying a view of Earth from outer space; behind them, a research assistant mans a table with three monitors, controlling the displays on the larger screen.
This is the Ultra High Resolution Science Observatory, the primary facility of the Visualization Research and Teaching Laboratory, headed by computer engineer and 3D artist Rus Gant. Gant has worked as director of Harvard’s Visualization Lab since 2012, before which he worked on initiatives at MIT and Carnegie Mellon, applying over 40 years of visualization skills to computer science, science education, archaeology, and museology. Currently, Harvard’s observatory houses one of Gant’s many projects: constructing a fully virtual model of the University.
Gant first conceived the idea to construct a virtual Harvard in the summer of 2019, inspired by past undertakings like the Giza Project — a Harvard-based digital archaeology initiative to build 3D reconstructions of the Giza Pyramids and surrounding artifacts — that focused on making education more engaging. When the pandemic struck months later, he decided that it was the time to turn his idea of making a 3D model of Harvard into a (virtual) reality. Over the course of the last 14 months, Gant and a member from his team, Sean O’Reilly, have created models of 115 out of 230 of Harvard’s main buildings, from Widener Library to the Harvard Museum of Natural History to Longfellow Hall.
“Quite honestly, it turned out to be really hard,” Gant says with a chuckle. “We knew it was going to take some time, but we just didn’t realize how much time.”
To generate each model, Gant and his team take panoramic photographs from various vantage points within each building. Taken from tripods controlled by an iPad using technology from Matterport — a company that enables users to create virtual 3D models of buildings — the photographs are then compiled into a dollhouse, a 3D model of the entire scanned building showing the general layout and the specific spaces one can access. Small circles that mark each camera point act as links to additional information about the building.
The second stage of the project involves transforming the models into digital classrooms. To do this, Gant and his team use Unreal Engine, a gaming program traditionally used for large multiplayer games, which allows users to fully immerse themselves in and navigate a virtual environment.
At this point, Gant and his team can use the SmartStage, a small platform placed before a full-body green screen. Those standing on the stage, like lecturers or tour guides, can be inserted into the virtual environment as the green screen displays a scanned model. The SmartStage, created by the media company White Light, was loaned to the Virtual Harvard project by the Verizon Innovation Lab.
However, Gant says getting permission to scan Harvard’s buildings has taken almost as much effort as the scanning itself. The project was initially met with resistance from administrators and lawyers as well as almost every department in the school, primarily because of privacy concerns.
“We had to get permission from everybody, every step of the way,” Gant says. “Each laboratory wants to control their own department. The security guys were like, ‘Well, this is a roadmap for terrorists.’ Museum guys [said], ‘Well, now they know where to steal things.’ But once we showed them the result, and we said, ‘You own this, nobody else can show it unless you say it’s okay,’ they were comfortable.”
After a month of discussions, Gant’s team finally received the green light for the project in August 2020, sponsored by the Vice Provost for Advances in Learning. From there, resistance transformed into gratitude as staff members found novel ways to utilize the virtual models outside the realm of education.
Senior planner Cara Noferi, who manages the FAS models, noticed that the models could be put to good use with RoomBook, the platform that faculty and students use to reserve rooms for meetings, as well as with Centerstone, the database of floor plans for the University.
By incorporating virtual models into RoomBook, students and professors can preview classrooms, seeing how much space they have and what type of equipment comes with each room. The images also come in handy for the maintenance and facilities workers who use Centerstone to locate specific building areas for tasks like fixing pipe breaks.
“For these two groups, this is phenomenal,” Gant says. “They never had pictures before. So now we’re just going to hand it to them, like, ‘Oh, here, the future just arrived. And it’s free!’”
Moving forward, Gant and his team want to finish scanning the remaining 115 buildings of Harvard’s main campus and use Unreal Engine to transform them into immersive virtual environments. From there, students would be able to join their virtual classes on Virtual Harvard using their computers, smartphones, and AR/VR glasses. For Gant, these efforts represent a way to not only revolutionize education, but chart an entirely new path in AR/VR technology.
“We’re trying to bring Harvard into this environment several years before everybody else,” Gant says. “This will be a future of both human- and machine-mediated teaching and learning, and we see Virtual Harvard as our first portal to this world.”
CORRECTION: October 25, 2021
A previous caption on this article misidentifed the developer of the SmartStage. It was developed by the media company White Light.
A previous version of this article misidentifed the future plans for use of Virtual Harvard. Students would be able to join virtual classes on Virtual Harvard using their computers, smartphones, and AR/VR glasses, not personal SmartStages.