Scientists are looking into the virtual world

Developing a VR Software for Out-of-Body Experiences at a Biomedical Laboratories: A Case Study of Hilton

All of them reported that the virtual reality version of the video made them concentrate and absorb information better, when they were shown it on a computer screen and through a virtual reality headset.

He has also been experimenting with VR to study out-of-body experiences — a phenomenon estimated to be experienced by 10% of the global population, the reasons for which are not fully understood3.

Blanke hopes that understanding the field will lead to better treatment for people with neurological disorders. But there are still technological limitations. Users can feel nauseous if the headsets get heavy after a while. He says that it is always a challenge to get enough programmers. ”It would be great to have more software that was easily adaptable for non-programmers, too. However, scientists are a very small fraction of the market and so I do not see that changing any time soon.

It’s not easy to develop a virtual reality program. When Hilton was planning his own virtual lab — a project that started in 2000 but was spurred on by the pandemic in 2020 — he says he was “a bit giddy with hubris”. He adds: “Having developed our own 3D printing software and PC software for digital flow, we thought that we could easily make our own VR software, too. However, it proved harder than we had initially imagined.” It has taken many years for Hilton to make his software to a quality that is proud of, thanks to the help from his researchers and their 3D computer-aided design skills.

“We have around ten people in the lab with a mix of backgrounds — biomedical, pharmacy and chemistry degrees, but no computing — and we train everyone from the ground up. First in CAD design of real equipment, then AI programming and then CAD to VR. All [lab members] run the sessions with users, so they then get used to VR training of others,” says Hilton. It is not every lab that has time and resources.

Virtual reality in medicine: a virtual reality experience from a medical student living in South Africa as part of the DELTAS Africa research funding programme

Many regions do not have a fast Internet connection. In 2021, there were still 2.9 billion people without Internet, according to the United Nations. Even major cities in South Africa struggle with Internet access due to energy shortages, and rural areas have never received internet access. Ensuring all of the programs De Beers helps to develop are designed to work with mobile-phone data and don’t rely on broadband internet is a priority for him. But even this can be problematic, he says. Access to Space X’s low-latency satellite internet, called Starlink, has made this easier, as he said, for instance using satellites.

A lack of understanding around VR technologies also remains a fundamental barrier to wider adoption. For a lot of the professors, virtual reality was seen as weird and unnatural. For medical students, it’s really no big deal.

I spent six months at the University of Helsinki in 2018 as part of my PhD funding, which was provided by the DELTAS Africa programme. I was chatting with a student from India at the international science open day. It was something I had heard about but had never tried, and she explained to me how new technology was being used in her work to teach science to young people.

Virtual reality is a smart mirror: Identifying new molecules and generating artificial life forms at the university of Glasgow, UK, and the next few decades

He says virtual reality is a very smart mirror. “As adults, it’s very easy for us to decide when we’re brushing our teeth that we’re not the image we see in the mirror — it’s a reflection. But once you have an avatar, you can play around with lots of different perspectives. It allows us to ask questions of ourselves, that have not been asked before.

According to Cronin, his Glasgow-based spin-out company, Chemify, has raised more than £33 million from international investors who have backed his plan to automate the design of new molecules to speed up drug development.

He thinks that VR used in this way could have an important role in many other potentially dangerous scientific scenarios, including nuclear decommissioning and bomb-disposal research. And there are productivity benefits, too: “We need to invent more molecules if we’re going to solve big societal problems, like finding new drugs to target diseases. But chemical space is really hard to research,” he explains. We tried billions of molecules to get the 2,800 smallmolecules approved in the clinic. Cures for cancers are out there, but we don’t physically have the capacity to go through each chemical reaction one by one ourselves.”

The addition of the VR technology allows Cronin to move components and give instructions to the robots, wherever he is. “It also allows us to make shortcuts,” he explains. “I can be holding the handset and have a virtual conical flask in one hand. If I make a pouring motion, the robot knows to interpret that as moving liquid from hand a to hand b, and so it can turn on a pump and a valve. It’s not fully deployed yet, but the operating system is there,” he adds. “It’s very similar to the way a flight simulator works.”

In 2020, chemist Lee Cronin and his colleagues at the University of Glasgow, UK, designed a VR programme that allows him to control the robots in his lab remotely. Cronin and his research group use what he calls a “chemputer” — a series of robots that can be programmed to synthesize organic molecules automatically — as part of their quest to discover new molecules and generate artificial life forms. The chemical experiments are safer due to the automation and the robots working 24–7 every day.

Hilton’s lab is a digital one, experienced through virtual reality (VR) headsets, so the chaos being created is benign. But the environment mirrors the space and design of the real-life wet lab he runs in the School of Pharmacy at University College London — where his group researches synthetic chemistry, including drug discovery — albeit with brighter colours and some supernatural add-ons.

The Meta Quest 2 headset, which costs around US $380, is relatively low in cost and allows one to have multiple headsets on the go at the same time. The software can be used on a PC, enabling PC–headset interaction, PC–PC or headset–headset combinations. His virtual lab was built using the 3D design software Unreal Engine from US developer Epic Games, and runs on a local server for single users, switching to a cloud-based one for multiplayer and voice-connectivity programs.

XR: Experimenting with Face-to-Face with Real-Time Speech Recognition in Video-on-Realization in South Africa

He says that they have created a tool for the virtual assistants to speak ten foreign languages. “The knowledge base translates in real time, meaning I could speak directly in English to answer a student’s query, and they’ll hear the answer in their local language. We believe there is a huge opportunity for this in countries in the global south.

One of the simplest ways to make virtual training more accessible could be by avoiding spoken languages altogether. The ability to use gestures and body language without having to know the language at all is enabled by XR, according to a researcher in South Africa.

There are a lot of challenges to producing some videos in Africa, it’s a new technology and not many people have experience with it. What I really want is to be able to allow students to go into the lab to do their own experiments, even if we don’t have the budget or skill to do it, just because we can make the videos. Nevertheless, I hope to generate more interest and collaborate with others who are using these tools across Africa.

The virtual reality user used to not have to look or speak in a certain way. They can be shown as machines, without a need for gender or race, but they still have some challenges. For now, he says, his program uses robot characters that have masculine torsos.

Virtual Reality Education for Sepsis Recognition in Pregnant Women: A Project Sepsis Experience for Students from a Medical-Study Lab

Using medical dummies to teach students how to diagnose and treat sepsis is considered the gold standard for training, she explains, but the pandemic made in-person training difficult. It is very expensive and resources-rich due to the fact that it is only in specialist centres and the cost of the dummies is around £40,000. And if you happen to miss a day’s training because you’re sick, it’s tough luck,” she adds.

Sharma was inspired to change this in 2019 after witnessing pregnant people become dangerously ill from sepsis during labour. While a research fellow at Cardiff University, she helped to develop a VR education tool for sepsis recognition as part of Project Sepsis, a UK research collaboration across medicine and science. Students are shown a simulation of the differences between sepsis symptoms and similar conditions in a Virtual Reality project. Unlike Hilton’s avatar-populated virtual lab, Sharma opted to film actors playing out realistic medical scenes, which students can watch and interact with using VR headset and handsets. The students are then tested on the knowledge gained during the experience through on-screen multiple-choice tests.

I want to help young people across Africa understand what a researcher does. I hope that by showing and not just telling, VR will inspire more people to look into careers in the sciences. I hope that it will help the families to appreciate the role of science in society. Science is largely funded by the public, but the public doesn’t necessarily know what we do.

It was far from being a dream of my upbringing to work in a research lab. This is still the case for many children in my country, especially those from remote regions with few classroom resources.

A 360-degree video of the KEMRI labs for secondary-school students in Kenya: What we’ve learned and what we don’t

One of the first films I produced, in 2019, won me the 2022 Eric and Wendy Schmidt Award for Excellence in Science Communications, hosted by the US National Academies of Science, Engineering and Medicine, which was a career-defining moment.

The scientist explains to young people what science can do for society, using the latest technologies to do so.

The resulting videos can be uploaded to virtual reality headsets and shown to students without the need for a fast internet connection.

I worked with a production company to shoot six of the KEMRI lab areas: biobank, entomology, immunology, proteomics, genetics and the cleaning and reagent-preparation section. We were able to demonstrate some of the research we did and chat with some of the researchers.

My research team wanted to shoot a 360-degree video of the KEMRI-Wellcome Trust Research Programme labs. As part of the script-writing process, we invited secondary-school students to a guided tour of the labs, and asked them what they understood and whether they found it relatable to what they were learning in school.

Although it is one of the fastest growing economies in Africa, like many African countries, it faces challenges in science, technology, engineering and mathematics. A lot more needs to be done to increase the interest in these subjects.