The growing interest in Augmented Reality (AR) and the renaissance of Virtual Reality (VR) have opened new approaches and techniques on how professionals interact with medical imagery, plan, train, perform surgeries, and help people with special needs in Rehabilitation tasks. Indeed, many medical specialties already rely on 2D and 3D image data for diagnosis, surgical planning, surgical navigation, medical education, or patient-clinician communication. However, the vast majority of current medical interfaces and interaction techniques continue unchanged. At the same time, the most innovative solutions have yet to unleash the full potential of VR and AR. This is probably because extending conventional workstations to accommodate VR and AR interaction paradigms is not free of challenges. Notably, VR and AR-based workstations, besides rendering complex anatomical data in interactive frame rates, must promote proper anatomical insight and boost visual memory through seamless visual collaboration between professionals. Moreover, they should free interaction from being seated at a desk (e.g., using a mouse and keyboard) to adopt non-stationary postures and freely walk within a workspace. They must also support a fluid exchange of image data and 3D models, fostering interesting discussions to solve clinical cases. Additionally, VR and AR-based techniques must be designed according to sound human-computer interaction principles. Since it is well known that medical professionals can resist changes in their workflow. In this talk, I will survey recent approaches to healthcare, including diagnosis, surgical training, planning, etc. I will discuss challenges, techniques, and principles in applying Extended Reality in these contexts and outline opportunities for future research.

Joaquim Jorge holds a Ph.D. in Computer Science from Rensselaer Polytechnic Institute (NY). A Full Professor at IST, the School of Engineering of Universidade de Lisboa (UL), Portugal, he teaches Virtual&Augmented Reality (VR/AR), Computer Graphics (CG), and HCI. Joaquim holds the UNESCO Chair on AI&XR and leads the Graphics & Interaction Research Group at INESC-ID, an Institute Affiliated with UL, having supervised 17 doctoral theses and 60+ MSc dissertations. A (co-)author of 380+ peer-reviewed publications, including 66 journal articles, and four Springer/Nature books (latest in 2021), he is among the top 2% of scientists worldwide (according to Stanford University).

Joaquim Jorge is an accomplished public speaker and lecturer. For many years he has developed a consistent collaboration with national and international educational and research institutions. He has been a Distinguished Invited Professor at PUC-Rio since February 2021 and was an Adjunct Professor at the University of Calgary from 2006-2018. Joaquim Jorge has been an ACM Distinguished Speaker (2015—), proffered 20 keynote Talks, taught six Tutorials, and participated in eight panels at International Conferences. He also lectured 24 invited Classes/seminars and presented 46 invited lectures at international Universities.

He organized IEEE VR2021 and VR2022 and the 2021 IEEE/ACM Presidential panel on the Future of Computing, featuring leaders from both organizations. He served as conference chair or co-chaired the Scientific Program Committee of 35+ international conferences, including IEEE/VR, IEEE/EG/Eurovis, IEEE/AIVR, ACM IUI, and Eurographics, and served in the international program committee of 200+ conferences, including IEEE/VR, IEEE/ISMAR, IEEE/MIPR, and IEEE/ICIR.