Kolloquium WiSe 2023/24

Alexander Richard

Meta Reality Labs Research (RLR), Pittsburgh

Wann: Mo, 16.10.2022, 16:15 Uhr (dieses Semester geänderte Zeit!)

Wo: Konrad-Zuse-Hörsaal (Raum B-201)

Thema

Achieving Social Telepresence

Sprache: English

Abstract

These days, physical distance between people is one of the biggest obstacles to maintaining meaningful social relationships with family, friends, and co-workers. Even with today’s technology, remote communication is limited to a two-dimensional audio-visual experience and lacks the availability of a shared, three-dimensional space in which people can interact with each other over the distance. Our mission at Reality Labs Research (RLR) in Pittsburgh is to develop a telepresence system that is indistinguishable from reality, i.e., a system that provides photo- and phono-realistic social interactions in VR. Building such a system requires to combine representations of 3D spaces, lifelike avatars, and acoustics.

In this presentation, I will discuss the three major challenges to achieve such a system: completeness, usability, and scale. Completeness describes the challenge to build not only full-body avatars, but environments and acoustic models that can represent the full audio-visual experience. Usability refers to drivability of the avatars, particularly considering that users will not have access to a multi-camera setup, such that our avatars need to be animated from headset input only. Finally, scale refers to the problem of deploying this technology to everyone in the world, i.e., being able to generate a photorealistic representation of everyone from minimal amounts of data.

Bio

Alexander Richard is a Research Scientist at Reality Labs Research (RLR) in Pittsburgh leading the audio-visual research team. With his team, he concentrates on audio-visual learning to build photo- and phono-realistic immersive experiences in Virtual Reality that enable remote communication indistinguishable from reality. Combining computer vision, machine learning, and audio processing, he develops key technologies for audio-visual lifelike avatars and novel 3D rendering approaches for spatial and binaural audio. Before joining RLR, Alexander was a Speech Scientist at Amazon Alexa in Aachen, Germany. He received his PhD from the University of Bonn for his work on temporal segmentation of human actions in videos.