Generating drone footage in nonoptimal visual conditions in UnityBachelor Thesis

24 May 2024, by Konstanze Reupert

The use of digital twins (DT) for 3D simulation, particularly of different lighting conditions and weather effects, is key to unlocking the promising possibility of generating artificial footage inside such a twin instead of having to carry out the expensive and time-consuming collection of data in the real world. Not only could this be more efficient, but it can sometimes even be the only reasonable alternative if you take drones as an example which can fly safely only up to a certain degree of impairment through adverse weather. From the DT however, you can get an infinite number of any image at any time which is not possible in the real world.

Our main concern is to generate synthetic data from our DT to examine how visual conditions like weather and light influence the detection of containers in the context of the project InteGreatDrones. This project aims at transforming two inland container ports into more sustainable ones by reducing emissions there with the help of drones and DTs which can be taken as an example for port environments in general.

A lot of research has been done in the field of DTs, drones, object detection, and the 3D modeling of weather effects. However, none of them has explored all this in combination and above all in the specific context we want to investigate.

Our contribution will be a DT in Unity that can simulate different weather and lighting conditions as well as a synthetic dataset created inside our digital 3D environment. We will then use this afterward to train an object detection model to improve its performance in adverse weather conditions.

Supervised by:
Prof. Dr. Janick Edinger, Tim Rolff