Dr. Jae Hee Lee

About Me

I am a postdoctoral research associate in the Knowledge Technology Group, University of Hamburg.

My research aims to develop deep learning models for language understanding by leveraging multimodal information (e.g., vision, proprioception) with a particular focus on robustness and explainability.

News

• Program committee member, ACL 2023
• Program committee member, IJCAI 2023
• Received ~10K € research funding for a student project on “explainable visual question answering” that I am advising. Check out our paper!

Blog Posts

• Deep Learning Courses and Books for Beginners

Research Interests

• Multimodal Deep Learning
• Explainable Artificial Intelligence
• Neuro-Symbolic Artificial Intelligence
• Spatial Reasoning

Research Experience

• Neurocognitive Models of Crossmodal Language Learning, University of Hamburg, Hamburg, Germany (2020–present)
• Formal Lexically Informed Logics for Searching the Web, Cardiff University, Cardiff, UK (2018–2020)
• Feodor Lynen Research Fellowship by Alexander von Humboldt Foundation, University of Technology Sydney, Sydney, Australia (2016–2017)
• Artificial Intelligence Meets Wireless Sensor Networks, The Australian National University, Canberra, Australia (2015)
• Reasoning about Paths, Shapes, and Configuration, University of Bremen, Bremen, Germany (2009–2014)

Education

• Dr. rer. nat. in Computer Science, University of Bremen, Germany (2013)
• Diplom in Mathematics, University of Bremen, Germany (2009)

Selected Recent Papers

(Full publication list available on Google Scholar and DBLP)

• J. H. Lee, M. Kerzel, K. Ahrens, C. Weber, S. Wermter. What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning, IJCAI 2022. [PDF] [Code]

• J. H. Lee, M. Sioutis, K. Ahrens, M. Alirezaie, M. Kerzel, S. Wermter. Neuro-Symbolic Spatio-Temporal Reasoning, arXiv preprint, Nov. 28, 2022 (to appear in A Compendium of Neuro-Symbolic Artificial Intelligence, IOS Pres). [PDF] [Code]

• J. H. Lee et al. Spatial relation learning in complementary scenarios with deep neural networks, Frontiers in Neurorobotics vol. 16, 2022. [PDF]

• B. Plüster, J. Ambsdorf, L. Braach, J. H. Lee, S. Wermter. Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations, arXiv preprint, Dec. 08, 2022. [PDF] [Code]

Recent Teaching Activities

• Lecture on “Transformers and Crossmodal Learning”, University of Hamburg (2022)
• Supervisor, Bio-inspired Artificial Intelligence Seminar, University of Hamburg (2022)
• Supervisor, Neural Networks Seminar, University of Hamburg (2022)
• Organizer, WISDUM Reading Group, Knowledge Technology, University of Hamburg (2022)

Thesis Supervision

• Learning Concepts a Developmental Lifelong Learning Approach to Visual Question Answering, Ramin Farkhondeh, BSc (2022)
• Benchmarking Faithfulness: Towards Accurate Natural Language Explanations in Vision-Language Tasks, Jakob Ambsdorf, MSc (2022), now PhD student at University of Copenhagen
• Tackling The Binding Problem And Compositional Generalization In Multimodal Language Learning, Caspar Volquardsen, BSc (2021), ICANN 2022 [PDF]
• Learning Bidirectional Translation Between Robot Actions and Linguistic Descriptions, Markus Heidrich, BSc (2021)
• Using the Reformer for Efficient Summarization, Yannick Wehr, BSc (2020)
• Generalization in Multi-Modal Language Learning from Simulation, Aaron Eisermann, BSc (2020), IJCNN 2021 [PDF]
• Commonsense Validation and Explanation, Christian Rahe, BSc (2020)