Dr. Jae Hee Lee
Photo: Jae Hee Lee
Postdoctoral Research Associate TRR CML Project
Knowledge Technology
Address
Universität Hamburg
Faculty of Mathematics, Informatics and Natural Sciences
Department of Informatics
Knowledge Technology Research Group
Vogt-Koelln-Str. 30
22527 Hamburg
Office
Room: F-228a
Contact
Tel: +49 40 42883-2530
Fax: +49 40 4273-14634
ORCID: 0000-0001-9840-780X
Homepage: http://jaeheelee.de
About Me
I am a postdoctoral research associate in the Knowledge Technology Group, University of Hamburg. I develop representation learning models for grounded language understanding, with an emphasis on explainability and neuro-symbolic integration. My prior work includes symbolic approaches to AI, specifically spatio-temporal reasoning and multiagent systems.
Research Interests
- Multimodal Deep Learning
- Explainable Artificial Intelligence
- Neuro-Symbolic Artificial Intelligence
- Spatio-Temporal Reasoning and Learning
Research Experience
- Neurocognitive Models of Crossmodal Language Learning, University of Hamburg, Germany (2020–present)
- Formal Lexically Informed Logics for Searching the Web, Cardiff University, UK (2018–2020)
- Feodor Lynen Research Fellowship by Alexander von Humboldt Foundation, University of Technology Sydney, Australia (2016–2017)
- Artificial Intelligence Meets Wireless Sensor Networks, The Australian National University, Australia (2015)
- Reasoning about Paths, Shapes, and Configuration, University of Bremen, Germany (2009–2014)
Education
- Dr. rer. nat. in Computer Science, University of Bremen, Germany (2009–2013)
- Visiting PhD student, North Carolina State University, USA (2012)
- Visiting PhD student, University at Buffalo, USA (2011)
- Diplom in Mathematics, University of Bremen, Germany (2003–2009)
- DAAD exchange student, Seoul National University, South Korea (2008)
News
- Our paper “Visually Grounded Continual Language Learning with Selective Specialization” has been accepted to the Findings of EMNLP 2023.
- Program committee member, LREC-COLING 2024
- MSc student Björn Plüster, whom I advise, developed a German LLM called LeoLM with LAION and hessian.AI.
- Our paper Internally Rewarded Reinforcement Learning has been accepted to ICML 2023.
- I am co-organizing the 2nd International Workshop on Spatio-Temporal Reasoning and Learning.
- Program committee member, EMNLP 2023.
- Program committee member, ECAI 2023.
- Program committee member, ACL 2023.
- Program committee member, IJCAI 2023.
- Received ~10K € research funding for a student project on “Explainable Visual Question Answering” that I am advising. Check out our paper and code.
Recent Teaching Activities
- Lecture on “Transformers and Crossmodal Learning”, University of Hamburg (2023) [Slides]
- Supervisor, Neural Networks Seminar, University of Hamburg (2023)
- Organizer, WISDUM Reading Group, Knowledge Technology, University of Hamburg (2023)
- Supervisor, Bio-inspired Artificial Intelligence Seminar, University of Hamburg (2022)
Thesis Supervision
- Generalization of Transformer-Based Models on Visual Question Answering Tasks, Frederic Voigt, MSc (2023)
- Improving Compositional Generalization By Learning Concepts Individually, Ramtin Nouri, MSc (2023)
- Learning Concepts a Developmental Lifelong Learning Approach to Visual Question Answering, Ramin Farkhondeh, BSc (2022)
- Benchmarking Faithfulness: Towards Accurate Natural Language Explanations in Vision-Language Tasks, Jakob Ambsdorf, MSc (2022), now PhD student at University of Copenhagen
- Tackling The Binding Problem And Compositional Generalization In Multimodal Language Learning, Caspar Volquardsen, BSc (2021), ICANN 2022
- Learning Bidirectional Translation Between Robot Actions and Linguistic Descriptions, Markus Heidrich, BSc (2021)
- Using the Reformer for Efficient Summarization, Yannick Wehr, BSc (2020)
- Generalization in Multi-Modal Language Learning from Simulation, Aaron Eisermann, BSc (2020), IJCNN 2021
- Commonsense Validation and Explanation, Christian Rahe, BSc (2020)
Selected Publications
Full publication list available on DBLP, Google Scholar, and Semantic Scholar.
Multimodal Deep Learning:
- J.H. Lee, M. Kerzel, K. Ahrens, C. Weber, S. Wermter, What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning, IJCAI 2022. [Code]
- B. Plüster, J. Ambsdorf, L. Braach, J.H. Lee, S. Wermter, Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations, arXiv preprint, 2022. [Code]
- J.H. Lee, Y. Yao, O. Özdemir, M. Li, C. Weber, Z. Liu, S. Wermter, Spatial relation learning in complementary scenarios with deep neural networks, Frontiers in Neurorobotics, 2022. [Code]
- O. Özdemir, M. Kerzel, C. Weber, J.H. Lee, and S. Wermter, Language Model-Based Paired Variational Autoencoders for Robotic Language Learning, IEEE Transactions on Cognitive and Developmental Systems, 2022.
- C. Volquardsen, J.H. Lee, C. Weber, and S. Wermter, More Diverse Training, Better Compositionality! Evidence from Multimodal Language Learning, ICANN 2022.
- M. Li, C. Weber, M. Kerzel, J.H. Lee, Z. Zeng, Z. Liu, S. Wermter, Robotic Occlusion Reasoning for Efficient Object Existence Prediction IROS 2021. [Code]
- A. Eisermann, J.H. Lee, C. Weber, S. Wermter, Generalization in Multimodal Language Learning from Simulation, IJCNN 2021.
Explainable Artificial Intelligence:
- J.H. Lee, S. Lanza, S. Wermter, From Neural Activations to Concepts: A Survey on Explaining Concepts in Neural Networks, (submitted to Neurosymbolic Artificial Intelligence).
- B. Plüster, J. Ambsdorf, L. Braach, J.H. Lee, S. Wermter, Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations, arXiv preprint, 2022. [Code]
Neuro-Symbolic Artificial Intelligence:
- J.H. Lee, S. Lanza, S. Wermter, From Neural Activations to Concepts: A Survey on Explaining Concepts in Neural Networks, (submitted to Neurosymbolic Artificial Intelligence).
- X. Zhao, M. Li, W. Lu, C. Weber, J.H. Lee, K. Chu, S. Wermter, Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic, arXiv preprint, 2023.
- J.H. Lee, M. Sioutis, K. Ahrens, M. Alirezaie, M. Kerzel, S. Wermter, Neuro-Symbolic Spatio-Temporal Reasoning, in Compendium of Neurosymbolic Artificial Intelligence, IOS Press, 2023, pp. 410–429. [arXiv Preprint]
- J.H. Lee, Y. Yao, O. Özdemir, M. Li, C. Weber, Z. Liu, S. Wermter, Spatial relation learning in complementary scenarios with deep neural networks, Frontiers in Neurorobotics, 2022. [Code]
Other Machine Learning Topics:
- M. Li, X. Zhao, J.H. Lee, C. Weber, S. Wermter, Internally Rewarded Reinforcement Learning, ICML 2023. [Project page] [Code]
- J.H. Lee, J. Camacho-Collados, L. Espinosa Anke, S. Schockaert, Capturing Word Order in Averaging Based Sentence Embeddings, ECAI 2020. [Code]
- S. Kong, J. Bai, J.H. Lee, D. Chen, A. Allyn, M. Stuart, M. Pinsky, K. Mills, C. Gomes, Deep Hurdle Networks for Zero-Inflated Multi-Target Regression: Application to Multiple Species Abundance Estimation, IJCAI 2020.
Spatio-Temporal Reasoning and Learning:
- J.H. Lee, M. Sioutis, K. Ahrens, M. Alirezaie, M. Kerzel, S. Wermter, Neuro-Symbolic Spatio-Temporal Reasoning, in Compendium of Neurosymbolic Artificial Intelligence, IOS Press, 2023, pp. 410–429. [arXiv Preprint]
- J.H. Lee, M. Kerzel, K. Ahrens, C. Weber, S. Wermter, What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning, IJCAI 2022. [Code]
- J.H. Lee et al. Spatial relation learning in complementary scenarios with deep neural networks, Frontiers in Neurorobotics, 2022. [Code]
- S. Kong, J.H. Lee, S. Li, Multiagent Simple Temporal Problem: The Arc-Consistency Approach AAAI 2018.
- F. Dylla, J.H. Lee, T. Mossakowski, T. Schneider, A.V. Delden, J.V.D. Ven, D. Wolter, A Survey of Qualitative Spatial and Temporal Calculi: Algebraic and Computational Properties, ACM Computing Surveys, 2017.
- J.H. Lee, S. Li, Z. Long, M. Sioutis, On Redundancy in Simple Temporal Networks, ECAI 2016.
- D. Wolter, J.H. Lee, Connecting Qualitative Spatial and Temporal Representations by Propositional Closure, IJCAI 2016.
- X. Ge, J.H. Lee, J. Renz, P. Zhang, Trend-Based Prediction of Spatial Change, IJCAI 2016.
- S. Schockaert, J.H. Lee, Qualitative Reasoning About Directions in Semantic Spaces, IJCAI 2015.
- P. Zhang, J.H. Lee, J. Renz, From Raw Sensor Data to Detailed Spatial Knowledge, IJCAI 2015.
- J.H. Lee, The Complexity of Reasoning with Relative Directions, ECAI 2014.
- J.H. Lee, J. Renz, D. Wolter, StarVars: Effective Reasoning About Relative Directions, IJCAI 2013.
- D. Wolter, J.H. Lee, Qualitative reasoning with directional relations, Artificial Intelligence, 2010.
Multiagent Systems:
- S. Kong, J.H. Lee, S. Li, Multiagent Simple Temporal Problem: The Arc-Consistency Approach AAAI 2018.
- S. Kong, J.H. Lee, S. Li, A new distributed algorithm for efficient generalized arc-consistency propagation, Autonomous Agent Multi-Agent Systems 32, 2018.
- S. Kong, J.H. Lee, S. Li, A Deterministic Distributed Algorithm for Reasoning with Connected Row-Convex Constraints, AAMAS 2017.