Teaching

Language Technology Seminar: Deep Learning for NLP and Speech

Course Description

In this seminar, we discuss a language technology topic in depth. Topics change every year, past examples include crowdsourcing and unsupervised language processing. This year, the topic is applying deep learning for written natural language and speech processing.

Are you curious about state-of-the-art machine learning and text processing? And what about automatically recognising who you are or detecting if you are drunken from speech recordings? Want to find out what a microphone has to do with graphic cards?

Machine learning tasks such as classification often require input that is mathematically and computationally convenient to process. However, real-world data such as text, speech and images is usually complex, redundant, and highly variable. For these problems, good features are needed that capture relevant information. Traditional hand-crafted features often require expensive human labor and often rely on expert knowledge. However, in some cases, feature learning can be a viable alternative to hand crafted features. Since the resurgence of deep neural networks for feature learning, it has become part of many state-of-the-art systems in different disciplines, particularly that of computer vision, automatic speech recognition (ASR) and natural language processing (NLP).

You can chose one of many provided tasks for your experiment, and will be provided a dataset suited for both supervised and unsupervised feature learning. Example topics include text segmentation and classification, learning to generate text in a certain style or form, chat bots that can e.g. respond with smileys, speaker recognition by voice, speech classification e.g. recognizing that someone is drunken from speech recordings. You can also propose your own dataset, as long as it contains language in written or spoken form. Experiments can be conducted on the Lichtenberg High Performance Computing Cluster, which gives you access to powerful GPUs to accelerate your experiments.

This seminar is being held in the format of a scientific mini workshop: After an introductory lecture, individual topics are assigned. Introductory literature by topic, as well as a basic software environment and data sets will be provided. Students write a paper, consisting of a literature overview and a description of their own experiment. Papers are mutually peer-reviewed. In a final workshop, the work is presented in a 15-20 minute presentation. Outstanding papers will be submitted to national or international conferences.