Web Interfaces for Language Processing Systems SS2020
In the summer 2020, 6 groups collaborate to build different NLP components to the BigBlueButton web conferencing system. One of the team was the winner of the won EXPO 2020
Team one: BigBlueButton video conference transcription and keyword lookup extension
Team
- Fabian Rausch and Felix Welter
Project overview
- The Related Items project enables users to transcribe speech and lookup related items (e.g. lecture slides or a wikipedia article) via an easy to use interface from within a BigBlueButton conference. For that purpose BigBlueButton was connected to the speech recognition software Kaldi. The transcribed text is available to all participants in the meeting. Every client can select the transcription language for itself. This allows the possibility to read the transcribed text in the users preferred language while the current speaker speaks in a different language. Futhermore, information sources (called indices) were connected to BigBlueButton, therefore enabling all users to search for terms that have been mentioned during the talk quickly.
- Detail documentation is available here
Demo:
Team two: Automated Speech Recognition in BBB
Team
- Frederik Wille
Projecr Overview
- The main goal of this project was to integrate a speech recognition pipeline into the BigBlueButton Conference System. It further allows to display recognized speech as closed captions in BBB's HTML5 client.
- Detail documentation is available here
Demo
Team three: bbb-summarizer
Team
- Hoa Josef Nguyen
Project overview
- The project could be renamed, because currently it is an alignment of meeting transcription and its corresponding presentation files, but not an actual summarization. But still, this prototype can be further developed and the alignment can be a good start-point for creating good summaries. That is the goal and therefore, I think, that the name should stay. It implies the goal and its needed steps to reach it.
- Prototype
Demo
Team four: Taking notes in BigBlueButton
team
- Mozzam Motiwala
project overview
- Step1: Read all the messages from redis channel and put them into a text file
- Step2: Read content of the text file and pass it to Disfluency Removal Module
- Step3: save the result: text without disfluency in another text file
- Step4: Read the content of this new file and pass it to Punctuation Add Module
- Step5: The output of ‘Add Punctuation’ module is again a text file that contains Necessary punctuation like ‘.’ ‘?’ ‘!’. This is the final output.
- More here and here
Demo
Team five: Personal voice & chat assistant
Team
- Niklas von Boguszewski
Project overview
- Extension
- This project is about integrating an extension in the bigbluebutton (open source web conferencing) application which makes it possible to either chat or speak with a personal assistent by communicating with a Natural Language Understanding API in English.
- Details
- API
- This project is about building an API for a personal assistant, e.g. a bot in web conferencing systems which receives a text from the client and responds with a JSON containing pre-defined intents, entities and their confidences.
- In other words if you want an API which can make sense out of the string "Hey my friend <bot_name> mute Christian for me please" and identifies that Christian should be muted here then you are at the right address.
- Project details
Demo
Team six: On/Offline Meeting Summarization
Team
- Soniya Vijayakumar
Project overview
- Backend
- Summarizer Model: Transformer model that summarizes the meeting transcripts
- Persistence: Knowledge base created to persist the transcript and summary of each meeting
- Frontend
- Client that renders the summary as per user “refresh” request
- HTTPS request/response to obtain summaries as per the meeting ID
- PDF Download option during/post meeting
Demo