Deep Learning for Natural Language Processing

By completing the course, participants will obtain the knowledge and skills to solve a wide range of applied problems in Natural Language Processing. To achieve this goal, the participants will get to know successful methods for solving sub-problems, such as text representation, information extraction, text mining, word sense disambiguation, language modeling, similarity detection, and text summarization. The participants will understand the conceptual requirements of specific NLP tasks and be able to devise approaches to address these tasks in practice. The approaches studied in this course are centered on neural network architectures such as recurrent neural networks, sequence-to-sequence, and transformers. The participants will be able to assess the strengths and limitations of state-of-the-art NLP approaches and propose solutions for interdisciplinary NLP problems.

The lecture will cover the following topics:

  • Basics: Text representation, text processing, regular expressions, tokenization, stemming, lemmatization 
    • Bag-of-Words, weighting schemes (e.g., tf-idf), language modeling 
    • Language models, N-grams, perplexity, information gain, smoothing 
  • Word sense, lexical databases, distance measures 
  • Applications and tasts: document classification, text summarization, named entity recognition,  word sense disambiguation
  • Word embeddings and dense vector representations models: word2vec, GloVe, fastText, paragraph-vectors, multi-sense embeddings 
  • Basics on neural networks, feed-forward networks 
  • Activation functions, cost function, gradient descent, regularization, backpropagation
  • Neural language models, recurrent neural networks, vanishing gradients 
  • Architectures used in NLP: LSTM, GRU, biLSTM, Seq2Seq, Attention, CNN, Transformers 
  • Bias and ethics in NLP

Requirements

  • Knowledge of at least one object-oriented programming language, preferably Python, is required.
  • Python is used as part of the course. For participants who are unfamiliar with Python, we provide a free introduction course covering all essential topics required.
  • Basic knowledge of neural networks is desired to participate in this course. We recommend that participants are familiar with basic neural network architectures, hidden layers, activation functions, derivatives, classification, training and test strategies, precision, recall, backpropagation, gradients, and other foundational topics in machine learning and artificial neural networks. For participants who are unfamiliar with these topics, an integrated and fast-paced introduction focused on the use case of natural language processing will be provided. At the University of Göttingen’s computer science department, the course B.Inf.1236 Machine Learning provides an excellent foundation for this course. 

Exam (TBD)

  • Successful completion of programming projects (includes teaser, intermediate, and final presentation) (50%)
  • Final exam (oral or written) (50%)

Time schedule

Day Time Periodicity Duration Room Type
Tue 10:15 – 11:45 weekly 11.04.2023 – 14.07.2023 TBD lecture
Tue 12:15 – 13:45 weekly 11.04.2023 – 14.07.2023 TBD exercise
view on uni-vz

The exercise sessions will focus on individual programming projects (teamwork is possible) that will address complex information retrieval tasks.

Using the programming language Python and presenting the intermediate and final results of the projects is mandatory.