Natural Language Processing: Projects

Language Modeling

Can we build a state-of-the-art language model for Slovak, using some of the new tricks (like massive pre-training and then fine-tuning) which can then be used in all the other tasks?

Since we already have all these Camembert's, BERTje and what not, can we build some sort of other language variation of BERT (like say BERToSlav)?

Text Classification

Pick any interesting text dataset, the text of which can be classified into various categories and try a couple of methods.

• Language identification
• Sentiment analysis of Anketa comments
• Detection of inappropriate comments in Anketa
• Adaptation of Label-wise attention to other non-Twitter data (or to new Twitter data)
• Applications of Active Learning in context of classification

Key Phrase Extraction

Given a body of text (say a document or a news article), can we extract the most important phrases out of it (and hence help summarize it a bit)?

Various approaches do exist (many of the popular ones are implemented as part of the pke module) -- could you think of a dataset where something like this could be useful? Could we perhaps build one automatically (with the expected keywords attached)?

Summarization

Can we build a Slovak/Czech/Croatian/Serbian/[any other language] summarization dataset that would allow us to "compress" news articles into a smaller list of sentences?

Can we use the TextRank algorithm to perform extractive summarization, or generate a list of keywords for a given body of text? Can you think of some interesting text for which this method would be a good fit?

Portmanteau Generation

Given words A and B, can we create their portmanteau C?

Or better yet, given the words A and B can we create a portmanteau C that would not directly feature A or B but would still be related? Such as for instance

Here is a quick demo of such approach, which requires a pretty complex setup and external tools, such as the CMU Pronouncing Dictionary, which makes it pretty difficult to port this approach for different languages.

Is there something we can do to work around that? Some neural approach perhaps?

Shared tasks

Shared tasks are essentially "academic Kaggle": you get a task, some data and produce a model that tries to do well on it. During the evaluation period, you normally produce a prediction on the text set. It's a relatively straightforward way of going

• CheckThat which focuses on detecting check-worthy claims and related tasks
• ADoBo: Automatic Detection of Borrowings (English terms borrowed to Spanish language)
• ProtestNews at CASE @ ACL-IJCNLP 2021 which is trying to put together a system for detecting mentions of protests in news text.
• MultiLexNorm which is about normalizing all sorts of text (slang and what not) to the "canonical form" (that's what most standard NLP systems expect).
• Sentence End and Punctuation Prediction in NLG Text: turns out predicting the end of sentence and missing punctuation is far from a solved problem.

Your own idea!

Feel free to come up with an idea on your own -- if you are working on something NLP-related for your thesis, that would be a good candidate. But in general, I'd be happy to talk about any NLP-related idea you may have.

Alternatively feel free to check out the sites below, find a NLP task you find interesting and see if you can make an interesting project out of it!

• https://nlpprogress.com/
• https://paperswithcode.com/area/nlp
• https://www.stateoftheart.ai/?area=Natural%20Language%20Processing