Ruder 2019 – The 4 Biggest Open Problems in NLP
1 Natural language understanding
1.1 program synthesis
- underlying NLU are the logical form representations of language
- let's try predicting them directly
1.2 embodied intelligence
- with massive compute, could place agent in the real world and try to get it to learn language from the ground up
- in the meantime, can ground in simulated environment
1.3 inductive bias
- related to the above, what sort of priors should we build into the model
1.5 cog sci
- build approaches inspired by brain science
2 NLP for low resource languages
2.1 universal language model
2.2 cross lingual representations
- aligning word embeddings for different languages
3 reasoning about large/many documents
- what does unsupervised language modelling look like for very long documents?
- how do life long learning and memory work for language models
4 datasets and evaluation
Created: 2021-09-14 Tue 21:44