research reading list
1 Resources
2 NLP
2.1 ASR
2.2 general surveys
2.3 community
- How can we improve peer review in NLP – 2020
- An Adversarial Review of “Adversarial Generation of Natural Language” – Yoav Goldberg blog post from 2017
2.4 generalization
2.5 structured prediction
- Optimal Neural Program Synthesis from Multimodal Specifications – 2020 paper from TAUR lab
- [course] Structured Prediction – Yoav Artzi
- Learning Differentiable Programs with Admissible Neural Heuristics – 2020 shah20_learn_differ_progr_with_admis_neural_heuris
2.6 automata
2.7 multi-modal
- Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer – 2021
- VideoBERT: A Joint Model for Video and Language Representation Learning – 2019
2.8 grounding
- [course] Language Grounding to Vision and Control – Katerina Fragkiadaki
2.9 machine translation
2.10 attention
- LambdaNetworks: Modeling Long-Range Interactions Without Attention
- Attention is Not Explanation – 2019 jain19_atten_is_not_explan
2.11 language model fine tuning
3 Planning/Navigation
3.1 Transformers
3.2 compositional planners
3.3 NLU
3.4 seq2seq
- Sequence-to-Sequence Model for Trajectory Planning of Nonprehensile Manipulation Including Contact Model
- Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments – 2018
4 ML
4.1 math
- Parr and Howard 2018 – The Matrix Calculus You Need for Deep Learning parr18_matrix_calcul_you_need_deep_learn
- The Modern Mathematics of Deep Learning
- Deep Learning Goodfellow and Bengio
- Math for Deep Learning – Faisal et al
4.2 intersection with functional programming
- Backprop as Functor:A compositional perspective on supervised learning – 2021
- HaskTorch – Justin Le
4.3 probabilistic programming languages
4.4 opinion
- Machine Learning: The Great Stagnation – Mark Saroufim
4.5 machine learning
4.6 GANS
5 Linguistics
- [course] Computational Semantics – Ellie Pavlick
- 2021 – Comprehension of computer code relies primarily on domain-general executive brain regions ivanova20_compr_comput_code_relies_primar
- A case for deep learning in semantics – 2018
6 Data Science
7 Probability and Stats
- [course] Random – random variables and stats
- Bayesian Epsitemology – In particular, principle of conditionalization
8 HCI
- Joseph Chang – HCI at CMU
9 AI
9.1 TODO On the Opportunities and Risks of Foundational Models – Bommasani et al 2021
9.2 TODO The Scaling Hypothesis
9.3 TODO On the Measure of Intelligence
10 Research practice
11 robotics
- modern robotics – northwestern textbook and course
12 Theory
12.1 programming languages and logic
14 Bib
Bibliography
- [shah20_learn_differ_progr_with_admis_neural_heuris] Shah, Zhan, Sun, , Verma, Yue, Chaudhuri & Swarat, Learning Differentiable Programs With Admissible Neural Heuristics, CoRR, (2020). link.
- [jain19_atten_is_not_explan] Jain & Wallace, Attention Is Not Explanation, CoRR, (2019). link.
- [parr18_matrix_calcul_you_need_deep_learn] Parr & Howard, The Matrix Calculus You Need for Deep Learning, CoRR, (2018). link.
- [ivanova20_compr_comput_code_relies_primar] Anna A Ivanova, Shashank Srikant, Yotaro, Sueoka, Hope H Kean, Riva Dhamala, Una-May, O'Reilly, Marina U Bers & Evelina Fedorenko, Comprehension of Computer Code Relies Primarily on Domain-General Executive Brain Regions, eLife, 9(nil), nil (2020). link. doi.