Advanced General NLP Topics II (AY 2020/2021, Semester I (2010))

Lecturer and Support staff: Tongyao Zhu, Zijin Kong, Tianyang Zhang, Chenglei Si

[Lecture Slides] [Scribe Notes]

Reading List:

• Sutskever, Vinyals, & Le. (2014) “Sequence to sequence learning with neural networks.” : https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf [ Video Segment @ 29:55]
• Gehring, et al. (2017) "Convolutional sequence to sequence learning." : https://arxiv.org/abs/1705.03122 [ Video Segment @ 53:12 ]
• Vaswani, et al. (2017) "Attention is all you need." : https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf [ Video Segment @ 1:19:35 ]

Lecturer and Support staff: Lu Xinyuan, Abdul Waheed, Taha Aksu, Xiachong Feng, Yu Fan Cai, Liangming Pan, Jia Qing Yap, Laiba Mehnaz, Taru Jain

[Lecture Slides] [Scribe Notes]

Reading List:

• Decoding, Beam Search : https://arxiv.org/abs/1409.3215 [Video Segment @ 2:05]
• Attention Mechanism : https://arxiv.org/abs/1409.0473 https://arxiv.org/abs/1706.03762 [ Video Segment @ 12:59]
• CopyNet : https://arxiv.org/pdf/1603.06393.pdf [ Video Segment @ 44:15]
• Sequicity : https://www.aclweb.org/anthology/P18-1133.pdf [ Video Segment @ 1:09:47]

Lecturer and Support staff: Pan Liangming, Mingliang Ang, Wu Jiaying, Ding Xu, Zhang Jipeng, Khush Jammu, Brian Formento, Tongyao Zhu, Li Chuang, Rabiul Awal.

[Lecture Slides] [Scribe Notes]

Reading List:

• SeqGAN : https://arxiv.org/abs/1609.05473 [Video Segment @ 4:33]
• Improving Sequence-to-Sequence Learning via Optimal Transport : https://arxiv.org/abs/1609.05473 [Video Segment @ 44:58]
• The Curious Case of Neural Text Degeneration : https://arxiv.org/abs/1904.09751 [Video Segment @ 1:04:43]
• MASS: Masked Sequence to Sequence Pre-training for Language Generation : https://arxiv.org/abs/1905.02450 [Video Segment @ 1:28:05]
• Plug and Play Language Models : https://arxiv.org/abs/1905.02450 [Video Segment @ 1:50:18]

Lecturers and Support Staff: Liangming Pan, Xinyuan Lu, Shreyas Kuthanoor Prakash, Judith Jeyafreeda, Lin Xu, Hengchang Hu, Alexandre Gravier, Romain Iehl, Ming Liang Ang, Jipeng Zhang.

[Lecture Slides] [Scribe Notes]

Reading List:

• Wiseman et al. (2018) "Learning Neural Templates for Text Generation" https://arxiv.org/abs/1808.10122 EMNLP'18 [ Video Segment @ 5:43 ]
• Koncel-Kedziorski et al. (2019) "Text Generation from Knowledge Graphs with Graph Transformers" https://arxiv.org/abs/1904.02342 NAACL'19 [ Video Segment @ 53:26 ]
• Zhang, Xu, and Wang (2019) "Pretraining-Based Natural Language Generation for Text Summarization" https://arxiv.org/abs/1902.09243 CONLL'19 [ Video Segment @ 1:29:00 ]
• Chen et al. (2020) "Distilling Knowledge Learned in BERT for Text Generation" https://arxiv.org/abs/1911.03829 ACL'20 [ Video Segment @ 1:45:46 ]

Lecturers and Support Staff: Abhinav Ramesh Kashyap, Devamanyu Hazarika, Alexandre Gravier, Rabiul Awal, Yisong Miao, Zhang Ruixi, Ding Xu, Lin Xu.

[Lecture Slides] [Scribe Notes]

Reading List:

• Ganin et al. (2016) "Domain-Adversarial Training of Neural Networks" https://arxiv.org/abs/1505.07818 JMLR'16 [ Video Segment @ 1:25 ]
• Keneshloo et al. (2019) "Deep Transfer Reinforcement Learning for Text Summarization" https://arxiv.org/abs/1810.06667 SDM'19 [ Video Segment @ 1:11:20 ]
• Howard et al. (2018) "Universal Language Model Fine-tuning for Text Classification" https://arxiv.org/abs/1801.06146 ACL'18 [ Video Segment @ 1:46:44 ]

Lecturers and Support Staff: Zhang Ruixi, Laiba Mehraz, Zhang Yifan, Khush Jammu, Brian Formento, Taha Aksu, Abhinav Ramesh Kashyap, Shreyas Kutanoor Prakash, Bhavitvya Malik.

[Lecture Slides] [Scribe Notes]

Reading List:

• Devlin et al. (2019) "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" https://arxiv.org/abs/1810.04805 [ Video Segment @ 0:00:08 ]
• Sanh et al. (2020) "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter" https://arxiv.org/abs/1910.01108 [ Video Segment @ 0:09:03 ]
• Lan et al. (2020) "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations" https://arxiv.org/abs/1909.11942 ICLR'20 [ Video Segment @ 0:09:56 ]
• Radford et al. (2018) "Improving Language Understanding by Generative Pre-Training" https://openai.com/blog/language-unsupervised/ [ Video Segment @ 0:44:20 ]
• Radford et al. (2019) "Language Models are Unsupervised Multitask Learners" https://openai.com/blog/better-language-models/ [ Video Segment @ 0:47:43 ]
• Brown et al. (2020) "Language Models are Few-Shot Learners" https://arxiv.org/abs/2005.14165 [ Video Segment @ 0:50:33 ]
• Houlsby et al. (2019) "Parameter-Efficient Transfer Learning for NLP" https://arxiv.org/abs/1902.00751 PMLR'19 [ Video Segment @ 1:17:14 ]
• Yang et al. (2017) "Transfer Learning for Sequence Tagging with Hierarchical Recurrent Network" https://openreview.net/pdf?id=ByxpMd9lx ICLR'17 [ Video Segment @ 1:49:52 ]

Lecturers and Support Staff: Shuang Gao, Anab Maulana Barik, Si Chenglei, Li Xinze, Zhao Zirui, Devamanyu Hazarika, Wu Jiaying, Feng Xiachong and Fiona Tan.

[Lecture Slides] [Scribe Notes]

Reading List:

• Yadav et al. (2020) "Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering" https://arxiv.org/abs/2005.01218 [ Video Segment @ 0:03:47 ]
• Lewis et al. (2019) "Unsupervised Question Answering by Cloze Translation" https://arxiv.org/abs/1906.04980 [ Video Segment @ 0:41:29 ]
• Li et al. (2020) "Harvesting and Refining Question-Answer Pairs for Unsupervised QA." https://arxiv.org/abs/2005.02925 ICLR'20 [ Video Segment @ 1:13:51 ]
• Asai et al. (2020) " Logic-Guided Data Augmentation and Regularization for Consistent Question Answering" https://arxiv.org/abs/2004.10157 [ Video Segment @ 1:41:08 ]

Lecturers and Support Staff: Alex Foo Da Weng, Taha Aksu, Luong Quoc Trung, Yang Yajing, Xu Yiqing, Hiray Shruti Shivram, Yifan Zhang, Samson Tan, and Liu Jiahao.

[Lecture Slides] [Scribe Notes]

Reading List:

• Cao et al. (2020) "Unsupervised Domain Adaptation on Reading Comprehension" https://arxiv.org/abs/1911.06137 [ Video Segment @ 0:07:10 ]
• Wang et al. (2019) "Adversarial Domain Adaptation for Machine Reading Comprehension" https://arxiv.org/abs/1911.06137 [ Video Segment @ 0:33:46 ]
• Puri et al. (2020) "Training Question Answering Models From Synthetic Data" https://arxiv.org/abs/1911.06137 [ Video Segment @ 0:57:45 ]
• Liu et al. (2020) "Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus" https://arxiv.org/abs/1911.06137 [ Video Segment @ 1:21:28 ]

Lecturers and Support Staff: Henghchang Hu, Victor Li, Jiaqi Zheng, Jiahao Liu, Hansheng Ren, Zijin Kong, Yajing Yang, Yiqing Xu, Sunil Kumar Yadav, Yiding Ran.

[Lecture Slides] [Scribe Notes]

Reading List:

• Introduction to Conversational Recommender Systems [ Video Segment @ 0:06:45 ]
• Christakopoulou et al.(2018) "Q&R: A Two-Stage Approach toward Interactive Recommendation"http://alexbeutel.com/papers/q-and-r-kdd2018.pdf [ Video Segment @ 0:12:35 ]
• Zou et al. (2020) "Towards Question-based Recommender Systems"https://arxiv.org/pdf/2005.14255.pdf [ Video Segment @ 0:32:21 ]
• Lei et al. (2020) "Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems"https://dl.acm.org/doi/pdf/10.1145/3336191.3371769 [ Video Segment @ 1:00:11 ]
• Zeng et al. (2020) "Dynamic Online Conversation Recommendation"http://www4.comp.polyu.edu.hk/~jing1li/publication/zeng2020dynamic.pdf [ Video Segment @ 1:38:38 ]
• Chen, Z. (2020) "Towards Explainable Conversational Recommendation."https://www.ijcai.org/Proceedings/2020/0414.pdf [ Video Segment @ 1:59:31 ]

Lecturers and Support Staff: Sunil Kumar Yadav, Hannan Cao, Cai Yu Fan, Yisong Miao，Xinyuan Lu, Anab Maulana Barik, Jiaqi Zheng, Hansheng Ren, Zitai Chen.

[Lecture Slides] [Scribe Notes]

Reading List:

• Zhang et al.(2020) "Conversational Contextual Bandit: Algorithm and Application"https://arxiv.org/pdf/1906.01219.pdf [ Video Segment @ 0:04:20 ]
• Lei et al.(2020) "Interactive Path Reasoning on Graph for Conversational Recommendation"https://arxiv.org/pdf/2007.00194.pdf [ Video Segment @ 0:50:15 ]
• Liu et al.(2020) "Towards Conversational Recommendation over Multi-Type Dialogs"https://arxiv.org/pdf/2005.03954.pdf [ Video Segment @ 1:18:10 ]
• Chen et al.(2019) "Towards Knowledge-Based Recommender Dialog System"https://www.aclweb.org/anthology/D19-1189.pdf [ Video Segment @ 1:49:50 ]

Lecturers and Support Staff: Samson Tan, Hiray Shruti Shivram, Fiona Tan, Bhavitvya Malik, Abdul Waheed, Judith Jeyafreeda, Li Xinze, Alex Foo Da Weng

[Lecture Slides] [Scribe Notes]

Reading List:

• Suresh and Guttag (2019) "A Framework for Understanding Unintended Consequences of Machine Learning"http://arxiv.org/pdf/1901.10002.pdf [ Video Segment @ 0:00:35 ]
• Bolukbasi et al. (2016) "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings"https://proceedings.neurips.cc/paper/2016/file/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf [ Video Segment @ 0:45:50 ]
• Maudslay et al. (2019) "It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution"https://www.aclweb.org/anthology/D19-1530/ [ Video Segment @ 1:18:28 ]
• Saunders and Byrne (2020) "Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem"https://www.aclweb.org/anthology/2020.acl-main.690/ [ Video Segment @ 1:50:52 ]

Lecturers and Support Staff: Zitai Chen, Zirui Zhao , Samson Tan, Shuang Gao, Hannan Cao, Laiba Mehnaz

[Lecture Slides] [Scribe Notes]

Reading List:

• Liang et al. (2020) "Towards Debiasing Sentence Representations"https://www.aclweb.org/anthology/2020.acl-main.488.pdf [ Video Segment @ 0:06:13 ]
• Wang et al. (2020) "Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation"https://arxiv.org/pdf/2005.00965.pdf [ Video Segment @ 0:41:00 ]
• Blodgett et al. (2020) "Language (Technology) is Power: A Critical Survey of `Bias` in NLP"https://arxiv.org/pdf/2005.14050.pdf [ Video Segment @ 1:14:14 ]

[ 17TH STEPS Program]