WING.NUS’s CS6101 lab section will be conducted as a public course, with class participants nominating themselves and presenting the materials and leading the discussion. In Semester I of AY2023/2024, we will focus on the topics of
Large Language Models (LLMs).
This course is based on materials from Percy Liang, Tatsunori Hashimoto and Christopher Ré’s CS324 Large Language Models course at Stanford; Danqi Chen’s COS 597G Understanding Large Language Models course at Princeton; and Ryan Cotterell’s Large Language Models course at ETH. All three sets of instructors have given their explicit permission to allow us to re-use and build upon their fantastic resources. There will be 14 reading sessions, which will be held from 16:00 pm to 18:00 pm on Fridays. On alternate Thursdays, 13:00-15:00 pm, we will have project consultation sessions.
Click on Details to see how to participate in the course.
Discussion Group. A mandatory discussion group is on Slack. Students and guests, please login when you are free. If you have a @comp.nus.edu.sg, @u.nus.edu, @nus.edu.sg, @a-star.edu.sg, @dsi.a-star.edu.sg or @i2r.a-star.edu.sg. email address you can create your Slack account for the group discussion without needing an invite.
If you need an invite to the Slack group. The Slack group is being reused from previous semesters. Once you are in the Slack group, you can consider yourself registered for the group.
It is not a lecture-oriented course and not as in-depth as the original source courses we are drawing from (with explicit permission). Hence, our course is not a replacement, but rather a class to spur local interest in the topics of large language models.
Please see the detailed schedule in the table.
Schedule
The sessions will be broadcast live via the customized Zoom link as advertised on the Slack group. For semi-privacy reasons, the group meeting ID and access is not publicly disseminated here on the webpage.
The schedule below is preliminary (last updated 18 Jul 2023.) We may make slight changes on the allocation for topics and provide alternative suggestions for papers.
| Date | Description |
|---|---|
| NUS Week 00 Fri, 11 Aug NUS Calendar (PDF) | Introduction and Orientation Lecturers: Kan Min Yen, Michael Qizhe Xie [ Lecture Slides ] |
| NUS Week 01 Fri, 18 Aug | What are Large Language Models? Lecturers: Michael Qizhe Xie [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 02 Fri, 25 Aug | Adaptation Lecturers: Hengchang Hu, Shen Ting Ang, Xiao Xu, Yanxia Qin, Yajing Yang Supporters: Arnav Aggarwal, Warren Low Keng Hoong, Lawrence Wong, Nicholas Wong [ Lecture Slides ] [ Scribe Notes ] [ Summary ] |
| NUS Week 03 Tue, 5 Sep (special timing due to Polling Day) | Prompting and Zero-shot Inference Lecturers: Esther, Victor Li, Richmond Sin, Suveen Ellawela, Lawrence Wong, Warren Low Keng Hoong Supporters: Alson Jiang, Tongyao Zhu, Qijiong Liu, Yajing Yang, Yixi Ding [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 04 Fri, 8 Sep | Representation Capacity Lecturers: Wee Yen Zhe, Wong Jin Jye, Yisong Miao, Liew Wei Pyn, Yap Kim Thow Supporters: Bhavitvya Malik, Disha Ponganti, Gan Kah Ee, Naomi Leow, Tan Khay Liang, Yanxia Qin [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 05 Fri, 15 Sep | Modeling Lecturers: Bhavitvya Malik, Yisong Miao, Yan Zehong, Markus Zhang, Agarwal Vatsalya Supporters: Yanxia Qin, Cecilia Soh, Shen Ting Ang, Winston Ho [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 06 Fri, 22 Sep | Scaling Up, Training and Parallelism Lecturers: Arnav Aggarwal, Conrad Soon, Disha Ponganti, William Teo, Filbert Phang Supporters: Yisong Miao, Hannah Brown, Jiaxiang Yu, Xiao Xu [ Lecture Slides ] [ Scribe Notes ] |
| NUS Recess Week Fri, 29 Sep | Preliminary Project Reports |
| NUS Week 07 Fri, 6 Oct | Data and Knowledge Lecturers: Hannah Brown, Nic Wong, Yixi Ding, Yuxi Xie, Liangtao Lin, Khay Liang Supporters: Victor Li, Liew Wei Pyn, Nicholas Chen, Guanzhen Li, Keyou Li, Zehong Yan [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 08 Fri, 13 Oct | Reasoning |
| NUS Week 09 Fri, 20 Oct | Transfer Learning Lecturers: Lawrence Wong, Yanxia Qin, Victor Li, Wei Soon Cheong Supporters: Warren Low Keng Hoong, William Teo, Tan Yan Liong, Filbert Phang [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 10 Fri, 27 Oct | Retrieval Based LLMs Lecturers: Alson Jiang, Qijiong, Tongyao Zhu, Hu Hengchang, Winston Ho, and Jiaxiang Yu Supporters: Miao Yisong, Wee Yenzhe, Conrad Soon, Agarwal Vatsalya, Sunil Kumar, Wei Soon Cheong [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 11 Fri, 3 Nov | Multimodal LLMs Lecturers: Do Xuan Long, Tan Yan Liong, Guanzhen Li, Deng Yuhong, Low Keng Hoong Warren, Tongyao Zhu, Zenghao Chai Supporters: Yuxi Xie, Victor Li, Yixi Ding, Hengchang Hu, Ellawela Suveen Thinusha Bandara, Sherman Chann, Li Minzhi [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 12 Fri, 10 Nov | Harms Lecturers: Minzhi Li, Xinyuan Lu, Ibrahim Taha Aksu, Richmond Sin, Shivshankar Umashankar, Yuxi Xie, Cecilia Soh, Li Keyou, Tan Yi Ting Supporters: Ng Hui Ling, Tiviatis Sim, Reuben Thomas, Guanzhen Li, Hannah Brown [ Lecture Slides ] [ Scribe Notes ] |
| NUS Week 13 Fri, 17 Nov | Privacy and Memorization Lecturers: Li Minzhi, Zhang Ze Yu, Hannah Brown, Yixi Ding, Ng Hui Ling Supporters: Xinyuan Lu, Shivshankar Umashankar, Tan Yi Ting, Taha Aksu [ Lecture Slides ] [ Scribe Notes ] |
Organizers
This version of CS6101 is jointly run by WING@NUS and incoming faculty Michael Shieh
The current organisation team includes following members (alphabetical order):
Hengchang Hu, Min-Yen Kan, Xinyuan Lu, Yisong Miao Michael Shieh