WING.NUS’s CS6101 lab section will be conducted as a public course, with class participants nominating themselves and presenting the materials and leading the discussion. In Semester I of AY2023/2024, we will focus on the topics of

Large Language Models (LLMs).

This course is based on materials from Percy Liang, Tatsunori Hashimoto and Christopher Ré’s CS324 Large Language Models course at Stanford; Danqi Chen’s COS 597G Understanding Large Language Models course at Princeton; and Ryan Cotterell’s Large Language Models course at ETH. All three sets of instructors have given their explicit permission to allow us to re-use and build upon their fantastic resources. There will be 14 reading sessions, which will be held from 16:00 pm to 18:00 pm on Fridays. On alternate Thursdays, 13:00-15:00 pm, we will have project consultation sessions.

Click on Details to see how to participate in the course.

Discussion Group. A mandatory discussion group is on Slack. Students and guests, please login when you are free. If you have a @comp.nus.edu.sg, @u.nus.edu, @nus.edu.sg, @a-star.edu.sg, @dsi.a-star.edu.sg or @i2r.a-star.edu.sg. email address you can create your Slack account for the group discussion without needing an invite.

If you need an invite to the Slack group. The Slack group is being reused from previous semesters. Once you are in the Slack group, you can consider yourself registered for the group.

It is not a lecture-oriented course and not as in-depth as the original source courses we are drawing from (with explicit permission). Hence, our course is not a replacement, but rather a class to spur local interest in the topics of large language models.

Please see the detailed schedule in the table.

Schedule

The sessions will be broadcast live via the customized Zoom link as advertised on the Slack group. For semi-privacy reasons, the group meeting ID and access is not publicly disseminated here on the webpage.

The schedule below is preliminary (last updated 18 Jul 2023.) We may make slight changes on the allocation for topics and provide alternative suggestions for papers.

DateDescription
NUS Week 00
Fri, 11 Aug
NUS Calendar (PDF)

Introduction and Orientation

Lecturers: Kan Min Yen, Michael Qizhe Xie

Lecture Slides ]

NUS Week 01
Fri, 18 Aug

What are Large Language Models?
Video intentionally truncated as lecturer's request.

Lecturers: Michael Qizhe Xie

Lecture Slides ] [ Scribe Notes ]

NUS Week 02
Fri, 25 Aug

Adaptation

Lecturers: Hengchang Hu, Shen Ting Ang, Xiao Xu, Yanxia Qin, Yajing Yang

Supporters: Arnav Aggarwal, Warren Low Keng Hoong, Lawrence Wong, Nicholas Wong

Lecture Slides ] [ Scribe Notes ] [ Summary ]

NUS Week 03
Tue, 5 Sep (special timing due to Polling Day)

Prompting and Zero-shot Inference

Lecturers: Esther, Victor Li, Richmond Sin, Suveen Ellawela, Lawrence Wong, Warren Low Keng Hoong

Supporters: Alson Jiang, Tongyao Zhu, Qijiong Liu, Yajing Yang, Yixi Ding

Lecture Slides ] [ Scribe Notes ]

NUS Week 04
Fri, 8 Sep

Representation Capacity

Lecturers: Wee Yen Zhe, Wong Jin Jye, Yisong Miao, Liew Wei Pyn, Yap Kim Thow

Supporters: Bhavitvya Malik, Disha Ponganti, Gan Kah Ee, Naomi Leow, Tan Khay Liang, Yanxia Qin

Lecture Slides ] [ Scribe Notes ]

NUS Week 05
Fri, 15 Sep

Modeling

Lecturers: Bhavitvya Malik, Yisong Miao, Yan Zehong, Markus Zhang, Agarwal Vatsalya

Supporters: Yanxia Qin, Cecilia Soh, Shen Ting Ang, Winston Ho

Lecture Slides ] [ Scribe Notes ]

NUS Week 06
Fri, 22 Sep

Scaling Up, Training and Parallelism

Lecturers: Arnav Aggarwal, Conrad Soon, Disha Ponganti, William Teo, Filbert Phang

Supporters: Yisong Miao, Hannah Brown, Jiaxiang Yu, Xiao Xu

Lecture Slides ] [ Scribe Notes ]

NUS Recess Week
Fri, 29 Sep

Preliminary Project Reports

NUS Week 07
Fri, 6 Oct

Data and Knowledge

Lecturers: Hannah Brown, Nic Wong, Yixi Ding, Yuxi Xie, Liangtao Lin, Khay Liang

Supporters: Victor Li, Liew Wei Pyn, Nicholas Chen, Guanzhen Li, Keyou Li, Zehong Yan

Lecture Slides ] [ Scribe Notes ]

NUS Week 08
Fri, 13 Oct

Reasoning

NUS Week 09
Fri, 20 Oct

Transfer Learning

Lecturers: Lawrence Wong, Yanxia Qin, Victor Li, Wei Soon Cheong

Supporters: Warren Low Keng Hoong, William Teo, Tan Yan Liong, Filbert Phang

Lecture Slides ] [ Scribe Notes ]

NUS Week 10
Fri, 27 Oct

Retrieval Based LLMs

Lecturers: Alson Jiang, Qijiong, Tongyao Zhu, Hu Hengchang, Winston Ho, and Jiaxiang Yu

Supporters: Miao Yisong, Wee Yenzhe, Conrad Soon, Agarwal Vatsalya, Sunil Kumar, Wei Soon Cheong

Lecture Slides ] [ Scribe Notes ]

NUS Week 11
Fri, 3 Nov

Multimodal LLMs

Lecturers: Do Xuan Long, Tan Yan Liong, Guanzhen Li, Deng Yuhong, Low Keng Hoong Warren, Tongyao Zhu, Zenghao Chai

Supporters: Yuxi Xie, Victor Li, Yixi Ding, Hengchang Hu, Ellawela Suveen Thinusha Bandara, Sherman Chann, Li Minzhi

Lecture Slides ] [ Scribe Notes ]

NUS Week 12
Fri, 10 Nov

Harms

Lecturers: Minzhi Li, Xinyuan Lu, Ibrahim Taha Aksu, Richmond Sin, Shivshankar Umashankar, Yuxi Xie, Cecilia Soh, Li Keyou, Tan Yi Ting

Supporters: Ng Hui Ling, Tiviatis Sim, Reuben Thomas, Guanzhen Li, Hannah Brown

Lecture Slides ] [ Scribe Notes ]

NUS Week 13
Fri, 17 Nov

Privacy and Memorization

Lecturers: Li Minzhi, Zhang Ze Yu, Hannah Brown, Yixi Ding, Ng Hui Ling

Supporters: Xinyuan Lu, Shivshankar Umashankar, Tan Yi Ting, Taha Aksu

Lecture Slides ] [ Scribe Notes ]
N.B. 23rd STePS is on Wed, 15 Nov

Organizers

This version of CS6101 is jointly run by WING@NUS and incoming faculty Michael Shieh

The current organisation team includes following members (alphabetical order):

Hengchang Hu, Min-Yen Kan, Xinyuan Lu, Yisong Miao Michael Shieh