CS224N 学习计划
进度: 0/22 (0%)
讲座 + 作业
0/ 23
论文阅读
0/ 12
周进度 Checklist
Week 1 (Jan 6-10)(0/3)
- L01 History of NLP — Jan 6
- L02 Word Vectors — Jan 8
- A1 Word Vectors 发布 (Jan 6)，开始做
Week 2 (Jan 13-17)(0/3)
- L03 Neural Nets — Jan 13
- L04 RNN & LM — Jan 15
- •A1 Due (Jan 15)
- A2 Neural Nets 发布 (Jan 13)，开始做
Week 3 (Jan 20-24)(0/3)
- L05 Transformers — Jan 21
- L06 Final Project — Jan 23
- •A2 Due (Jan 22)
- A3 Transformers 发布 (Jan 22)，开始做
Week 4 (Jan 27-31)(0/2)
- L07 Pretraining — Jan 27
- L08 Post-training — Jan 29
Week 5 (Feb 3-7)(0/3)
- L09 PEFT — Feb 3
- L10 RAG & Agents — Feb 5
- •A3 Due (Feb 5)
- A4 LLM Evaluation 发布 (Feb 5)，开始做
Week 6 (Feb 10-14)(0/2)
- L11 Evaluation — Feb 10
- L12 Reasoning Part 1 — Feb 12
- •Project Proposal Due (Feb 10)
Week 7 (Feb 17-21)(0/2)
- L13 Reasoning Part 2 — Feb 18
- L14 Tokenization & Multilinguality — Feb 20
- •A4 Due (Feb 19)
Week 8 (Feb 24-28)(0/2)
- L15 Interpretability — Feb 24
- L16 Social Impact — Feb 26
- •Project Milestone Due (Feb 26)
Week 9 (Mar 2-6)(0/2)
- L17 Multimodality — Mar 3
- L18 LoRA Without Regret — Mar 5
Week 10 (Mar 9-16)(0/1)
- L19 Open Questions — Mar 10
- •Final Report Due (Mar 12)
- •Poster Session (Mar 16)
核心论文阅读进度
基础必读(0/5)
- Mikolov et al. 2013 — Word2Vec
- Pennington et al. 2014 — GloVe
- Vaswani et al. 2017 — Attention Is All You Need
- Devlin et al. 2019 — BERT
- Radford et al. 2018/2019 — GPT / GPT-2
进阶精读(0/5)
- Ouyang et al. 2022 — InstructGPT (RLHF)
- Hu et al. 2022 — LoRA
- Wei et al. 2022 — Chain-of-Thought Prompting
- Bai et al. 2022 — Constitutional AI
- Kalai & Vempala 2024 — Calibrated LMs Must Hallucinate
前沿选读(0/2)
- ProRL (Liu et al. NeurIPS 2025) — Prolonged RL
- RLP (Hatamizadeh et al. ICLR 2026) — RL as Pretraining
CS224N 学习计划
AdaGrow 研究交叉思考
与 CS224N 内容的关联点
- L05 Transformers ← AdaGrow 的 Transformer adapter 架构增长策略
- L07 Pretraining ← 模型增长与预训练效率的关系: 渐进式增长是否能替代大模型从零训练？
- L09 PEFT ← AdaGrow 的宽度/深度增长 vs. LoRA 的低秩适应: 两种”高效扩展”范式的对比
- L12/L13 Reasoning ← ProRL 的 sustainable entropy 机制可借鉴到 AdaGrow 的训练调度
- L16 Social Impact ← “Smaller but Better” 理念与 AdaGrow 的高效增长完美契合
- L19 Open Questions ← Prismatic Synthesis 的 G-Vendi Score (梯度多样性度量) 可用于评估增长策略的多样性
潜在研究方向
- AdaGrow + RL: 用 RL 信号指导模型何时增长、往哪里增长
- 增长策略中的 entropy 控制: 借鉴 ProRL 的 decoupled clipping 防止增长后的 entropy collapse
- 梯度驱动的增长点选择: 类似 Prismatic Synthesis 用梯度表示来选择最有价值的增长位置