Northeastern University: Foundations of Large Language Models

2025-01-27

Summary of https://arxiv.org/pdf/2501.09223 Detail foundational concepts and advanced techniques in large language model (LLM) development. It covers pre-training methods, including masked language modeling and discriminative training, and explores generative model architectures like Transformers. The text also examines scaling LLMs for size and context length, along with alignment strategies such as reinforcement learning from human feedback (RLHF) and instruction fine-tuning. Finally, it...

Summary of https://arxiv.org/pdf/2501.09223

Detail foundational concepts and advanced techniques in large language model (LLM) development. It covers pre-training methods, including masked language modeling and discriminative training, and explores generative model architectures like Transformers.

The text also examines scaling LLMs for size and context length, along with alignment strategies such as reinforcement learning from human feedback (RLHF) and instruction fine-tuning.

Finally, it discusses prompting techniques, including chain-of-thought prompting and prompt optimization methods to improve LLM performance and alignment with human preferences.

Comments (3)