Open-source LLM Resources

Open-source LLM Resources

Knowledge

• Paper: "Attention Is All You Need" (Vaswani et al., 2017)

The original transformer paper. Use for: the architecture behind most modern LLMs and the idea of attention.

• Paper: "Language Models are Few-Shot Learners" (Brown et al., 2020)

A landmark account of large autoregressive language models. Use for: next-token prediction and in-context behavior.

• Paper: "Training language models to follow instructions with human feedback" (Ouyang et al., 2022)

Primary-source description of supervised fine-tuning and preference-based post-training. Use for: why a chat model differs from a raw pretrained model.

• Paper: "Training Compute-Optimal Large Language Models" (Hoffmann et al., 2022)

Experimental evidence on balancing model size, training tokens, and compute. Use for: why more data and training are not automatically better.

• Paper: "Learning representations by back-propagating errors" (Rumelhart, Hinton & Williams, 1986)

Foundational primary source for the mechanism used to compute gradients through layered neural networks. Use for: the chain rule and weight updates.

• Paper: "Datasheets for Datasets" (Gebru et al., 2021)

A practical documentation framework for dataset motivation, collection, composition, and risks. Use for: dataset provenance and responsible preparation.

• Hugging Face Datasets documentation: Process

Maintained implementation guide for filtering, mapping, and splitting datasets. Use for: turning preparation principles into reproducible data-processing steps.

• Hugging Face LLM Course

A maintained, practical introduction with visual and coding-oriented explanations. Use for: revisiting terminology and later hands-on study.

• Stanford CS224N: Language Modeling and RNNs

University course materials that give broader NLP context. Use for: deeper study after the course basics.

Wisdom (Communities)

• Hugging Face Forums

Maintained community for asking practical questions about open models and their tooling.

Gaps

• The course currently prioritizes conceptual understanding; sources on selecting and running local models will be added only if that becomes part of the mission.