知识盲盒

参考书目与文献索引 (References and Bibliography)

发布于: 2026-07-17作者: GPT 5.5、GPT 5.6 Solpublished

ai-generatedstochastic-parrotstochastic-parrot-anatomyai

本列表收录书中引用的经典论文、核心书籍、技术报告与协议规范，按主题分类整理。2024 年以后仍可能变化的模型、产品和协议条目校准至 2026 年 7 月 12 日；动态文档的日期表示本书核验口径，而不等同于论文发表时间。

1. 基础理论 (Foundations)

[McCulloch & Pitts, 1943] A Logical Calculus of the Ideas Immanent in Nervous Activity. (M-P 神经元模型的提出)
[Turing, 1950] Computing Machinery and Intelligence. (图灵测试)
[McCarthy et al., 1955] A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence. (人工智能夏季研究项目提案)
[Rosenblatt, 1958] The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. (感知机)
[Novikoff, 1962] On Convergence Proofs for Perceptrons. (感知机错误次数界的经典证明)
[Minsky & Papert, 1969] Perceptrons. (单层感知机能力边界及相关分析)
[Rumelhart, Hinton & Williams, 1986] Learning representations by back-propagating errors. (反向传播的经典应用)
[Cybenko, 1989] Approximation by Superpositions of a Sigmoidal Function. (连续 sigmoidal/discriminatory 激活的一致逼近定理)
[Hornik, 1991] Approximation Capabilities of Multilayer Feedforward Networks. (前馈网络逼近能力的推广)
[Leshno et al., 1993] Multilayer Feedforward Networks with a Nonpolynomial Activation Function Can Approximate Any Function. (非多项式激活条件的刻画)
[Cortes & Vapnik, 1995] Support-Vector Networks. (软间隔支持向量机)
[Breiman, 2001] Random Forests. (随机森林)
[Friedman, 2001] Greedy Function Approximation: A Gradient Boosting Machine. (梯度提升)

1.1 统计学习与随机逼近

[Robbins & Monro, 1951] A Stochastic Approximation Method. (随机逼近与衰减步长框架)
[Hoeffding, 1963] Probability Inequalities for Sums of Bounded Random Variables. (有界独立随机变量的集中不等式)
[Vapnik & Chervonenkis, 1971] On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities. (VC 维与一致收敛)
[Sauer, 1972] On the Density of Families of Sets. (增长函数的组合上界)

1.2 优化、初始化与正则化

[Glorot & Bengio, 2010] Understanding the difficulty of training deep feedforward neural networks. (Xavier 初始化与信号尺度)
[He et al., 2015] Delving Deep into Rectifiers. (适配 rectifier 的初始化)
[Srivastava et al., 2014] Dropout: A Simple Way to Prevent Neural Networks from Overfitting. (dropout)
[Kingma & Ba, 2014] Adam: A Method for Stochastic Optimization. (Adam)
[Loshchilov & Hutter, 2019] Decoupled Weight Decay Regularization. (AdamW 与解耦 weight decay)

2. 卷积神经网络 (CNNs)

[LeCun et al., 1998] Gradient-based learning applied to document recognition. (LeNet-5)
[Krizhevsky et al., 2012] ImageNet Classification with Deep Convolutional Neural Networks. (AlexNet)
[Simonyan & Zisserman, 2014] Very Deep Convolutional Networks for Large-Scale Image Recognition. (VGGNet)
[He et al., 2016] Deep Residual Learning for Image Recognition. (ResNet: 残差连接)

3. 序列模型与 RNN (Sequence Models)

[Hochreiter & Schmidhuber, 1997] Long Short-Term Memory. (LSTM 的提出)
[Cho et al., 2014] Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. (GRU 与 Seq2Seq)
[Bahdanau, Cho & Bengio, 2014] Neural Machine Translation by Jointly Learning to Align and Translate. (加性注意力)
[Luong, Pham & Manning, 2015] Effective Approaches to Attention-based Neural Machine Translation. (乘性注意力)
[Peters et al., 2018] Deep Contextualized Word Representations. (ELMo: 动态词向量)

4. Transformer 与预训练语言模型 (Transformer & PLMs)

[Vaswani et al., 2017] Attention Is All You Need. (Transformer 架构)
[Ba, Kiros & Hinton, 2016] Layer Normalization. (LayerNorm)
[Zhang & Sennrich, 2019] Root Mean Square Layer Normalization. (RMSNorm)
[Shazeer, 2020] GLU Variants Improve Transformer. (GLU、GEGLU 与 SwiGLU 变体)
[Shazeer, 2019] Fast Transformer Decoding: One Write-Head is All You Need. (Multi-Query Attention)
[Ainslie et al., 2023] GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. (Grouped-Query Attention)
[Su et al., 2021] RoFormer: Enhanced Transformer with Rotary Position Embedding. (RoPE)
[Chen et al., 2023] Extending Context Window of Large Language Models via Positional Interpolation. (位置插值与上下文扩展)
[Liu et al., 2023] Ring Attention with Blockwise Transformers for Near-Infinite Context. (分布式长上下文注意力)
[Child et al., 2019] Generating Long Sequences with Sparse Transformers. (稀疏注意力与长序列生成)
[Dao et al., 2022] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. (IO-aware 精确注意力)
[Dao, 2023] FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. (更高并行度的注意力 kernel)
[Fedus, Zoph & Shazeer, 2021] Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. (稀疏专家、容量与路由均衡)
[Gu & Dao, 2023] Mamba: Linear-Time Sequence Modeling with Selective State Spaces. (选择性状态空间模型)
[Dao & Gu, 2024] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality. (Mamba-2 与 SSM/Attention 统一视角)
[Devlin et al., 2018] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. (BERT)
[Liu et al., 2019] RoBERTa: A Robustly Optimized BERT Pretraining Approach. (移除 NSP 并联合调整数据与训练配方)
[Radford et al., 2018] Improving Language Understanding by Generative Pre-Training. (GPT-1)
[Radford et al., 2019] Language Models are Unsupervised Multitask Learners. (GPT-2)
[Brown et al., 2020] Language Models are Few-Shot Learners. (GPT-3)
[Raffel et al., 2020] Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. (T5)
[Lewis et al., 2019] BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. (BART)
[Kudo & Richardson, 2018] SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. (从原始句子训练的子词 tokenizer)
[Loshchilov & Hutter, 2019] Decoupled Weight Decay Regularization. (AdamW 与解耦 weight decay)
[Lee et al., 2022] Deduplicating Training Data Makes Language Models Better. (训练语料与 train-test 重复)

5. 大模型对齐与优化 (Alignment & Optimization)

[Wei et al., 2021] Finetuned Language Models Are Zero-Shot Learners. (FLAN: 指令微调)
[Ouyang et al., 2022] Training language models to follow instructions with human feedback. (InstructGPT: RLHF 的应用)
[Schulman et al., 2017] Proximal Policy Optimization Algorithms. (PPO 算法)
[Amodei et al., 2016] Concrete Problems in AI Safety. (reward hacking、负副作用与安全问题)
[Rafailov et al., 2023] Direct Preference Optimization: Your Language Model is Secretly a Reward Model. (DPO)
[Li & Liang, 2021] Prefix-Tuning: Optimizing Continuous Prompts for Generation. (每层 prefix 参数)
[Lester, Al-Rfou & Constant, 2021] The Power of Scale for Parameter-Efficient Prompt Tuning. (输入层 soft prompt)
[Hu et al., 2021] LoRA: Low-Rank Adaptation of Large Language Models. (LoRA)
[Dettmers et al., 2023] QLoRA: Efficient Finetuning of Quantized LLMs. (QLoRA)
[Frantar et al., 2022] GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. (GPTQ 权重量化)
[Xiao et al., 2022] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. (激活/权重量化平滑)
[Lin et al., 2023] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. (AWQ 混合/保护式量化)
[Kaplan et al., 2020] Scaling Laws for Neural Language Models. (缩放定律)
[Hoffmann et al., 2022] Training Compute-Optimal Large Language Models. (固定计算预算下的模型规模与训练 token 配置)
[Kwon et al., 2023] Efficient Memory Management for Large Language Model Serving with PagedAttention. (PagedAttention 与 vLLM)
[DeepSeek-AI, 2024] DeepSeek-V3 Technical Report. (MLA、MoE 与高效训练)
[DeepSeek-AI, 2025] Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention. (NSA: 硬件对齐、可训练稀疏注意力)
[DeepSeek-AI, 2025] DeepSeek-V3.2-Exp: Boosting Long-Context Efficiency with DeepSeek Sparse Attention. (DSA 实验性长上下文效率模型)
[Zhou et al., 2026] MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference. (独立后续研究：DSA 索引器的 head 轴路由加速，并非 DeepSeek 团队论文)
[Shao et al., 2024] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. (GRPO 的提出)
[DeepSeek-AI, 2025] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. (推理模型、GRPO 与 RL 后训练)
[DeepSeek-AI, 2025] DeepSeek-OCR: Contexts Optical Compression. (作者所称的 initial investigation：视觉 token 压缩与 OCR 重建，不代表通用长上下文压缩已解决)
[Cheng et al., 2026] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models. (Engram：哈希寻址 N-gram embedding 与层内条件记忆)
[OpenAI, 2024] Learning to Reason with LLMs. (o1、强化学习与测试时计算的公开说明)
[Dubey et al., 2024] The Llama 3 Herd of Models. (开放权重模型、预训练与后训练技术报告)
[OpenAI, 2026-07-09] Introducing GPT-5.6. (GPT-5.6 Sol / Terra / Luna 已 GA；图像输入等能力以发布页为准；核验于 2026-07-12)
[DeepSeek, 2026] DeepSeek API Change Log 与 Models and Pricing. (deepseek-v4-pro / deepseek-v4-flash 官方 API 标识；核验于 2026-07-12)
[Hinton et al., 2015] Distilling the Knowledge in a Neural Network. (知识蒸馏)
[Wang et al., 2022] Self-Instruct: Aligning Language Models with Self-Generated Instructions. (合成指令数据)
[Maini et al., 2024] TOFU: A Task of Fictitious Unlearning for LLMs. (面向个体虚构资料的遗忘集/保留集、多指标评测与重训参照；基准结果不证明任意观察下的精确遗忘)
[Guo et al., 2020] Certified Data Removal from Machine Learning Models. (以重训输出分布为参照的 certified removal 定义及线性分类器机制)
[Mitchell et al., 2019] Model Cards for Model Reporting. (模型用途、评测条件、群体差异与限制的结构化发布文档)
[Gebru et al., 2021] Datasheets for Datasets. (数据动机、组成、收集、处理、用途和维护的结构化文档框架)
[Bai et al., 2022] Constitutional AI: Harmlessness from AI Feedback. (AI 反馈与安全后训练)
[Korbak et al., 2025] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. (CoT 监控与安全训练风险)
[Wortsman et al., 2022] Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. (权重平均与模型合并)
[Ilharco et al., 2022] Editing Models with Task Arithmetic. (任务向量与模型编辑)
[Yadav et al., 2023] TIES-Merging: Resolving Interference When Merging Models. (模型合并干扰处理)
[Yu et al., 2023] Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. (DARE 与模型合并)
[Leviathan et al., 2023] Fast Inference from Transformers via Speculative Decoding. (保持目标分布的投机解码)
[Cai et al., 2024] Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads. (多解码头推理加速)
[Li et al., 2024] EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty. (特征级投机解码)

6. 多模态与智能体 (Multimodal & Agents)

[Dosovitskiy et al., 2020] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. (ViT)
[Radford et al., 2021] Learning Transferable Visual Models From Natural Language Supervision. (CLIP)
[van den Oord, Li & Vinyals, 2018] Representation Learning with Contrastive Predictive Coding. (InfoNCE 与对比预测编码)
[Liu et al., 2023] Visual Instruction Tuning. (LLaVA)
[Alayrac et al., 2022] Flamingo: a Visual Language Model for Few-Shot Learning. (冻结视觉与语言组件之间的 Perceiver Resampler、门控交叉注意力及交错图文训练)
[Li et al., 2023] BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. (Q-Former 作为冻结图像编码器与语言模型之间的查询瓶颈)
[Radford et al., 2022] Robust Speech Recognition via Large-Scale Weak Supervision. (Whisper 的 log-Mel 输入、编码器—解码器结构与多任务语音训练)
[Kingma & Welling, 2013] Auto-Encoding Variational Bayes. (证据下界、重参数化估计与变分自编码器)
[Goodfellow et al., 2014] Generative Adversarial Nets. (原始 GAN 极小极大目标、最优判别器与 Jensen—Shannon 散度分析)
[Ho, Jain & Abbeel, 2020] Denoising Diffusion Probabilistic Models. (离散高斯前向过程、反向模型与简化噪声预测目标)
[Song, Meng & Ermon, 2020] Denoising Diffusion Implicit Models. (与 DDPM 共享训练目标的非 Markov 采样过程及确定性极限)
[Song et al., 2021] Score-Based Generative Modeling through Stochastic Differential Equations. (反向时间 SDE、概率流 ODE 与连续时间 score 模型)
[Austin et al., 2021] Structured Denoising Diffusion Models in Discrete State-Spaces. (D3PM 的离散转移矩阵与反向去噪)
[Rombach et al., 2022] High-Resolution Image Synthesis with Latent Diffusion Models. (Latent Diffusion / Stable Diffusion 基础)
[Peebles & Xie, 2023] Scalable Diffusion Models with Transformers. (DiT)
[Lipman et al., 2022] Flow Matching for Generative Modeling. (Flow Matching 与连续生成路径)
[Heusel et al., 2017] GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. (Fréchet Inception Distance 的提出及其特征高斯矩定义)
[Bińkowski et al., 2018] Demystifying MMD GANs. (Kernel Inception Distance、无偏 MMD 估计及有限样本比较)
[Betker et al., 2023] Improving Image Generation with Better Captions. (DALL-E 3 与高质量图文描述)
[OpenAI, 2024] GPT-4o System Card. (文本、视觉、音频的端到端多模态评测与安全分析)
[OpenAI, 2024] Video Generation Models as World Simulators. (Sora：时空 patch 上的扩散 Transformer；技术细节披露有限)
[Google DeepMind, 核验于 2026-07-12] Veo. (视频生成模型族的官方能力材料；不据此推断未公开架构)
[Kondratyuk et al., 2023] VideoPoet: A Large Language Model for Zero-Shot Video Generation. (视频生成的语言模型路线)
[Reid et al., 2024] Gemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens of Context. (长上下文多模态理解)
[Google, 核验于 2026-07-12] Gemini 3.5 Flash Model Documentation. (stable/GA；text/image/video/audio/PDF 输入、text 输出；不支持 Live API)
[Ha & Schmidhuber, 2018] World Models. (世界模型经典工作)
[Hafner et al., 2019] Dream to Control: Learning Behaviors by Latent Imagination. (Dreamer 与潜在想象)
[Hafner et al., 2023] Mastering Diverse Domains through World Models. (DreamerV3 的 RSSM、想象 actor—critic 与跨域训练配方)
[Hansen et al., 2024] TD-MPC2: Scalable, Robust World Models for Continuous Control. (潜空间局部模型、价值学习与基于 CEM 的模型预测控制)
[Assran et al., 2023] Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture. (I-JEPA)
[Bruce et al., 2024] Genie: Generative Interactive Environments. (从视频学习可交互环境)
[Zitkovich et al., 2023] RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. (把视觉、语言和机器人动作接入同一 VLA 接口；具体实验不外推为开放环境安全保证)
[Kim et al., 2024] OpenVLA: An Open-Source Vision-Language-Action Model. (开放 VLA 模型、动作 token 化与跨机器人数据训练)
[Chi et al., 2023] Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. (用条件 diffusion 生成连续动作序列与 receding-horizon 执行)
[Google DeepMind et al., 2025] Gemini Robotics: Bringing AI into the Physical World. (VLA 通用模型、机器人形态适配与物理行动研究入口)
[Wei et al., 2022] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. (CoT)
[Kojima et al., 2022] Large Language Models are Zero-Shot Reasoners. (Zero-shot CoT)
[Yao et al., 2022] ReAct: Synergizing Reasoning and Acting in Language Models. (ReAct 框架)
[Schick et al., 2023] Toolformer: Language Models Can Teach Themselves to Use Tools. (在语言建模训练中学习 API 调用位置、工具与参数)
[Zhou et al., 2024] WebArena: A Realistic Web Environment for Building Autonomous Agents. (可重置网站环境、长程任务和功能正确性评测；结果受其网站与任务快照约束)
[Greshake et al., 2023] Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. (攻击者控制的外部数据通过有权限应用触发间接 prompt injection)
[Lewis et al., 2020] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. (RAG)
[Robertson & Zaragoza, 2009] The Probabilistic Relevance Framework: BM25 and Beyond. (BM25 的概率相关性框架、词频饱和与文档长度归一化)
[Karpukhin et al., 2020] Dense Passage Retrieval for Open-Domain Question Answering. (DPR 双编码器、困难负例与最大内积检索)
[Khattab & Zaharia, 2020] ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. (token 级 MaxSim 晚交互与可预计算文档表示)
[Malkov & Yashunin, 2018] Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. (HNSW 图索引与近似最近邻的延迟—召回折中)
[Liu et al., 2023] Lost in the Middle: How Language Models Use Long Contexts. (多文档问答与键值检索中的证据位置效应；结论受模型和任务协议约束)
[Model Context Protocol, 2025-11-25] Specification. (模型应用与工具、资源、提示模板等上下文集成；latest 核验于 2026-07-12)
[A2A Project, 2026] A2A Protocol Specification v1.0.0. (2026 年首个 stable 规范：Agent 间消息、任务与 artifact 协调；Google 发起，现为 Linux Foundation 项目；核验于 2026-07-12)
[OpenAI, 核验于 2026-07-12] OpenAI Agents SDK Documentation. (Agent 运行时、工具、handoff、guardrails 与 tracing 的实现文档)
[LangChain, 核验于 2026-07-12] LangGraph Overview. (持久化执行、状态图与多 Agent 编排的实现文档)
[Google, 核验于 2026-07-12] Agent Development Kit Documentation. (Agent 开发、会话、记忆、工具与上下文管理的实现文档)

7. 推荐书籍 (Recommended Books)

Ian Goodfellow, Yoshua Bengio, Aaron Courville. Deep Learning. MIT Press, 2016. (深度学习教材)
Sutton & Barto. Reinforcement Learning: An Introduction. MIT Press, 2018. (强化学习入门)
Daniel Kahneman. Thinking, Fast and Slow. (对 System 1 / System 2 表述的普及；双过程理论来自更广泛研究传统)

本书目录