AI圈报
论文研究普通

MAPLE: A Multi-Axis Parametric Knowledge Extraction Lens for LLM Evaluation

信息来源：arXiv·
QwenLlama

内容摘要

LLMs absorb vast knowledge from training data, yet their inherently probabilistic nature offers no guarantee that this knowledge will surface in their outputs. To bridge this gap, Knowledge Extraction Enhancement (KEE) techniques such as activation steering, decoding adjustments, and prompting modify inference, potentially reinforcing existing predictions or surfacing less accessible answers. Yet their evaluations centered on aggregate accuracy, which obscures diverse effects by collapsing recovered failures and disrupted successes into a single score. We introduce MAPLE, a unified framework for characterizing how KEE techniques affect parametric knowledge extraction. Its diagnostic metrics disentangle these effects at the surface, probability, and beyond-sampling levels by measuring answer recovery and disruption, gold-token rank shifts on unrecovered failures, and recovery of answers missed by fixed-budget sampling. We evaluate six KEE techniques spanning decoding, activation steering, and prompting across three model families: OLMo2, Qwen2.5, and Llama3. Our analysis reveals distinct effects and trade-offs across techniques, highlighting the value of MAPLE for understanding and comparing KEE beyond aggregate accuracy.
内容分类AI 论文与研究
内容层级普通情报
发布时间（北京时间）
本站收录时间（北京时间）
信息来源arXiv
站内情报编号intel-819aa715a6efe4376ea845a7
阅读原始信息 ↗查看数据来源更多论文研究

相关推荐

综合分类、实体标签与信息来源匹配

论文研究普通
Energy Consumption Analysis of Discrete Diffusion and Autoregressive Language Models
As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) models are the industry standard, Diffusion Language Models (DLLMs) have emerged as a pro…
信息来源：arXiv · Qwen / Llama
论文研究普通
Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too
Post-training pruning enables efficient deployment, but conventional methods do not explicitly preserve internal-gradient alignment between sparse and dense models. We study novel connections between downstream performance of compressed mo…
信息来源：arXiv · Qwen / Llama
论文研究普通
Thinking Outside the Box: Retention and Transmission of Information in Sliding-Window KV Inference
Sliding-window KV inference refers to processing a sequence incrementally while retaining only a fixed-size cache of recent key and value states. It can be applied to pretrained causal transformers at inference time without additional trai…
信息来源：arXiv · Qwen / Llama / Mistral
论文研究普通
Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers
Retrieval-augmented generation (RAG) is often used to address outdated knowledge by providing external evidence. But retrieval helps only when that evidence is still valid. We identify a temporal alignment failure, stale-document poisoning…
信息来源：arXiv · Qwen / Llama
论文研究普通
Disaggregated Quantization: Specializing LLM Prefill and Decode
Prefill and decode reward different approaches to quantization: low-precision arithmetic accelerates prompt processing, while compact weights reduce memory traffic during generation. We propose "disaggregated quantization" (DQ), which spec…
信息来源：arXiv · Qwen / Llama
论文研究普通
Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol States
Long-context language models interface with external knowledge through raw natural language. In retrieval-augmented systems, this creates a persistent index-payload schism: dense vectors enable searchable routing, but models must re-ingest…
信息来源：arXiv · GPT / Qwen / Llama
论文研究普通
A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Reveal
Large language models can hold knowledge they do not report. A model may sandbag on a capability evaluation, or answer against what it internally knows, and its outputs alone cannot tell whether it is hiding an answer or simply does not ha…
信息来源：arXiv · Qwen / Llama / Mistral
论文研究普通
To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals
Speculative Decoding (SD) has significantly accelerated Large Language Model (LLM) inference, yet existing approaches face a fundamental tradeoff between two drafting strategies: neural drafting and context-based copying. Neural drafts (e.…
信息来源：arXiv · Qwen / Llama
论文研究普通
A Zeroth-Order Paradigm for LLM Preference Alignment
Direct preference alignment methods are widely used to align large language models (LLMs) with human preferences because of their computational and memory efficiency. However, likelihood displacement motivates alternative ways to extract i…
信息来源：arXiv · Qwen / Llama / Mistral
论文研究普通
An Empirical Study of Counterfactual Self-Explanations in LLMs
Large language models can easily generate explanations for their own outputs, but such self-explanations are not necessarily faithful to the model's behavior. We study this issue through counterfactual self-explanations, where a model mini…
信息来源：arXiv · Qwen / Llama