AIQB
ResearchOrdinary

Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation

Source: arXiv·

Summary

In this paper, we present 1020 Group's submission to the WMT26 shared task on automated quality evaluation. Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation. It is based on an LLM-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis. We also developed several variants of Pterodactyl that were specifically adapted for subtasks 3.
CategoryAI Research & Papers
TierOrdinary
Published
Indexed by AIQB
SourcearXiv
AIQB record IDintel-0dad14ebc38fcf4b4932c204
Read original source ↗View data sourceMore Research

Related stories

Matched by category, entity tags and source

ResearchOrdinary
Medical Image Segmentation: Foundation Model Adaptation versus Task-Specific Training
Foundation models pretrained on large natural-image collections are increasingly adapted for medical image segmentation, in the expectation that they will replace models trained for each task. Most comparisons, however, use data from the t…
Source: arXiv
ResearchOrdinary
Toward Foundation Detection Models via Example-Based Grounding in EM-DETR
DeepLearning (DL)-based Computer-Aided Diagnostic (CAD) systems have shown great promise in supporting clinicians, but their performance heavily depends on the availability of large, well-annotated datasets. In medical lesion detection, th…
Source: arXiv
ResearchOrdinary
TorchCurves: Differentiable Parametric Curves
We present TorchCurves, a PyTorch library that treats vector-valued parametric curves as learnable primitives. Its central operation maps batches of scalar inputs to points on curves represented in B-spline or Legendre polynomial bases, wi…
Source: arXiv
ResearchOrdinary
AutoScaffold: AI-Guided Scaffolding for Reinforcement Learning of Language Models
Reinforcement learning with verifiable rewards (RLVR) produces rich training experience, yet outcome rewards reduce each attempt to a single number and leave most of this information unused. Training-time scaffolds, such as strategies or h…
Source: arXiv
ResearchOrdinary
Contrastive Discovery: Open-Ended Scientific Discovery over Competing Explanations
LLM agents are increasingly used for open-ended, data-driven scientific discovery, yet most of them evaluate hypotheses pointwise, testing an empirical claim $h$ only against its complement $\neg h$. When evidence rejects $h$, the agent le…
Source: arXiv
ResearchOrdinary
Pyrite: Building and Benchmarking Protein-Ligand Pose Ranking Agents
Predicting how a ligand binds to its target is central to small-molecule drug discovery. Modern cofolding models can generate diverse candidate poses, but reliably ranking these candidates remains challenging. We introduce Pyrite, a pairwi…
Source: arXiv
ResearchOrdinary
When Does the Real Wiring Win? Testing connectome structure-function claims against unmeasured physiology
Connectome-constrained models are often used to argue that a measured wiring diagram outperforms shuffled or rewired wiring, and hence that structure shapes function. Such comparisons can confound wiring with how the models are tuned: a co…
Source: arXiv
ResearchOrdinary
Continual-ARC: Measuring Skill Acquisition over a Working Lifetime
Agent harnesses have emerged as a promising mechanism for continual learning, alongside test-time parametric learning. However, the current benchmarks have several limitations: they generally focus on a single applied domain, serve each in…
Source: arXiv · Qwen / GLM
ResearchOrdinary
Emergent Biological Capabilities in a Foundation Model for Molecular Interactions
Foundation models trained on large unlabeled corpora develop emergent capabilities, tasks that go beyond the training objective. We extend this paradigm to multimodal biology with Bonbon, a foundation model for molecular interactions train…
Source: arXiv
ResearchOrdinary
Comparative Analysis of Agent Evaluation Frameworks: Stability, Detection, and Discovery in Enterprise Analytics Agents
Deploying LLM-based analytics agents in enterprise settings requires evaluation frameworks that can reliably detect failures across complex, multi-tool workflows. We present a three-phase comparative study of three evaluation frameworks, e…
Source: arXiv