Carnegie Mellon College at ICLR 2025 – Machine Studying Weblog | ML@CMU
CMU researchers are presenting 143 papers on the Thirteenth Worldwide Convention on Studying Representations (ICLR 2025), held from April 24 – 28 on the Singapore EXPO. Here’s a fast overview of the areas our researchers are engaged on:
And listed here are our most frequent collaborator establishments:
Desk of Contents
- Oral Papers
- Spotlight Papers
- Poster Papers
- Alignment, Fairness, Safety, Privacy, And Societal Considerations
- Applications to Computer Vision, Audio, Language, And Other Modalities
- Applications to Neuroscience & Cognitive Science
- Applications to Physical Sciences (Physics, Chemistry, Biology, Etc.)
- Applications to Robotics, Autonomy, Planning
- Causal Reasoning
- Datasets and Benchmarks
- Foundation or Frontier Models, Including LLMs
- Generative Models
- Infrastructure, Software Libraries, Hardware, Systems, etc.
- Interpretability and Explainable AI
- Learning on Graphs and Other Geometries & Topologies
- Learning Theory
- Neurosymbolic & Hybrid AI Systems (Physics-Informed, Logic & Formal Reasoning, etc.)
- Optimization
- Other Topics in Machine Learning (i.e., none of the above)
- Probabilistic Methods (Bayesian Methods, Variational Inference, Sampling, Uncertainty Quantification, etc.)
- Reinforcement Learning
- Transfer Learning, Meta Learning, and Lifelong Learning
- Unsupervised, Self-supervised, Semi-supervised, and Supervised Representation Learning
Oral Papers
Backtracking Improves Generation Safety
This paper introduces backtracking, a brand new approach that enables language fashions to get well from unsafe textual content technology by utilizing a particular [RESET] token to “undo” problematic outputs. Not like conventional security strategies that intention to stop dangerous responses outright, backtracking trains the mannequin to self-correct mid-generation. The authors exhibit that backtracking considerably improves security with out sacrificing helpfulness, and it additionally gives robustness in opposition to a number of adversarial assaults.
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
Current advances in LLMs have enabled job automation via Python code, however current benchmarks primarily concentrate on easy, self-contained duties. To evaluate LLMs’ skill to deal with extra sensible challenges requiring numerous and compositional perform use, the authors introduce BigCodeBench—a benchmark protecting 1,140 duties throughout 139 libraries and seven domains. Every job contains rigorous testing with excessive department protection, and a variant, BigCodeBench-Instruct, reformulates directions for pure language analysis. Outcomes from testing 60 LLMs reveal important efficiency gaps, highlighting that present fashions battle to comply with advanced directions and compose perform calls precisely in comparison with human efficiency.
Context-Parametric Inversion: Why Instruction Finetuning May Not Actually Improve Context Reliance
LLMs are anticipated to comply with user-provided context, particularly once they include new or conflicting data. Whereas instruction finetuning ought to enhance this skill, the authors uncover a stunning failure mode referred to as context-parametric inversion: fashions initially rely extra on enter context, however this reliance decreases as finetuning continues—at the same time as benchmark efficiency improves. Via managed experiments and theoretical evaluation, the authors hint the trigger to coaching examples the place context aligns with pretraining data, reinforcing parametric reliance. They counsel mitigation methods and spotlight this as a key problem in instruction tuning.
EmbodiedSAM: Online Segment Any 3D Thing in Real Time
Embodied duties demand fine-grained 3D notion, which is tough to attain resulting from restricted high-quality 3D knowledge. To deal with this, the authors suggest a technique that leverages the Section Something Mannequin (SAM) for on-line 3D occasion segmentation by remodeling 2D masks into 3D-aware queries. Their method permits real-time object matching throughout video frames and environment friendly inference utilizing a similarity matrix. Experiments throughout a number of datasets present that the strategy outperforms offline options and generalizes nicely to new settings with minimal knowledge.
LLM-SR: Scientific Equation Discovery via Programming with Large Language Models
Mathematical equations are remarkably efficient at describing pure phenomena, however discovering them from knowledge is difficult resulting from huge combinatorial search areas. Current symbolic regression strategies typically overlook area data and depend on restricted representations. To deal with this, the authors suggest LLM-SR, a novel method that makes use of Giant Language Fashions to generate equation hypotheses knowledgeable by scientific priors and refines them via evolutionary search. Evaluated throughout a number of scientific domains, LLM-SR outperforms current strategies, significantly in generalization, by effectively exploring the equation area and producing correct, interpretable fashions.
Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
Self-improvement in Giant Language Fashions entails the mannequin verifying its outputs, filtering knowledge accordingly, and utilizing the refined knowledge for additional studying. Whereas efficient in follow, there was little theoretical grounding for this system. This work presents a complete examine of LLM self-improvement, introducing a proper framework centered on the generation-verification hole—a key amount that governs self-improvement. Experiments reveal that this hole scales persistently with pretraining FLOPs throughout duties and mannequin households. The authors additionally discover when and the way iterative self-improvement works and supply insights and methods to reinforce it.
On the Benefits of Memory for Modeling Time-Dependent PDEs
Information-driven strategies supply an environment friendly different to conventional numerical solvers for PDEs, however most current approaches assume Markovian dynamics, limiting their effectiveness when enter indicators are distorted. Impressed by the Mori-Zwanzig principle, the authors suggest MemNO, a Reminiscence Neural Operator that explicitly incorporates previous states utilizing structured state-space fashions and the Fourier Neural Operator. MemNO demonstrates robust efficiency on numerous PDE households, particularly on low-resolution inputs, attaining over six occasions decrease error than memoryless baselines.
On the Identification of Temporal Causal Representation with Instantaneous Dependence
This work introduces IDOL (Identification framework for Instantaneous Latent dynamics), a technique designed to determine latent causal processes in time sequence knowledge, even when instantaneous relationships are current. Not like current strategies that require interventions or grouping of observations, IDOL imposes a sparse affect constraint, permitting each time-delayed and instantaneous causal relations to be captured. Via a temporally variational inference structure and gradient-based sparsity regularization, IDOL successfully estimates latent variables. Experimental outcomes present that IDOL can determine latent causal processes in simulations and real-world human movement forecasting duties, demonstrating its sensible applicability.
Progressive distillation induces an implicit curriculum
This work explores the idea of progressive distillation, the place a pupil mannequin learns from intermediate checkpoints of a instructor mannequin, fairly than simply the ultimate mannequin. The authors determine an “implicit curriculum” that emerges via these intermediate checkpoints, which accelerates the coed’s studying and gives a pattern complexity profit. Utilizing sparse parity as a sandbox, they exhibit that this curriculum imparts priceless studying steps which might be unavailable from the ultimate instructor mannequin. The examine extends this concept to Transformers skilled on probabilistic context-free grammars (PCFGs) and real-world datasets, displaying that the instructor progressively teaches the coed to seize longer contexts. Each theoretical and empirical outcomes spotlight the effectiveness of progressive distillation throughout totally different duties.
This work introduces precision-aware scaling legal guidelines that stretch conventional scaling frameworks to account for the consequences of low-precision coaching and inference in language fashions. The authors present that decrease precision successfully reduces a mannequin’s usable parameter depend, enabling predictions of efficiency degradation resulting from quantization. For inference, they discover that post-training quantization causes rising degradation with extra pretraining knowledge, doubtlessly making further coaching counterproductive. Their unified framework predicts loss throughout various precisions and means that coaching bigger fashions in decrease precision could also be extra compute-efficient. These predictions are validated on over 465 pretraining runs, together with fashions as much as 1.7B parameters.
Self-Improvement in Language Models: The Sharpening Mechanism
This paper presents a theoretical framework for understanding how LLMs can self-improve by utilizing themselves as verifiers to refine their very own outputs; a course of the authors name “sharpening.” The important thing perception is that LLMs are sometimes higher at judging response high quality than producing high-quality responses outright, so sharpening helps focus likelihood mass on higher sequences. The paper analyzes two households of self-improvement algorithms: one based mostly on supervised fine-tuning (SFT) and one on reinforcement studying (RLHF). They present that whereas the SFT-based method is perfect beneath sure situations, the RLHF-based method can outperform it by actively exploring past the mannequin’s current data.
When Selection meets Intervention: Additional Complexities in Causal Discovery
This work tackles the often-overlooked difficulty of choice bias in interventional research, the place members are selectively included based mostly on particular standards. Current causal discovery strategies sometimes ignore this bias, resulting in inaccurate conclusions. To deal with this, the authors introduce a novel graphical mannequin that distinguishes between the noticed world with interventions and the counterfactual world the place choice happens. They develop a sound algorithm that identifies each causal relationships and choice mechanisms, demonstrating its effectiveness via experiments on each artificial and real-world knowledge.
miniCTX: Neural Theorem Proving with (Long-)Contexts
Actual-world formal theorem proving depends closely on wealthy contextual data, which is usually absent from conventional benchmarks. To deal with this, the authors introduce miniCTX, a benchmark designed to check fashions’ skill to show theorems utilizing beforehand unseen, intensive context from actual Lean tasks and textbooks. Not like prior benchmarks, miniCTX contains giant repositories with related definitions, lemmas, and buildings. Baseline experiments present that fashions conditioned on this broader context considerably outperform these relying solely on the native state. The authors additionally present a toolkit to facilitate the enlargement of the benchmark.
Highlight Papers
ADIFF: Explaining audio difference using natural language
This paper tackles the novel job of explaining variations between audio recordings, which is necessary for functions like audio forensics, high quality evaluation, and generative audio programs. The authors introduce two new datasets and suggest a three-tiered clarification framework—starting from concise occasion descriptions to wealthy, emotionally grounded narratives—generated utilizing giant language fashions. They current ADIFF, a brand new technique that improves on baselines by incorporating audio cross-projection, position-aware captioning, and multi-stage coaching, and present that it considerably outperforms current audio-language fashions each quantitatively and through human analysis.
Better Instruction-Following Through Minimum Bayes Risk
This paper explores how LLMs can be utilized as judges to guage and enhance different LLMs. The authors present that utilizing a technique referred to as Minimal Bayes Threat (MBR) decoding—the place an LLM choose selects the perfect output from a set—can considerably enhance mannequin efficiency in comparison with normal decoding strategies. Additionally they discover that coaching fashions on these high-quality outputs can result in robust beneficial properties even with out counting on MBR at take a look at time, making the fashions quicker and extra environment friendly whereas sustaining or exceeding earlier efficiency.
DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
This paper introduces DeFT, a brand new algorithm that quickens how giant language fashions deal with duties involving tree-like buildings with shared textual content prefixes, resembling multi-step reasoning or few-shot prompting. Current strategies waste time and reminiscence by repeatedly accessing the identical knowledge and poorly distributing the workload throughout the GPU. DeFT solves this by neatly grouping and splitting reminiscence utilization to keep away from redundant operations and higher steadiness the work, resulting in as much as 3.6x quicker efficiency on key duties in comparison with present approaches.
Holistically Evaluating the Environmental Impact of Creating Language Models
This paper estimates the complete environmental influence of creating giant language fashions, together with not simply the ultimate coaching runs but additionally mannequin improvement and {hardware} manufacturing—areas sometimes underreported. The authors discovered that coaching a sequence of fashions launched 493 metric tons of carbon emissions and used 2.769 million liters of water, even in a extremely environment friendly knowledge heart. Notably, round half of the carbon emissions got here from the event part alone, and energy utilization throughout coaching various considerably, elevating considerations for power grid planning as AI programs develop.
Language Model Alignment in Multilingual Trolley Problems
This paper evaluates how nicely LLMs align with human ethical preferences throughout languages utilizing multilingual trolley issues. The authors introduce MultiTP, a brand new dataset of ethical dilemmas in over 100 languages based mostly on the Ethical Machine experiment, enabling cross-lingual evaluation of LLM decision-making. By assessing 19 fashions throughout six ethical dimensions and analyzing demographic correlations and immediate consistency, they uncover important variation in ethical alignment throughout languages—highlighting moral biases and the necessity for extra inclusive, multilingual approaches to accountable AI improvement.
Lean-STaR: Learning to Interleave Thinking and Proving
This paper introduces Lean-STaR, a framework that improves language model-based theorem proving by incorporating casual “ideas” earlier than every proof step. Not like conventional approaches that rely solely on formal proof knowledge, Lean-STaR generates artificial thought processes utilizing retrospective proof techniques throughout coaching. At inference time, the mannequin generates these ideas to information its subsequent motion, and knowledgeable iteration additional refines its efficiency utilizing the Lean theorem prover. This method boosts proof success charges and gives new insights into how structured reasoning improves formal mathematical drawback fixing.
MagicPIG: LSH Sampling for Efficient LLM Generation
This paper introduces MagicPIG, a brand new system that quickens LLM inference by approximating consideration extra effectively. Whereas many strategies assume consideration is sparse and use TopK approximations, the authors present this isn’t at all times correct and may harm efficiency. As an alternative, MagicPIG makes use of a sampling technique backed by theoretical ensures and accelerates it utilizing Locality Delicate Hashing, offloading computations to the CPU to assist longer inputs and bigger batches with out sacrificing accuracy.
Multi-Robot Motion Planning with Diffusion Models
This paper introduces a technique for planning coordinated, collision-free actions for a lot of robots utilizing solely knowledge from particular person robots. The authors mix discovered diffusion fashions with classical planning algorithms to generate real looking, secure multi-robot trajectories. Their method, referred to as Multi-robot Multi-model planning Diffusion, additionally scales to giant environments by stitching collectively a number of diffusion fashions, displaying robust leads to simulated logistics situations.
Reinforcement Learning for Control of Non-Markovian Cellular Population Dynamics
This paper explores how reinforcement studying can be utilized to develop drug dosing methods for controlling cell populations that adapt over time, resembling most cancers cells switching between resistant and vulnerable states. Conventional strategies battle when the system’s dynamics are unknown or contain reminiscence of previous environments, making optimum management tough. The authors present that deep RL can efficiently be taught efficient methods even in advanced, memory-based programs, providing a promising method for real-world biomedical functions.
Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
This paper explores enhance giant language fashions’ reasoning by giving suggestions at every step of their pondering course of, fairly than solely on the remaining reply. The authors introduce a technique the place suggestions—referred to as a course of reward—is predicated on whether or not a step helps make an accurate remaining reply extra doubtless, as judged by a separate mannequin (a “prover”) that may acknowledge progress higher than the mannequin being skilled. They present each theoretically and experimentally that this technique makes studying extra environment friendly, resulting in considerably higher and quicker outcomes than conventional outcome-based suggestions strategies.
SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models
This paper introduces SVDQuant, a technique for considerably rushing up diffusion fashions by quantizing each weights and activations to 4 bits. Since such aggressive quantization can harm picture high quality, the authors use a intelligent approach: they shift problematic “outlier” values right into a separate low-rank part dealt with with larger precision, whereas the remainder is processed with environment friendly low-bit operations. To keep away from slowing issues down resulting from further computation, additionally they design a customized inference engine referred to as Nunchaku, which merges the processing steps to reduce reminiscence entry. Collectively, these methods scale back reminiscence utilization and ship over 3x speedups with out sacrificing picture high quality.
Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation
This paper tackles the problem of making use of reinforcement studying (RL) to soft-body robotics, the place simulations are normally too sluggish for data-hungry RL algorithms. The authors introduce SAPO, a brand new model-based RL algorithm that effectively learns from differentiable simulations utilizing analytic gradients. The authors additionally current Rewarped, a quick, parallel simulation platform that helps each inflexible and deformable supplies, demonstrating that their method outperforms current strategies on advanced manipulation and locomotion duties.
Streaming Algorithms For $ell_p$ Flows and $ell_p$ Regression
This paper investigates resolve underdetermined linear regression issues in a streaming setting, the place the information arrives one column at a time and storing the complete dataset is impractical. The authors develop algorithms that approximate the regression price or output a near-optimal answer utilizing a lot much less reminiscence than storing the whole dataset—significantly related for functions like computing flows on giant graphs. Additionally they set up area decrease bounds, displaying the restrictions of what’s doable, and supply the primary algorithms that obtain nontrivial approximations utilizing sublinear area in numerous settings.
