On Preference Coverage Collapse from Hindsight Relabeling in Multi-Objective Reinforcement Learning
Uncovering how retroactive goal assignment causes performance instability in preference-conditioned reinforcement learning.
Hindsight relabeling which retroactively replacing a transition's goal with the outcome the agent actually achieved is an effective tool for improving sample-efficiency in Reinforcement Learning (RL). A natural extension to preference-conditioned multi-objective RL (MORL) relabel...
ArXiv CS.LG
Source
Generated insight by OnAirToday AI
The research identifies a critical failure mode where hindsight relabeling creates a feedback loop that forces agents to prioritize only a subset of objectives. This insight is essential for building robust multi-objective systems that can balance competing user preferences without collapsing into narrow behavioral patterns.
Comments
Keep comments relevant and respectful. No links, advertising, self-promotion, or spam.