Refusal Beyond a Single Direction: A Preliminary Comparison of Diff-in-Means and INLP
Comparing linear intervention techniques to decode and steer refusal behavior in safety-tuned language models.
Arditi et al. (2024) has shown that refusal in safety fine-tuned chat models is mediated by a single linear direction in the residual stream, recoverable by a difference-in-means (DiM) of harmful and harmless activations. We compare DiM-based interventions (activation addition an...
ArXiv CS.AI
Source
Generated insight by OnAirToday AI
This research reveals that AI safety mechanisms are not monolithic, demonstrating that suppressing refusal behaviors is significantly more effective by pushing activations toward opposite concepts rather than simply neutralizing them. These findings offer a more nuanced framework for controlling model behavior while maintaining performance, highlighting the geometric complexities inherent in steering neural network outputs.
Comments
Keep comments relevant and respectful. No links, advertising, self-promotion, or spam.