Finding Actual Descent Directions for Adversarial Training

February 1, 2023

The inner step in standard adversarial training is not, in general, a descent direction on the robust loss because Danskin doesn’t apply to non-convex settings. So the loop is optimizing something other than what you wrote down, and you need to pick a safe descent direction (we propose a method based on norm minimization in the space spanned by directions).