Combatting ill-conditioning and heavy-tailed noise in non-convex optimization