Yeah the staggering towards you is sooo annoying. Not because it makes it harder, it´s just so utterly wrong. It bothers me every single time. I gave him a good smash with a steel club or whatever i use. The chances of that actually happening when you hit someone are very near to zero.
Sadly i think it´s intentional. And a really cheap way of creating more difficulty.