I would say that this is almost certainly a tramming issue.
A tiny tilt in the bit angle will do some strange things.
While the bit is angled so that the leading edge is lower than the trailjng edge, you will get a nice clean cut and good finish. As soon as the trailing edge is lower, you get all kinds of weirdness because part of the bit that is not meant for cutting comes in contact with the material.