What would cause these cuts to not be round?

Should I use a 1/4” instead and then change to the 1/8” for the small screw holes?

keep 1/8” for now, don’t change too many things at the time. i would make a toolpath for a single hole first and get it cutting good before trying to do the full part. get it cutting a single hole with reasonable accuracy and then go from there. ±0.5mm for the first cut is good enough. you’ll be able to cut much much better but to make sure that there is not structural problems with setup

watch closely when it is cutting. should be “smooth” without any jumpimg

change the ramp entry to 45 degree just to make plunges a bit easier