This is a great description of the problem!
It’s common to get dimension issues: Parts Too Big and Holes Too Small
Your errors are consistent with what I’ve had in the past. Shorter in one dimension and larger in the other (with vs against grain?) but it’s also a little like “outside” cuts are larger than “inside” cuts (parts too big holes too small) which I’ve also struggled with, even to drill dog holes in the spoilboard.
I haven’t found a magic bullet yet either, sorry. I do the same as @turbinbjorn and cut test pieces/holes.