Part of that is just inkscape doing its best. You might be able to get better results by playing with the options in trace bitmap. I like the ones in the bottom pane for things like this, with fewer scans.
In practice, you’ll probably want to edit the svg when you’re done by selecting the node tool and removing everything that isn’t a corner.
It’s worth considering just finding a font you like and setting that up, then using object to path. It would certainly be faster.