Short answer is that Z_offset is only utilized / honored whenever a bed mesh is created, so unless you do a new bed mesh (“Auto Bed Level” button in Mainsail macros section) after doing the PROBE_CALIBRATE and SAVE_CONFIG then your Z_Offset goes unused.
More details / how-to in a long post here:
Oh wow. Interesting.
You can also load the bed mesh with BED_MESH_PROFILE LOAD.
I only probe my bed when I move it. It does not load any mesh by default.
The offset has given me lots of trouble. I wonder if this is exactly why.
I guess the ability to have different meshes stored could be handy for some people, especially when printing with different nozzles and different default layer heights! Since I mostly print with the same nozzle all the time, and mostly with the same filament, PLA, I got my Z_Offset good, and then did one good mesh and made it my default. I get great results this way.