I agree with you. The side rails do support the weight and can sag if they are very long without supports, but the two gantry rails are more important for stiffness. If it were me and I were reducing cost, I would lighten the side rails and not the gantry rails.
On my double-decker I used 3/4" EMT for the side rails and 1" welded 0.120 tube for the gantry and Z. (I was remixing a bunch of parts anyway so it was not an issue to mix sizes.)