In general, a slammed stem is safer than one with a bunch of spacers underneath. Adding spacers underneath creates more leverage on your steerer during out of the saddle efforts, but Cervelo does allow for 50mm total below the stem, 35mm of spacers and 15mm of bearing cover..
The thing i would be concerned about is, what is your stem clamping on to?
In a normal install, (meaning no steerer above the stem), the aluminum insert is glued, and the stem then clamps onto that reinforced area of the steerer. Even skipping the insert, you'd still have an expander in there to clamp onto.
In your case, I'm assuming the insert isn't glued, because that would prohibit ever cutting the fork down, which would be even worse for resale. So, I would check to see how far down the expander goes. I use Cannondale SI expanders and they def don't reach down 40mm..I wouldn't feel safe clamping onto unsupported steerer....