Looking at this article: http://www.yellow-bricks.com/2013/05/06/dynamic-versus-static-overhead-memory/
User "Nitro" posted a few graphs (in the comments) and this one http://files.hypervisor.fr/img/OVHD/ovhd_swmmu_1011gb.png really looks similar to my behaviour. 32vCPU, 512GB ~ 60000MB (60GB) of overhead.
Now, I am wondering what vCenter/ESXi selects automatically for MMU with my brand new top of the line servers. According to this doc http://communities.vmware.com/docs/DOC-9882, it should be all hardware. I'll talk to my SQL admin right away and ask him if I could reboot the server since it is still not in production yet. The sooner the better. I'll try all CPU/MMU virtualization options...