Have you tried downloading the vSphere VM directly using Fusion?
refer to
Downloading virtual machines from VMware vSphere to VMware Fusion Pro
Other option would be to export the VM in vSphere as OVF / OVA . Then, import that OVF file using Fusion.