"In this case the host OS needs to have special support for passing through the PCI card and a non specialized OS like Windows or Linux sadly has no support for that."
I know this is an ageing thread, but I feel i need to point out for anyone arriving via google. that this is at least no longer true, from personal experience, linux does in fact support PCI Passthrough as long as the board has IOMMU support in the bios (and respective VTx on the processor), the problem is not the Host system, so much as it is software support, At present, qemu is the only apparent solution that gives you the option for pci passthrough devices, though it is a little fiddly to set up, but fairly well documented online.
here is a video of this feature running on bog standard Desktop Linux/Ubuntu
https://www.youtube.com/watch?v=37D2bRsthfI
you are right to point out that the host loses access to the card when it becomes assigned to the virtual machine, but Linux is able to work completely headless, and remains active even if a graphics card is not present. this particular setup in the video also has 2 PCI graphics cards leaving one available to the host system at all times, i've managed to get this working with using the onboard gfx as the boot device, and assigning the disused PCI gfx card to a VM with success.