Didn't see your reply before writing my own, sorry about that LucD!
Yes, the user has logged on multiple times with this account.
I can run the "Install-PackageProvider nuget -force" command successfully each time, so powershell is correctly started and can add this provider, it's more like subsequent runs of powershell fails. Will tinker some more tomorrow and have a user logged on to the VM while running the command (which I did during development of the script, wanted to make sure things were happening).