Ok, I have figured out the problem.
Previously after uninstalling HGTortoise 0.x, it asked me to reboot, which I did.
I then installed HTTortoise 1.0 and it did not ask me to reboot.
Looking at my windows PATH environment variable through 'my computer', I can see HGTortoise path there.
If I go to a cmd prompt and run HGTK, it works.
However, if running an example like the above from VS or .net program, it cannot find it. Printing the environment variable from C# code shows the HGTortoise path is NOT present.
Rebooted machine, now everything works fine.
Here is an article on the path not updating unless you restart explorer.exe or make an API call: