You are using System Center Essentials (SCE) 2010 and you try to designate a host that is in a workgroup and not in a domain. The result is that it fails with the following error:
An internal error has occurred trying to contact an agent on the <server-name> server.
Ensure the agent is installed and running. Ensure the WS-Management service is installed and running, then restart the agent.
This can occur when the host is not in the domain and instead is in a workgroup, causing the authentication between them to fail.
Also SCE from its SCE console, does not support or in GUI of Designate a host, does not give option of adding a host from perimeter network, so you have to manually add the perimeter host using VMM Powershell available on SCE server.
You can take the steps below to make this work:
1. Connect to host computer via RDP, from control panel, Add/remove programs, un-install VMM 2008 R2 Agent. (If already installed manually).
2. Using source media for SCE, manually install VMM 2008 R2 Agent from CD drive x:\setup\vmm\amd64\SetupVMM.exe, select option to install Local Agent.
3. In this wizard, select option “This host is on a perimeter network” and specifiy password for Security file encryption key and do not select to use CA certificate.
4. Then select Use Local computer name and click next and install.
5. This installs VMM agent on that host. (You have to copy that SecurityFile.txt file from Host server to SCE server).
6. Now on SCE server, open Start Menu – Microsoft System Center – Virtual Machine Manager 2008 R2 – Windows Power Shell – Virtual Machine Manager.
7. In the power shell run below commands to add the host
a. >Get-VMMServer -ComputerName "<SCE-server-FQDN>" - (This server connects to SCE server which is a VMM server also)
b. >$Key = Get-Credential
(This prompts for user name and password, specify user name as Administrator and password as same which you specified during manual agent install and generating security key.)
c. >Add-VMHost "<hostname>" -Description "Perimeter host" -RemoteConnectEnabled $False -PerimeterNetworkHost -SecurityFile "C:\temp\SecurityFile.txt" -EncryptionKey $Key - (Note: Location of security file is any location where you copied file from host to SCE server)
8. Now Host should get added fine.
9. You may see below event in VM Manager event log on SCE Server:
Log Name: VM Manager
Source: Virtual Machine Manager
Date: <date, time>
Event ID: 1705
Task Category: None
Job 2bb6c79f-24de-4a9a-a279-036d72165b2e (Add perimeter network host) completed successfully.
10. Now in SCE console, you will see this computer as host.
11. In SCE console, below All Virtual Machines view, you will now see all VMs hosted by this host.
12. But while connecting to it, you may get prompted for credentials, after supplying correct user name and password, it shows one more prompt saying certificate is not trusted.
13. So from this prompt click on view certificate, go to details tab, click on copy to file, export certificate as .cer file.
14. From SCE server, MMC console, certificate snap-in, import this certificate under Trusted Root Certification Authorities.
15. Now exit and reopen SCE console.
16. SCE Console should be now able to see the host, all VMs and connect to all VMs fine.
“You can designate and configure servers as hosts for virtual machines in Essentials 2010 from the Essentials management server or from the Essentials console. You can also set up and manage the virtual machines on hosts in trusted domains, workgroups, or perimeter networks.”You can find this in the Technet Library at this URL: http://technet.microsoft.com/en-us/library/ff603625.aspx