How to deploy an SSTP-based VPN server behind a SSL load balancer in Windows Server 2008

Article translations Article translations
Article ID: 947030 - View products that this article applies to.
Beta Information
This article discusses a beta release of a Microsoft product. The information in this article is provided as-is and is subject to change without notice.

No formal product support is available from Microsoft for this beta product. For information about how to obtain support for a beta release, see the documentation that is included with the beta product files, or check the Web location where you downloaded the release.
Important This article contains information about how to modify the registry. Make sure that you back up the registry before you modify it. Make sure that you know how to restore the registry if a problem occurs. For more information about how to back up, restore, and modify the registry, click the following article number to view the article in the Microsoft Knowledge Base:
322756 How to back up and restore the registry in Windows XP and Windows Vista
Expand all | Collapse all

On This Page

INTRODUCTION

Consider the following scenario:
  • You have a Secure Socket Tunneling Protocol (SSTP)-based VPN server that is running Windows Server 2008.
  • The VPN server is located behind an SSL load balancer.
  • The server is assigned a private IP address.
  • The server has a public IP address that is registered to a DNS server.
This article describes how to deploy an SSTP-based VPN server behind an SSL load-balanced computer that is running Windows Server 2008.

SSTP is a new kind of VPN tunnel that is available in the Routing and Remote Access Server role in Windows Server 2008. SSTP enables the encapsulation of Point-to-Point Protocol (PPP) packets over HTTP. This lets you more easily establish a VPN connection through a firewall or through a Network Address Translation (NAT) device. This also lets you establish a VPN connection through an HTTP proxy device.

In large organizations, it is common to configure an SSL load balancer to close an HTTPS connection and then to open an HTTP connection to a Web server or to a Routing and Remote Access server.

MORE INFORMATION

Overview

The information in this article applies to the following network-configuration scenario:
  • The SSL load balancer has an IP address of 1.2.3.4.
  • The SSL load balancer has the DNS name of server.contoso.com.
  • There are two Routing and Remote Access servers that are located in a perimeter network (also known as DMZ, demilitarized zone, and screened subnet). One of these servers has an IP address of 1.2.3.5, and the other server has an IP address of 1.2.3.6.

Configuration information

Warning Serious problems might occur if you modify the registry incorrectly by using Registry Editor or by using another method. These problems might require that you reinstall the operating system. Microsoft cannot guarantee that these problems can be solved. Modify the registry at your own risk.

To configure an SSTP-based VPN server in the scenario that is described in the "Overview" section, follow these steps:
  1. Configure the SSL load balancer to close SSTP-based HTTPS connections that are directed to a specific uniform resource identifier (URI) in a pool of Routing and Remote Access servers. (These servers have IP addresses of 1.2.3.5 and 1.2.3.6, and they use port 80.) The following is an example of a URI:
    https://host_name:443/sra_{ BA195980-CD49-458b-9E23-C84EE0ADCD75})
  2. Install a computer certificate on the Windows Server 2008-based computer that serves as the SSL load balancer. The computer certificate must have the EKU type set to Server Authentication or to All Purpose. This certificate must have a subject name (CN) that is the same as the host name to which the VPN clients connect. This configuration is required for successful SSL negotiation.

    For example, consider the following scenario:
    • If a VPN client is configured to connect to the public IP address of the NAT device (1.2.3.4), the subject name of the certificate must also be 1.2.3.4.
    • If a VPN client is configured to connect to the FQDN (server.contoso.com) that can be accessed publicly, the subject name of the certificate must also be server.contoso.com.
  3. If you want to close the HTTP connection on the Routing and Remote Access servers, follow these steps:
    1. Set the following registry key to a value of 0 so that the Routing and Remote Access server can listen on a specific HTTP connection:
      HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Sstpsvc\Parameters\UseHTTPS
    2. By default, Routing and Remote Access servers listen on port 443 if the UseHTTPS option is set to 1 or on port 80 if the UseHTTPS option is set to 0. Complete steps 3c-3g if you want the Routing and Remote Access server to listen on another port. If you do not want the Routing and Remote Access server to listen on another port, go to step 3h.
    3. Start Registry Editor, and then locate the following registry subkey:
      HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SstpSvc\Parameters
    4. In the details pane, right-click ListenerPort, and then click Modify.
    5. Click Decimal, type an alternative port number such as 5000, and then click OK.
    6. Exit Registry Editor.
    7. Restart the Routing and Remote Access service.
    8. Even though HTTPS connections are closed on the SSL load balancer, the Routing and Remote Access server needs the hash value of the computer certificate that is installed on the SSL load balancer to improve security. To make sure that the Routing and Remote Access server has the hash value of the computer certificate that is installed on the SSL load balancer, configure the following REG_BINARY subkey by using the SHA256 certificate hash of the computer certificate that is installed on the SSL load balancer:
      HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Sstpsvc\Parameters\Sha256CertificateHash
      Note If you cannot locate the SHA256 certificate hash of the computer certificate that is installed on the SSL load balancer, complete steps 3i-3k to configure a temporary value for the Sha256CertificateHash registry key.
    9. On a different computer, create a VPN connection that is SSTP-based and for which the VPN destination is the host name of the SSL load balancer.
    10. Click Connect to connect the VPN connection. The connection attempt fails. Open Event Viewer on the client computer, and then locate the following event:
      Cryptographic binding failed for the client connection. The reason is the client and the server have mismatched certificate hashes configured. SHA1 Certificate Hash: ... SHA256 Certificate Hash: ...
    11. Note the value of SHA256 certificate hash in the event on the client computer, and then use this value for the Sha256CertificateHash registry key on the Routing and Remote Access server.
    12. Restart the Routing and Remote Access server.
    13. Reestablish the SSTP connection.
  4. Install the Routing and Remote Access server role by using Server Manager on all the Routing and Remote Access servers.
  5. Configure the Routing and Remote Access server by running through the Routing and Remote Access Configuration Wizard.
  6. If Windows Firewall is enabled on the Routing and Remote Access server, manually open the appropriate port number, such as port 80, in Windows Firewall. If the Routing and Remote Access server's incoming and outgoing filters are enabled on the Routing and Remote Access server, manually open the appropriate port number on the Routing and Remote Access server's incoming and outgoing filters.
  7. Set the Routing and Remote Access server to listen for HTTP-based VPN connections on a specific port number.

REFERENCES

For more information, click the following article number to view the article in the Microsoft Knowledge Base:
947031 How to troubleshoot Secure Socket Tunneling Protocol (SSTP)-based connection failures in Windows Server 2008

Properties

Article ID: 947030 - Last Review: February 7, 2008 - Revision: 1.2
APPLIES TO
  • Windows Server 2008 Datacenter
  • Windows Server 2008 Enterprise
  • Windows Server 2008 for Itanium-Based Systems
  • Windows Server 2008 Standard
  • Windows Web Server 2008
Keywords: 
kbexpertiseinter kbhowto kbinfo KB947030

Give Feedback

 

Contact us for more help

Contact us for more help
Connect with Answer Desk for expert help.
Get more support from smallbusiness.support.microsoft.com