FIX: A nonweb proxy client in a Forefront Threat Management Gateway (TMG) 2010 environment cannot open certain load-balanced websites when TMG HTTPS inspection is enabled

Article translations Article translations
Article ID: 2780594 - View products that this article applies to.
Expand all | Collapse all

Symptoms

Consider the following scenario:
  • You use a client that is not a web proxy client such as a firewall client or a SecureNAT client on your corporate network.
  • The client tries to connect to an HTTPS website through a server that is running Microsoft Forefront Threat Management Gateway (TMG) 2010. For example, the client tries to connect to https://contoso.com.
  • HTTPS inspection is enabled on the server that is running TMG 2010.
  • The HTTPS website uses a special Domain Name System (DNS)-based load-balancing algorithm in which the responsible DNS server returns an alternating IP address that has a low "Time to Live" value. In this case, successive queries for the website (contoso.com) result in different IP addresses.

In this scenario, the client may be unable to browse the website. Additionally, the following may be logged in the Application log on the server that is running TMG 2010:
Status 12227 The name on the SSL server certificate supplied by a destination server does not match the name of the host requested.

Cause

This issue occurs because of the special DNS-based load-balancing algorithm.

When a nonweb proxy client opens a website such as https://contoso.com, the client resolves the name itself and tries to establish a Secure Sockets Layer (SSL) connection to the destination IP address such as, for example, IP-1.

When HTTPS inspection is enabled, TMG 2010 establishes a connection on behalf of the client and tries to validate the server certificate before the client connection is enabled. One of the many checks that are performed (for example, validity and revocation) verifies that the correct website is being connected.

The verification is performed as follows:
  1. TMG 2010 reads the Subject Name and Subject Alternate Name fields of the retrieved certificate, such as Contoso.com in this case.
  2. TMG 2010 tries to resolve the certificate by using DNS.
  3. TMG 2010 checks whether the result matches the destination IP address that the client used when the connection was established.

Because of how load-balancing is resolved for the website, name resolution in TMG 2010 yields a different IP address, IP-2. Therefore, because the two IP addresses are not the same (IP-1 versus IP-2), TMG 2010 denies the connection.

Resolution

To resolve this problem, install the hotfix package that is described in the following Microsoft Knowledge Base article:
2735208 Rollup 3 for Forefront Threat Management Gateway (TMG) 2010 Service Pack 2
Note After you install this fix, you can configure an exclusion Domain Name Set for HTTPS-inspection IP address validation. These are domain names for which the address validation part of the HTTPS-inspection is suppressed. However, other validation steps are still performed.

The following script configures the exclusion Domain Name Set to be the one that is named in the beginning of the script. The script also creates the exclusion Domain Name Set if it does not already exist. By using the regular TMG management tools such as the administrative console and scripting, the administrator can populate the DomainNameSet variable as appropriate.


' The domain name set for the exclusion list
const strDomainNameSetName = _
    "HTTPS-inspection IP address validation exception"
const strDomainNameSetDescription = "HTTPS sites whose certificate's domain-name matches this set are excluded from the validation that the domain-name indeed resolves to the IP address that the Web client originally connected to"
 
 
Const strVpsGUID = "{143F5698-103B-12D4-FF34-1F34767DEABC}"
Const strVpsPropertyName = "HTTPSiIpAddressValidationExclusionDNSet"
Const Error_FileNotFound = &H80070002
 
Set objArray = CreateObject("FPC.Root").GetContainingArray()
Set objDNSet = OpenDNSet( _
                    objArray.RuleElements.DomainNameSets, _
                    strDomainNameSetName, _
                    strDomainNameSetDescription _
                    )
Set objVPSet = OpenVPSet(objArray.ArrayPolicy, strVpsGUID)
objVPSet.Value(strVpsPropertyName) = objDNSet.PersistentName
objArray.Save
 
function OpenDNSet(objDNSets, strDNSetName, strDNSetDescription)
    On Error Resume Next
    Set objDNSet = objDNSets.Item(strDNSetName)
    ' Save the Err properties in case it needs to be re-raised
    errNumber      = Err.Number
    errSource      = Err.Source
    errDescription = Err.Description
    errHelpFile    = Err.HelpFile
    errHelpContext = Err.HelpContext
    
    On Error GoTo 0
    
    if errNumber = Error_FileNotFound Then
        Set objDNSet = objDNSets.Add(strDNSetName)
        objDNSet.Description = strDNSetDescription
    Elseif errNumber < 0 Then
        ' An error other than "file not found" occured -- re-raise the error,
        ' this time not under "On Error Resume Next"
        Err.Raise errNumber, errSource, errDescription, errHelpFile, errHelpContext
    End If
     
    Set OpenDNSet = objDNSet
end function
 
function OpenVPSet(objParent, strVpsGUID)
    Set objVPSets = objParent.VendorParametersSets
    On Error Resume Next
    Set OpenVPSet = objVPSets.Item(strVpsGUID)
    ' Save the Err properties in case it needs to be re-raised
    errNumber      = Err.Number
    errSource      = Err.Source
    errDescription = Err.Description
    errHelpFile    = Err.HelpFile
    errHelpContext = Err.HelpContext
    
    On Error GoTo 0
    
    if errNumber = Error_FileNotFound Then
        Set OpenVPSet = objVPSets.Add(strVpsGUID)
    Elseif errNumber < 0 Then
        ' An error other than "file not found" occured -- re-raise the error,
        ' this time not under "On Error Resume Next"
        Err.Raise errNumber, errSource, errDescription, errHelpFile, errHelpContext
     End If
end function

Workaround

To work around this issue, configure the client to act as a web proxy client. In this case, name resolution occurs only on the TMG server. Or, exclude the affected client from HTTPS inspection, or exclude the problematic destination website, as described on the following Microsoft TechNet website:
Excluding sources and destinations from HTTPS inspection

Status

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

References

For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:
824684 Description of the standard terminology that is used to describe Microsoft software updates

Properties

Article ID: 2780594 - Last Review: January 10, 2013 - Revision: 4.0
Applies to
  • Microsoft Forefront Threat Management Gateway 2010 Service Pack 2, when used with:
    • Microsoft Forefront Threat Management Gateway 2010 Enterprise
    • Microsoft Forefront Threat Management Gateway 2010 Standard
Keywords: 
kbqfe kbfix kbexpertiseinter kbbug kbsurveynew KB2780594

Give Feedback

 

Contact us for more help

Contact us for more help
Connect with Answer Desk for expert help.
Get more support from smallbusiness.support.microsoft.com