FIX: Error message occurs when you try to access a web server that requires a client certificate if HTTPS inspection is enabled in Forefront TMG 2010: "Error Code: 502 Proxy Error. The message received was unexpected or badly formatted. (-2146893018)"

Article ID: 982550 - View products that this article applies to.
Expand all | Collapse all

SYMPTOMS

Consider the following scenario:
  • You enable HTTPS inspection in Microsoft Forefront Threat Management Gateway (TMG) 2010.
  • You try to access a web server that requires a client certificate from the client.
  • You type the server name of this web server in the destination exception list for HTTPS inspection.
In this scenario, you cannot access the web server and receive the following error message:
Error Code: 502 Proxy Error. The message received was unexpected or badly formatted. (-2146893018)

CAUSE

This issue occurs because Forefront TMG 2010 sends an empty client certificate to the web server during the initial SSL handshake. Some web servers like IIS web servers accept and renegotiate the client certificate when a request that contains an empty client certificate is sent. However, other web servers may return an SSL error when the web server receives the empty client certificate. This behavior causes Forefront TMG not to correctly handle the server error.

RESOLUTION

Microsoft provides programming examples for illustration only, without warranty either expressed or implied. This includes, but is not limited to, the implied warranties of merchantability or fitness for a particular purpose. This article assumes that you are familiar with the programming language that is being demonstrated and with the tools that are used to create and to debug procedures. Microsoft support engineers can help explain the functionality of a particular procedure, but they will not modify these examples to provide added functionality or construct procedures to meet your specific requirements.

To resolve this problem, follow these steps:
  1. Install Forefront Threat Management Gateway 2010 Service Pack 1.
    For more information about how to obtain Forefront Threat Management Gateway 2010 Service Pack 1, click the following article number to view the article in the Microsoft Knowledge Base:
    981324 List of problems that are fixed in Forefront Threat Management Gateway 2010 Service Pack 1
  2. On the Destination Exceptions tab of the HTTPS Outbound Inspection dialog box, select the No Validation option for the web server.

    Note When the No Validation option is set, Forefront TMG 2010 cannot retrieve and validate the server certificate of Forefront TMG 2010.
  3. Start Notepad, and then copy the following script into a Notepad file.
    Const SE_VPS_GUID = "{143F5698-103B-12D4-FF34-1F34767DEabc}"
    Const SE_VPS_NAME = "EnableHotfix982550"
    Const SE_VPS_VALUE = true
    
    Sub SetValue()
    
        ' Create the root object.
        Dim root  ' The FPCLib.FPC root object
        Set root = CreateObject("FPC.Root")
    
        'Declare the other objects that are needed.
        Dim array       ' An FPCArray object
        Dim VendorSets  ' An FPCVendorParametersSets collection
        Dim VendorSet   ' An FPCVendorParametersSet object
    
        ' Get references to the array object
        ' and the network rules collection.
        Set array = root.GetContainingArray
        Set VendorSets = array.VendorParametersSets
    
        On Error Resume Next
        Set VendorSet = VendorSets.Item( SE_VPS_GUID )
    
        If Err.Number <> 0 Then
            Err.Clear
    
            ' Add the item
            Set VendorSet = VendorSets.Add( SE_VPS_GUID )
            CheckError
            WScript.Echo "New VendorSet added... " & VendorSet.Name
    
        Else
            WScript.Echo "Existing VendorSet found... value- " &  VendorSet.Value(SE_VPS_NAME)
        End If
    
        if VendorSet.Value(SE_VPS_NAME) <> SE_VPS_VALUE Then
    
            Err.Clear
            VendorSet.Value(SE_VPS_NAME) = SE_VPS_VALUE
    
            If Err.Number <> 0 Then
                CheckError
            Else
                VendorSets.Save false, true
                CheckError
    
                If Err.Number = 0 Then
                    WScript.Echo "Done with " & SE_VPS_NAME & ", saved!"
                End If
            End If
        Else
            WScript.Echo "Done with " & SE_VPS_NAME & ", no change!"
        End If
    
    End Sub
    
    Sub CheckError()
    
        If Err.Number <> 0 Then
            WScript.Echo "An error occurred: 0x" & Hex(Err.Number) & " " & Err.Description
            Err.Clear
        End If
    
    End Sub
    
    SetValue
  4. Save the file as a Microsoft Visual Basic script file by using the .vbs file name extension. For example, save the file by using the following name:
    EnableHotfix982550.vbs
  5. Double-click the .vbs file to run it.

STATUS

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

REFERENCES

For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:
824684 Description of the standard terminology that is used to describe Microsoft software updates

Properties

Article ID: 982550 - Last Review: June 22, 2010 - Revision: 2.0
APPLIES TO
  • Microsoft Forefront Threat Management Gateway 2010 Enterprise
  • Microsoft Forefront Threat Management Gateway 2010 Standard
Keywords: 
kbexpertiseinter kbqfe kbfix kbsurveynew KB982550

Give Feedback

 

Contact us for more help

Contact us for more help
Connect with Answer Desk for expert help.
Get more support from smallbusiness.support.microsoft.com