ISA Server 2006 or Forefront TMG 2010 resets the connection to the SMTP client when the SMTP server closes its connection to ISA Server

Symptoms

Consider the following scenario:
  • In Microsoft Internet Security and Acceleration (ISA) Server 2006 or Forefront TMG 2010, you enable SMTP filtering.
  • An SMTP client communicates with an SMTP server through ISA Server 2006 or Forefront TMG 2010.
  • The SMTP client uses one of the following verbs in its conversation with the SMTP server:
    • TLS
    • STARTTLS
    • X-AnonymousTLS
In this scenario, ISA Server 2006 or Forefront TMG 2010 resets the connection to the SMTP client when the SMTP server closes its connection to ISA Server or Forefront TMG 2010 with a TCP FIN packet. This behavior can cause some specific SMTP client applications to report that message delivery failed even though messages are sent correctly.

In ISA logging, you see the following result code for this connection:
0x80074e24 FWX_E_CONNECTION_KILLED

Cause

By design, the connection to the SMTP client is reset when the SMTP server closes its connection to ISA Server or Forefront TMG 2010. This is because ISA Server or Forefront TMG 2010 cannot inspect the SMTP traffic for the SMTP QUIT command when the traffic is encrypted. However, this behavior can cause specific SMTP client applications to report errors.

Resolution

A hotfix is now available to avoid incorrect error reports. This hotfix allows the default behavior to be changed so that ISA Server returns a TCP FIN packet to the SMTP client instead of a RESET packet when the SMTP server closes the TCP connection with a TCP FIN packet. To change the default behavior after you install this hotfix, you have to run a script on the ISA Server computer.

To apply this hotfix, follow these steps:
  1. Download and then install the hotfix package that is mentioned in the following Microsoft Knowledge Base article:
    959357 Description of the ISA Server 2006 hotfix package: October 29, 2008

    Note: If you are running Forefront TMG 2010, copy the script from step 2 into a Notepad file and save it. Open command prompt with elevated privilege and then run the script (ie: cscript smtp.vbs).
  2. Start Notepad, and then copy the following script into a Notepad file.
    Const SE_VPS_GUID = "{143F5698-103B-12D4-FF34-1F34767DEabc}"
    Const SE_VPS_NAME = "GracefulPassthroughServerShutdown"
    Const SE_VPS_VALUE = true

    Sub SetValue()

    ' Create the root obect.
    Dim root ' The FPCLib.FPC root object
    Set root = CreateObject("FPC.Root")

    'Declare the other objects needed.
    Dim array ' An FPCArray object
    Dim VendorSets ' An FPCVendorParametersSets collection
    Dim VendorSet ' An FPCVendorParametersSet object

    ' Get references to the array object
    ' and the network rules collection.
    Set array = root.GetContainingArray
    Set VendorSets = array.VendorParametersSets

    On Error Resume Next
    Set VendorSet = VendorSets.Item( SE_VPS_GUID )

    If Err.Number <> 0 Then
    Err.Clear

    ' Add the item
    Set VendorSet = VendorSets.Add( SE_VPS_GUID )
    CheckError
    WScript.Echo "New VendorSet added... " & VendorSet.Name

    Else
    WScript.Echo "Existing VendorSet found... value- " & VendorSet.Value(SE_VPS_NAME)
    End If

    if VendorSet.Value(SE_VPS_NAME) <> SE_VPS_VALUE Then

    Err.Clear
    VendorSet.Value(SE_VPS_NAME) = SE_VPS_VALUE

    If Err.Number <> 0 Then
    CheckError
    Else
    VendorSets.Save false, true
    CheckError

    If Err.Number = 0 Then
    WScript.Echo "Done with " & SE_VPS_NAME & ", saved!"
    End If
    End If
    Else
    WScript.Echo "Done with " & SE_VPS_NAME & ", no change!"
    End If

    End Sub

    Sub CheckError()

    If Err.Number <> 0 Then
    WScript.Echo "An error occurred: 0x" & Hex(Err.Number) & " " & Err.Description
    Err.Clear
    End If

    End Sub

    SetValue
  3. Save the file as a Microsoft Visual Basic script file by using the .vbs file name extension.
  4. Double-click the .vbs file to run the script.
Note If you want to revert to the default behavior, change the following line of the script:
Const SE_VPS_VALUE = true

Change this line to the following:
Const SE_VPS_VALUE = false

Then, rerun the script on the ISA Server or Forefront TMG computer.

Status

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

References

For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:
824684 Description of the standard terminology that is used to describe Microsoft software updates
Özellikler

Makale No: 959312 - Son İnceleme: 12 Şub 2011 - Düzeltme: 1

Geri bildirim