PRB: ISAXXMLReader::parseURL Consumes Large Amount of Memory Over HTTP

View products that this article applies to.

This article was previously published under Q291844

SYMPTOMS

When you use code such as the following, note that the parseURL method of the SAXXMLReader object consumes an unexpectedly large amount of memory when parsing a large XML file over HTTP:

Set saxReader = CreateObject("Msxml2.SAXXMLReader.3.0")
saxReader.putFeature "server-http-request", true

saxReader.parseURL "http://localhost/big.xml"

You expect that the parseURL method would not consume much more memory than the actual size of the XML file.

Back to the top

The current implementation of the WinHttpRequest component that is used by the ServerXMLHTTP object does not support "pull" semantics for retrieving the response data; that is, the WinHTTPRequest does not support fetching the response off the wire in chunks. As a result, the response data gets double buffered in the WinHttpRequest component.

The component itself maintains one copy internally and then makes a second copy to make the responseText, responseBody, and responseStream properties available for use with scripting languages. The double buffering may lead to large memory footprints when using the Simple API for XML (SAX) or Document Object Model (DOM) components to process large XML files over HTTP.

Back to the top

STATUS

Microsoft has confirmed that this is a problem in the Microsoft products that are listed at the beginning of this article.

Memory usage is reduced when you use Microsoft XML version 3.0, Service Pack 1 or MSXML 4.0.

Back to the top

MORE INFORMATION

Steps to Reproduce Behavior

Create the large XML document about 100 MB in size by following these steps:
1. In Microsoft Visual Basic, create a new Standard EXE project. Form1 is created by default.
2. Add a Command button to Form1. Command1 is created by default.
3. Paste the following code into the Click event for Command1:
  On Error GoTo handler Open "c:\inetpub\wwwroot\big.xml" For Output As #1 Print #1, "<?xml version=""1.0"" ?>" Print #1, "<XML>" Dim i As Long For i = 1 To 100000 Print #1, "<t>" & String(1000, "o") & "</t>" Next i Print #1, "</XML>" Close #1 Exit Sub handler: MsgBox Err.Number & ":" & Err.Description, vbExclamation, Err.Source
4. Create the test VBScript file: Paste the following code into the new text file:
  To run the code with MSXML 4.0 parser, change:
  Set saxReader = CreateObject("Msxml2.SAXXMLReader.3.0")
  To
  Set saxReader = CreateObject("Msxml2.SAXXMLReader.4.0")
5. Press the F5 key to run the project from Visual Basic.
  
  NOTE:The program creates a file of about 100 MB in size. The program takes quite a while to run. Be sure that you have sufficient free hard disk drive space. Be sure to delete the XML file after you complete the test below.

Create the test VBScript file:

Using Windows Notepad or another text editor, create a new text file.

Paste the following code into the new text file:

dim saxReader
Set saxReader = CreateObject("Msxml2.SAXXMLReader.3.0")
saxReader.putFeature "server-http-request", true
saxReader.parseURL "http://localhost/big.xml"
WScript.Echo "Done! saxReader.parseURL"

Save the text file as C:\Big.VBS.

Open Windows NT Performance Monitor to monitor memory usage:
1. In Microsoft Windows NT or Microsoft Windows 2000, press the CTRL+ALT+DEL keypad combination.
2. On the dialog box that is displayed, select Task Manager.
3. In Task Manager, click the Performance tab, and note the graph of Memory Usage History.
4. Size and position Performance Monitor so that you can see the Memory Usage History but still work on the Windows Desktop.
Test:
1. On the Windows Start menu, select Run.
2. In the Run dialog box, type the following command:
  wscript c:\big.vbs
3. Click OK, and note memory usage in Performance Monitor.