FIX: ITC OpenUrl Method Does Not Return Complete Files

Retired KB Content Disclaimer

This article was written about products for which Microsoft no longer offers support. Therefore, this article is offered "as is" and will no longer be updated.

Symptoms

When you use the OpenUrl method of Microsoft Internet Transfer Control (ITC) to download files from Web servers, the resulting files may not be complete if the server uses "chunked" transfer-encoding to send the HTTP response data. The chunked method is used by Internet Information Server (IIS) to optimize the sending of dynamically generated content, such as that sent by Active Server Pages (ASP).

Cause

ITC allocates a buffer of 64K bytes and calls InternetReadFile in a loop. Whenever the buffer containing the read data is not full after the API returns, ITC will exit the loop. However, InternetReadFile can return when less than the total number of bytes requested were read into the buffer, which causes the problem. This is the case if the transfer-encoding is chunked.

Resolution

A way to work around this problem is to call the WinInet APIs directly in Visual Basic, as described in the following two steps:


  1. Add a module to the project and paste the following code into it:
    Option Explicit

    Public Const INTERNET_OPEN_TYPE_PRECONFIG = 0
    Public Const INTERNET_OPEN_TYPE_DIRECT = 1
    Public Const INTERNET_OPEN_TYPE_PROXY = 3

    Public Const scUserAgent = "VB OpenUrl"
    Public Const INTERNET_FLAG_RELOAD = &H80000000

    Public Declare Function InternetOpen Lib "wininet.dll" Alias "InternetOpenA" _
    (ByVal sAgent As String, ByVal lAccessType As Long, ByVal sProxyName As String, _
    ByVal sProxyBypass As String, ByVal lFlags As Long) As Long

    Public Declare Function InternetOpenUrl Lib "wininet.dll" Alias "InternetOpenUrlA" _
    (ByVal hOpen As Long, ByVal sUrl As String, ByVal sHeaders As String, _
    ByVal lLength As Long, ByVal lFlags As Long, ByVal lContext As Long) As Long

    Public Declare Function InternetReadFile Lib "wininet.dll" _
    (ByVal hFile As Long, ByVal sBuffer As String, ByVal lNumBytesToRead As Long, _
    lNumberOfBytesRead As Long) As Integer

    Public Declare Function InternetCloseHandle Lib "wininet.dll" _
    (ByVal hInet As Long) As Integer
  2. Paste the following code into the form:
    Private Sub Command1_Click()
    Dim hOpen As Long
    Dim hOpenUrl As Long
    Dim sUrl As String
    Dim bDoLoop As Boolean
    Dim bRet As Boolean
    Dim sReadBuffer As String * 2048
    Dim lNumberOfBytesRead As Long
    Dim sBuffer As String

    sUrl = "http://www.microsoft.com"

    hOpen = InternetOpen(scUserAgent, INTERNET_OPEN_TYPE_PRECONFIG, vbNullString, vbNullString, 0)
    hOpenUrl = InternetOpenUrl(hOpen, sUrl, vbNullString, 0, INTERNET_FLAG_RELOAD, 0)

    bDoLoop = True
    While bDoLoop
    sReadBuffer = vbNullString
    bRet = InternetReadFile(hOpenUrl, sReadBuffer, Len(sReadBuffer), lNumberOfBytesRead)
    sBuffer = sBuffer & Left$(sReadBuffer, lNumberOfBytesRead)
    If Not CBool(lNumberOfBytesRead) Then bDoLoop = False
    Wend

    Open "C:\Temp\log.txt" For Binary Access Write As #1
    Put #1, , sBuffer
    Close #1

    If hOpenUrl <> 0 Then InternetCloseHandle (hOpenUrl)
    If hOpen <> 0 Then InternetCloseHandle (hOpen)
    End Sub

Status

Microsoft has confirmed that this is a bug in the Microsoft products that are listed at the beginning of this article. This bug was corrected in the next service pack for Visual Studio 6.0.

For additional information about Visual Studio service packs, click the following article numbers to view the articles in the Microsoft Knowledge Base:
194022 INFO: Visual Studio 6.0 Service Packs, What, Where, Why

194295 HOWTO: Tell That a Visual Studio Service Pack Is Installed
To download the latest Visual Studio service pack, visit the following Microsoft Web site:

References

For an example of how to call the WinInet APIs directly from Visual Basic, please see the following article in the Microsoft Knowledge Base:
175179 FILE: VBFTP.EXE: Implementing FTP Using WinInet API from VB
Properties

Article ID: 232194 - Last Review: Aug 7, 2007 - Revision: 1

Feedback