How To Determine When a Shelled Process Has Terminated

Executing the Shell() function in a Visual Basic for Windows program starts another executable program asynchronously and returns control to the Visual Basic application. The shelled program continues to run indefinitely until the user closes it -- not until your Visual Basic program terminates. However, your program can wait until the shelled program has finished by polling the return value of the Windows API GetModuleUsage() function. This article describes the method and provides a code example.

NOTE: The GetModuleUsage() function does not exist in Windows NT.

There is a completely different process that would be used to accomplish the same thing from a 32-bit application. For additional information on the 32-bit implementation, please see the following article in the Microsoft Knowledge Base:

This information is included with the Help file provided with Microsoft Visual Basic version 3.0 for Windows, and on the Microsoft Developer Network (MSDN) Visual Basic Starter Kit provided with Microsoft Visual Basic version 4.0, Professional and Enterprise Editions.

Technique Also Works with MS-DOS Programs

The technique described in this article also works for MS-DOS programs. The return value from the Visual Basic Shell() function is a unique instance handle to the MS-DOS session that was started in the Shell(). If you call GetModuleUsage() with that handle after the MS-DOS session in question has ended, GetModuleUsage() will return 0 because the handle is no longer valid. This can be verified with the following code: Executing this code will show that the Shell() return value is unique for each of the shelled MS-DOS programs. Using GetModuleUsage() on one of these handles after the associated EDIT.COM program has been terminated will return zero (because the handle isn't valid anymore) and take you out of the wait loop.

Monitoring the Status of a Shelled Process

By using the Windows API GetModuleUsage() function, your Visual Basic program can monitor the status of a shelled process. The return value from the Shell() function can be used to call the GetModuleUsage() function continuously within a loop to find out if the shelled program has finished. If the Shell() function is successful, the return value is the instance handle for the shelled program. This instance handle can be passed to the GetModuleUsage() function to determine the reference count for the module. When the GetModuleUsage() function returns a value of 0 or less, the shelled program has finished.

This algorithm works correctly regardless of the WindowStyle used to shell the program. In addition, this method works correctly when:

• Shelling to Windows programs.
• Shelling to MS-DOS programs.
• Shelling to applications that do not display a window.

Below are the steps necessary to build a Visual Basic for Windows program that uses the Shell() function to execute the Windows Notepad accessory (NOTEPAD.EXE). The code shows by example how to use the Windows API GetModuleUsage() function to wait until a shelled process terminates before resuming execution.

Step-by-Step Example

1. Start Visual Basic for Windows or from the File menu, choose New Project (ALT, F, N) if Visual Basic for Windows is already running. Form1 is created by default.
2. Add the following code to the general declarations section of Form1:

         Private Declare Function GetModuleUsage% Lib "Kernel" _
            (ByVal hModule%)
   
         Private Function TestFunc(ByVal lVal As Long) As Integer
         'this function is necessary since the value returned by Shell is an
         'unsigned integer and may exceed the limits of a VB integer
            If (lVal And &H8000&) = 0 Then
              TestFunc = lVal And &HFFFF&
            Else
              TestFunc = &H8000 Or (lVal And &H7FFF&)
            End If
         End Function
   						

3. Add the following code to the Form_Click event procedure of Form1:

         Sub Form_Click()
           lRet& = Shell("NOTEPAD.EXE")       ' Modify the path as necessary.
           x% = TestFunc(lRet&)
           While GetModuleUsage(x%) > 0    ' Has Shelled program finished?
              z% = DoEvents()              ' If not, yield to Windows.
           Wend
           MsgBox "Shelled application just terminated", 64
         End Sub
   						

4. From the Run menu, choose Start (ALT, R, S) to run the program.
5. Using the mouse, click in the Form1 window. At this point, the Notepad application is shelled.

The MsgBox statement following the Shell() Function is not executed because the While loop prevents it. The message box does not appear until Notepad is closed when the user chooses Exit from Notepad's File menu (ALT, F, X).