How To Use VDMDBG Functions on Windows NT, Windows 2000, and Windows XP

This article was previously published under Q182559
This article has been archived. It is offered "as is" and will no longer be updated.
SUMMARY
Even when programming in a 32-bit environment, there are times when workingwith 16-bit applications is necessary. In Windows NT, 16-bit applicationsrun within a Virtual DOS Machine (VDM). VDMDBG.dll contains many functionsthat are useful for working with 16-bit applications. This library is partof the Microsoft Platform SDK.

VDMDBG functions provide a good way to enumerate, create, and terminate 16-bit processes (tasks) within a VDM. This article describes how to use thesefunctions on Microsoft Windows NT, Windows 2000, and Windows XP.
MORE INFORMATION
When you use any of the VDMDBG functions, you need to link VDMDBG.lib withyour project.

The VDMDBG functions mentioned below are only a subset of the availablefunctions. However, the functions that are not mentioned are only relevantto a debugger.

Enumerating VDMs

The VDMEnumProcessWOW() function provides a simple way to enumerate allVDMs running 16-bit Windows tasks. These VDMs contain the WowExec.exe task.DOS VDMs are not enumerated.

The declaration for this function is as follows:
   INT WINAPI VDMEnumProcessWOW( PROCESSENUMPROC fp, LPARAM lparam );				
The return value for this function is the number of VDMs currently running,or the number enumerated before enumeration was terminated. fp is a pointerto a callback function. The function is called for each VDM that isenumerated. lParam is a user-defined value that is passed to the callbackfunction.

PROCESSENUMPROC is declared as follows:
   typedef BOOL ( WINAPI *PROCESSENUMPROC )(      DWORD dwProcessId,      DWORD dwAttributes,      LPARAM lpUserDefined   );				
The function should return TRUE to stop enumeration or FALSE to continueenumeration. dwProcessId is the process ID of the NTVDM.exe process. Youwill need this ID when calling the other VDM functions mentioned below.

Enumerating 16-bit Windows Tasks

You can use VDMEnumTaskWOW() and VDMEnumTaskWOWEx() to enumerate taskswithin a particular VDM. The difference between the two is thatVDMEnumTaskWOWEx() provides more information to the callback function. Youshould only use VDMs returned by VDMEnumProcessWOW() with these taskenumeration functions. Using DOS VDMs does not make sense because every DOSapplication runs in its own VDM.

The declarations are:
   INT WINAPI VDMEnumTaskWOW( DWORD dwProcessId, TASKENUMPROC fp,                              LPARAM lparam );				
   INT WINAPI VDMEnumTaskWOWEx( DWORD dwProcessId, TASKENUMPROCEX fp,                                LPARAM lparam );				
The return value for each of these functions is the number of taskscurrently running within the indicated VDM, or the number enumerated beforeenumeration was terminated. dwProcessId is the process ID of the VDM. fp isa pointer to a callback function. The function is called for each task thatis enumerated. lparam is a user-defined value that is passed to thecallback function.

TASKENUMPROC and TASKENUMPROCEX are defined as follows:
   typedef BOOL ( WINAPI *TASKENUMPROC )(      DWORD dwThreadId,      WORD hMod16,      WORD hTask16,      LPARAM lpUserDefined   );				
   typedef BOOL ( WINAPI *TASKENUMPROCEX )(      DWORD dwThreadId,      WORD hMod16,      WORD hTask16,      PSZ pszModName,      PSZ pszFileName,      LPARAM lpUserDefined   );				
These functions should return TRUE to stop enumeration or FALSE to continueenumeration. You can use hTask16 in a call to terminate the task.

Example of Enumeration

   // Enumerate all 16-bit tasks on the system.   #include <windows.h>   #include <stdio.h>   #include <vdmdbg.h>   BOOL WINAPI ProcessEnumProc( DWORD, DWORD, LPARAM );   BOOL WINAPI TaskEnumProcEx( DWORD, WORD, WORD, PSZ, PSZ, LPARAM );   void main()   {      // Enumerate VDMs.      VDMEnumProcessWOW(         (PROCESSENUMPROC)ProcessEnumProc,         (LPARAM)NULL      );   }   BOOL WINAPI ProcessEnumProc( DWORD dwProcessId, DWORD dwAttrib,      LPARAM t )   {      printf("\nProcess ID: %d\n", dwProcessId);      // Use process ID of VDM to enumerate through its tasks.      VDMEnumTaskWOWEx(         dwProcessId,         (TASKENUMPROCEX)TaskEnumProcEx,         (LPARAM)NULL      );      // Keep enumerating.      return FALSE;   }   BOOL WINAPI TaskEnumProcEx( DWORD dwThreadId, WORD hMod16, WORD hTask16,      PSZ pszModName, PSZ pszFileName, LPARAM lParam )   {      //print task's information      printf("Thread ID: %d\n", dwThreadId);      printf("Module handle: %d\n", hMod16);      printf("Task handle: %d\n", hTask16);      printf("Module Name: %s\n", pszModName);      printf("File Name: %s\n", pszFileName);      // Keep enumerating.      return FALSE;   }				

Creating 16-bit Tasks

VDMStartTaskInWOW() creates a task within a VDM. The declaration is asfollows:
   BOOL WINAPI VDMStartTaskInWOW( DWORD dwProcessId, LPSTR lpCommandLine,                                  WORD wShow );				
The return value of this function is TRUE if the task is successfullystarted, otherwise it is FALSE. dwProcessId is the VDM process ID.lpCommandLine is a string indicating the filename of the 16-bit applicationalong with any command-line parameters. wShow indicates how the window willbe shown. wShow can be any value that is valid for the 16-bit ShowWindow()function.

Terminating 16-bit Tasks

To terminate a task within a VDM call VDMTerminateTaskWOW(). Thedeclaration is as follows:
   BOOL WINAPI VDMTerminateTaskWOW( DWORD dwProcessId, WORD htask );				
The return value of this function is TRUE if the task is successfullyterminated, otherwise it is FALSE. dwProcessId is the VDM process ID. hTaskis the handle to the task. This task handle can be obtained throughVDMEnumTaskWOW() or VDMEnumTaskWOWEx().

This method is a rough equivalent to TerminateProcess() in Win32. It shouldbe avoided, if possible. It does not give the task a chance to cleanlyexit, so data may be lost. Unlike Win32, the WowExec is not guaranteed toclean up after a terminated task. This can leave the VDM corrupt andunusable. To terminate the task cleanly, send a WM_CLOSE to its top-levelwindow.

Note Regarding 16-bit DOS Applications

None of the VDMDBG functions work with 16-bit DOS applications. Toenumerate DOS VDMs, you need to use another method. First, you could useVDMEnumProcessWOW() to make a list of all Win16 VDMs, and then enumerateall instances of NTVDM.exe using some other scheme (such as PSAPI). AnyNTVDM.exe from the full enumeration that was not in the Win16 list is a DOSVDM. You can create and terminate 16-bit DOS applications withCreateProcess() and TerminateProcess().
Properties

Article ID: 182559 - Last Review: 02/24/2014 08:32:37 - Revision: 3.4

  • Microsoft Win32 Application Programming Interface
  • kbnosurvey kbarchive kbapi kbhowto kbkernbase kbthread KB182559
Feedback