How To Use Windows NT WinDbg.exe with Visual Basic

Many Visual Basic application developers are moving to Windows NT as their development platform because of its robustness and excellent multitasking capabilities. This article shows by example how Visual Basic developers can use the Windbg debugger to debug a dynamic link library (.DLL file) or custom control (.VBX file) called from a Visual Basic application running under Windows NT.

CodeView for Windows doesn't run under Windows NT because CodeView uses a VxD (Virtual Device Driver) not supported by Windows NT.

For information on how to use CodeView for Windows to debug Visual Basic Custom controls (.VBX files), please see the following article in the Microsoft Knowledge Base:

WARNING: Using Windbg as described in this article is not supported by Microsoft, so you should save all source files before using this procedure. Windbg should exit without hanging the system. However, you should not modify your Visual Basic source while running under Windbg unless you always make sure you have saved any changes before running the code. This procedure has been tested on Windows NT version 3.5. Preliminary testing on Windows NT version 3.51 shows problems when switching to Visual Basic while running under Windbg, so you should not use this procedure with Windows NT version 3.51.

This article follows the steps and contains much of the material in 75612  (http://support.microsoft.com/kb/75612/EN-US/ ) , but it has been modified to describe how to use the Windbg debugger to debug a dynamic link library (.DLL file) or custom control (.VBX file) called from Visual Basic running under Windows NT.

Windbg is one of the debuggers supplied with the Win32 SDK. It is a Windows debugger but retains many of the commands of the NTSD debugger that is also supplied with the Win32 SDK. More information may be found for both Windbg and NTSD in the Win32 Tools Users Guide.

NOTE: You can build custom controls using the Control Development Kit (CDK) for Visual Basic. The Visual Basic CDK, formerly shipped separately as an add-on product from Microsoft, is now shipped as part of the Professional Edition of Microsoft Visual Basic for Windows.

Information on installing Windbg can be found in the Help files supplied with the Win32 SDK. There are several supporting .DLL files that are required to use Windbg, so it is not sufficient to copy just the WINDBG.EXE file from the SDK.

How to Invoke Windbg

Make sure your system is backed up and can be recovered if it should hang. This is highly unlikely because Windows NT has been designed to prevent this kind of problem.

You can invoke Windbg from the Windows Program Manager in any of the following ways:

• From the Windows Program Manager File menu, choose New, and specify WINDBG.EXE as a Program Item with proper arguments. You can then double-click the WINDBG icon to run WINDBG.EXE.
• From the Windows Program Manager File menu, choose Run, and enter WINDBG.EXE and its command line arguments.
• Invoke WINDBG with no arguments from a command prompt, and follow the steps below to load the various files and modules required for the debug process.

Step-by-Step Example

This example demonstrates how to invoke CIRC3.VBX, which comes with the Microsoft Visual Basic Control Development Kit (CDK) or the Visual Basic professional edition:

1. Run WINDBG.EXE from the Program Manager or command prompt:
   
   [path]WINDBG.EXE [path]VB.EXE
   
   NOTE: You can just specify an .EXE program that was written in the Visual Basic environment instead of specifying the VB.EXE environment itself. If you do this, skip steps 7, 8, and 9 below.
   
   The CIRC3.VBX is not loaded from the command line along with Windbg as it is with CodeView. It will automatically be loaded by Windbg when it is added to the Visual Basic development environment. Note that the .VBX file must be built so that it includes symbol information.
2. If you invoked WINDBG.EXE with no command line arguments, load VB.EXE or the Visual Basic application by choosing Open from the Program menu. Choose New to bring up the Open dialog box.
3. From the Open dialog box, locate VB.EXE or the application you want to run, and open it. Click OK when you return to the Open dialog box.
   
   NOTE: If you are debugging a Visual Basic application rather than loading VB.EXE, you must load the CIRC3.VBX explicitly to be able to set breakpoints. From the Options menu, choose User DLLs. Click the Browse button and locate the CIRC3.VBX. Click OK to add it to the list of modules. Click OK again to exit the dialog box.
4. From the File menu, choose Open to load the CIRC3.VBX source code. Locate the directory with the CIRC3.C file. Select and open this file. An MDI window will open in Windbg; it contains the source code for the .VBX file.
5. From the Options menu, choose Debug, and select the Separate WOW/VDM check box in the Debugger Options dialog box. Click OK.
6. Press the F5 key or choose Go from the Run menu. Visual Basic starts up. You should see various system-level .DLL files loaded. This information appears in the Command window in Windbg.
7. From the File menu in Visual Basic, choose Add File. In the Files box, select the CIRC3.VBX file. The CIRC3 tool appears in the toolbox. Again, you should see various .DLL files loaded. These are system files that support the loading of the .VBX file. You will also see the CIRC3.VBX file loaded.
8. Select the custom control from the toolbox, and add it to your form.
9. You may now set a breakpoint in the CIRC3.C source code. Locate the WM_LBUTTONDBLCLK message, and set a breakpoint on the first IF statement.
   
   If the line turns red, this indicates that the breakpoint has been set successfully. If it is purple, the breakpoint is not correctly set, so it is never triggered. This can occur if the .VBX file does not include symbol information.
10. Press the F5 key from within Visual Basic to run your program.
11. Double-click the circle. When your breakpoint is encountered, focus is set to the debugger, and execution stops at your breakpoint. You can now step through your program, set watchpoints, and so on.
12. Press the F5 key to return to the Visual Basic program and stop the debugging.

NOTE: When you exit Visual Basic, you receive an "Assertion Failed" message from within Windbg. Click OK and Windbg exits. You may get another dialog box depending on how you have your system configured for exception handling. Do not click Cancel to debug unless you have the kernel debugger on your system, otherwise your system will appear to be hung.

"Programming Windows: the Microsoft Guide to Writing Applications for Windows 3," by Charles Petzold, Microsoft Press, 1990.

"Peter Norton's Windows 3.0 Power Programming Techniques," by Peter Norton and Paul Yao, Bantam Computer Books, 1990.

"Microsoft Windows Software Development Kit: Programming Tools," version 3.0.

WINSDK.HLP file shipped with Microsoft Windows 3.0 Software Development Kit.

Tools User Guide shipped with the Win32 SDK.