How to Set Up a Remote Debug Session Using a Modem

ID: Q148954

The information in this article applies to:

SUMMARY

This article describes the procedures for preparing an X86 Intel- processor-based and RISC-processor-based computer running Windows NT Workstation or Server for a remote debug session.

MORE INFORMATION

- What Is the Need For the Kernel Debugger
- What Is the Process of Remote Debugging
- Configuring a System for Remote Debugging
- Debugger Options
- Editing the BOOT.INI file for Intel x86 Computers
- Changing Firmware Boot Options for RISC Computers
- Preparing the Modem

What Is the Need For the Kernel Debugger

The kernel debugger that comes with Windows NT will produce information about the system that is normally not available without the aid of the debugger. You can enable the kernel debugger from the Windows NT start-up menu. Enabling the kernel debugger will allow the Windows NT debugger to output debugging information to a remote debugger for a remote user to analyze. This is normally done at the request of a Microsoft Customer Support Professional for analyzing a fatal error in Windows NT that cannot be diagnosed from the Memory.dmp file or when a Memory.dmp file is not produced.

What Is the Process of Remote Debugging

The process of Remote debugging happens when two computers are connected together via modems over a phone line. The target computer and the host computer are running the Windows NT kernel debuggers which communicate using a special debug API and Protocol.

Configuring a System for Remote Debugging

Configuring a system for remote debugging involves changing the boot options to tell Windows NT to load the kernel debugger. On an Intel x86 based platform this is done by editing the Boot.ini file. On a RISC based system (DEC Alpha, MIPS and PPC based processors) this involves changing the boot options in the firmware menu. You will also need to have a modem and connect it to the appropriate COM port on the target computer and connect an inbound phone line to the modem.

Debugger Options

There are several boot options which can be used in configuring the system for debugging. These options are the same on Intel x86 and RISC platforms, although when used on a RISC platform the / is not required: 


/DEBUG      - When this option is used, the kernel debugger will be loaded
              during boot and kept in memory at all times. This means that
              a support professional can dial into the system being debugged
              and break into the debugger, even when the system is not
              suspended at a Kernel STOP screen.

/DEBUGPORT  - Specifies the serial port to be used by the kernel
              debugger. If no serial port is specified the debugger will
              default to COM2 on Intel x86 based computers and to
              COM1 on RISC computers.

/CRASHDEBUG - This option will cause the kernel debugger to be loaded
              during boot but swapped out to the pagefile after boot.
              As a result, a support professional will not be able to
              break into the debugger unless Windows NT is suspended
              at a Kernel STOP screen.

/BAUDRATE   - Sets the Baud rate that the kernel debugger will use.
              The default baud rate is 19200. A baud of 9600 is the
              normal rate for remote debugging over a modem.


Whenever DEBUGPORT or BAUDRATE is used, it is not necessary to use DEBUG, as Windows NT assumes that you want the computer to load in DEBUG mode. At least one of the above switches must be used to configure a computer for remote debugging, otherwise Windows NT will not load the debugger at all.

Editing the BOOT.INI file for Intel x86 Computers

To set up the target computer on an Intel x86 based computer, simply edit the Boot.ini file in the boot partition and add the appropriate debugger options. The Boot.ini file is edited using a standard ASCII text editor. The Boot.ini file is a file found on the root of the system partition (generally the C drive) and will have the Hidden, System, and Read-Only attributes set. These attributes must be changed.

To Change the Attributes of the Boot.ini File

To change the attributes of the Boot.ini file, follow Procedure I or II:

Procedure I:
  1. Type the following at a command prompt:

    attrib -s -h -r c:\boot.ini


  2. Edit BOOT.INI.

    To restore the Read-Only, Hidden, and System attributes when you finish editing the file, type the following at a command prompt:

    attrib +h +r +s c:\boot.ini \


Procedure II:
  1. In File Manager, choose By File Type in the View menu, and then select the Show Hidden/System Files box in the By File Type dialog box, and choose the OK button.


  2. In the File Manager window, select the Boot.ini file, and then choose Properties from the File menu.


  3. In the Properties dialog box, clear the Read Only, Hidden, and System attribute box and choose the OK button.


  4. Edit the Boot.ini using Notepad or another text editor and add the debug options as described below.


Configuring the Boot Options in the Boot.ini File

To configure a computer for debugging you must add one of the previous options to the Boot.ini. Generally, you will want to add /DEBUG and /BAUDRATE, to lower the baud rate to 9600 and put the computer in debug mode. You should use /DEBUGPORT=COMX if you cannot use the default COM port (COM2) for debugging. The following example shows how to add these options.

The following is an example of a typical Boot.ini file. Each entry in the [operating systems] section should correspond to the options listed in the boot menu during a normal system startup:

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 3.5"
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 3.5"
[VGA mode] /BASEVIDEO
C:\="MS-DOS"

To enable the debugger to use COM1 and to communicate at 9600 baud, add the following or see the example following these step-by-step instructions:
  1. Select the startup option that you normally use and add the /DEBUG switch at the end of the line.


  2. To specify the communications port, add the switch /DEBUGPORT=COM1.


  3. To specify the baud rate, add the switch /BAUDRATE=9600.


The following shows the above Boot.ini file after it has been modified by following steps 1-3 above (NOTE: Line 6 is the rest of line 5 that did not fit on the same line in this article):

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 3.5"
/debug /debugport=com1 /baudrate=9600
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT Version 3.5"
[VGA mode] /BASEVIDEO
C:\="MS-DOS"

Changing Firmware Boot Options for RISC Computers

Unlike the Intel x86 computers, which use a Boot.ini file, boot options on RISC computers are configured through the firmware. The following steps walk you through configuring the boot options on MIPS-, PPC- and DEC Alpha-based computers. Be sure to carry out step 10 for all DEC Alpha computers before a support engineer attempts to dial in and debug the system.
  1. Restart the computer. The ARC System screen appears, displaying the main menu from which you can select an action.


  2. On a MIPS and PPC RISC-based system, choose Run Setup to display the Setup menu and then choose Manage Startup to display a menu of the boot options.
    On a Digital Alpha AXP RISC-based system, choose Supplementary Menu, Set Up the System, and Manage Boot Selection Menu to display a menu of the boot options.


  3. Choose Change a Boot Selection to display a list of the operating systems that are installed on this computer.


  4. Choose the Windows NT operating system. If you have more than one version of Windows NT installed, select the one that you want to debug. A two-part screen appears for changing the current settings of the environment variables used to start the RISC-based computer. The environment variable that controls whether or not the RISC-based computer starts up in debug mode is the OSLOADOPTIONS variable.


  5. To edit the value for the OSLOADOPTIONS variable, use the arrow keys to select it from the list of variables. Once selected, it appears in the Name box at the top of the screen.


  6. Press ENTER to display the Value box.


  7. Type the options you want to add in the Value box, separated by spaces and press ENTER to save them. For example, if you want to turn on DEBUG mode and set the COM port to COM2, you type the following:

    OSLOADOPTIONS debug debugport=com2

    If you do not specify the debug port, the default debug port is set to COM1.

    Since RISC-based computers allow only a default baud rate of 19200, you do not need to specify the baud rate.


  8. Press ESC to stop editing.


  9. Based on the type of computer you have, do one of the following:

    - On a MIPS or PPC RISC-based system, choose Return To Main Menu and then Exit to return to the ARC System screen.

    -or-

    - On a Digital Alpha AXP RISC-based system, choose the option Supplementary Menu, save your changes and then choose Boot Menu to return to the ARC System screen.


  10. If this is the first time you debug this system you need to carry out additional configuration steps. For these steps, consult the Windows NT Resource kit Version 3.51 Update or Update 2, Appendix A.


  11. Restart the computer to run under Windows NT.


Preparing the Modem

To connect to the remote debugger successfully you must connect an external modem with auto-answer capabilities to the desired COM port. Most modems have this feature, but it is generally disabled. The modem should be set to 9600 baud. For a proper connection, the kernel debugger requires that hardware compression as well as error detection and flow control be turned off. Make sure the modem works correctly before proceeding any further. In most cases you will need to configure the modem by connecting to a spare COM port on the target machine or another machine, using TERMINAL.EXE or some other communication program to send configuration strings to the modem, then moving the modem to the target computer and COM port without turning off the modem. This is why an internal modem will not work, as restarting the system will generally reset any of the changes you have made to the modem settings.

To configure the modem:

NOTE: You can carry out all these steps on the target computer, using the COM port you want to configure as the debug port provided you carry them out before configuring the debugger on the target system.
  1. Connect a modem with auto-answer capabilities to an available COM port on the target computer or another computer.

    NOTE: Be sure you can move the modem to the target COM port without removing power from the modem.


  2. Run Terminal.exe or some other communications program and configure it to write to the modem.


  3. Set the modem to 9600 baud rate. Consult your modem documentation for the correct way to do this.


  4. Turn off all hardware compression, flow control, and error detection. The following commands are used for most US Robotics and 3Com modems. If you are using a modem from another manufacturer, consult your modem documentation for the correct strings to send to the modem.

    
AT&F    Set Back to Factory Defaults
AT&H0   Disable Transmit Data Flow Control
AT&I0   Disable Receive Data Flow Control
AT&K0   Disable Data Compression
AT&M0   Disable Error Control
ATS0=1  Auto Answer On
AT&D0   Disable Reset Modem on Loss of DTR
AT&W    Write to NVRAM


  5. Enable auto-answer on the modem. This can usually be done by sending it the string ATS0=1 (the "0" is the number 0, not the letter O). However, you should consult your modem documentation to verify this.


  6. Without removing power from the modem, move it to the correct COM port on the target system.


Additional Reading

For additional information, please see:

- The Windows NT Resource Kit.

- The Microsoft Driver Development Kit.

Additional query words: ntblue prodnt debugref live debug 


Keywords          : 
Version           : winnt:3.5,3.51,4.0
Platform          : winnt 
Issue type        :

Last Reviewed: July 20, 1999