How to Set Up a Remote Debug Session Using a Null Modem Cable

ID: Q151981

The information in this article applies to:

SUMMARY

This article is intended to describe the procedures for setting up the debugger on a Windows NT system using a Null modem cable attached to another Windows NT system.


MORE INFORMATION

The process of setting up the debugger with a Null modem cable is broken down into the following topics:


What is the need for the kernel debugger?

The kernel debugger that comes with Microsoft Windows NT provides information about the system that is normally not available without the aid of the debugger. Windows NT has the ability to enable the kernel debugger from the Windows NT start-up menu (Boot.ini). Enabling the kernel debugger allows the Windows NT debugger to output debugging information to a remote debugger for a remote user to analyze. This is normally done at the request of a Microsoft Support Engineer for analyzing a fatal error in Windows NT that cannot be diagnosed from the Memory.dmp file or when a Memory.dmp file is not produced.

What is the process of remote debugging?

For our discussion the target machine will be the system that is being debugged and the host machine will be the system doing the debugging. The process of remote debugging happens when two computers are connected by the serial communication ports. The target computer and the host computer are running the Windows NT kernel debuggers, which communicate using a special debug API and Protocol.

Why use a Null modem cable?

Connecting the Null modem cable to a serial port allows the debugger to run at its current top speed of 19200, compared to the speed of 9600 over a modem. It is generally considered more reliable than using a modem for the following reasons: it does not involve configuring a modem, there is no possibility of a modem or line failure, a system can be debugged locally, and finally it provides faster debugging.

The most common method is to set up a target computer for debugging and the host computer for doing the debugging using a Remote.exe session. This configuration allows a technical support person to connect to the Remote.exe session by means of a RAS connection and debug the target computer without physically being present at the target computer.

Changing the attributes on the Boot.ini file

To set up the target computer, simply edit the Boot.ini file in the boot partition and add the appropriate debugger options. The Boot.ini is edited using a standard ASCII text editor. The Boot.ini is hidden in the boot partition directory. The Boot.ini attributes are hidden, read-only and system. These attributes must be changed to edit the Boot.ini file.

To change the attributes of the Boot.ini file:

At the command prompt, type attrib -h -r -s c:\boot.ini and press Enter, and then edit the file. To restore the read-only, hidden, and system attributes when you finish editing the file, type attrib +h +r +s c:\boot.ini at the command prompt.

- Or -
  1. In File Manager, choose By File Type on the View menu. Select the Show Hidden/System Files box in the By File Type dialog box, and click OK.


  2. In the File Manager window, select the Boot.ini file, and then choose Properties on the File menu.


  3. In the Properties dialog box, clear the Read Only, Hidden, and System Attribute box and click OK.


  4. Edit the Boot.ini using Notepad or another text editor.


Debugger Options

Once the Boot.ini on the target computer is opened for editing, several options can be added to the Boot.ini file to configure the debugger environment on the target computer. The various options available are listed and described below. 


/DEBUG      - This option is used to enable the kernel debugger

/DEBUGPORT  - Specifies the serial port to be used by the kernel
              debugger. If no serial port is specified the debugger will
              default to COM2.

/CRASHDEBUG - Sends debug information only when a fatal system 
              error (FSE) occurs.

/BAUDRATE   - Sets the Baud rate that the kernel debugger will use. The
              default baud rate is 19200. A baud of 9600 is the normal
              rate for remote debugging over a modem.

Configuring the debugger for the system

There are several options available for configuring the debugger on the target computer. To add any one of the previous options to the Boot.ini, use the following example.

The following is a typical Boot.ini. Each entry in the [operating systems] section should correspond to the options listed in the boot menu during a normal system startup. 


[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT" Version 3.5"
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT" Version 3.5 [VGA
mode]" /BASEVIDEO
C:\="MS-DOS"


To enable the debugger, use COM1 to do the following.
  1. Select the startup option that you normally use and add either the /debug switch at the end of the line.


  2. To specify the communications port, add the switch /debugport=com1 



[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT" Version 3.5"
/debug /debugport=com1
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT" Version 3.5 [VGA
mode]" /BASEVIDEO
C:\="MS-DOS"

Setting up the Null modem cable

A Null modem cable is a serial cable that has been configured to allow data to be sent between two serial ports. Normally you connect a serial port to a serial device. In this case we need to configure the target computer to accept a serial-port-to-serial-port connection. It is important that a Null modem cable be used because if a normal serial cable is used the target system expects a serial device, not another serial port, and the system cannot be debugged.

When you use a NULL modem to make a direct serial connection between two computers, your cable must be wired as shown in the following tables. 


9-Pin NULL Modem Cabling

Remote host
serial port connector Host system
serial port connector

Signal
3  2  Transmit Data
2  3  Receive Data
7  8  Request to Send
8  7  Clear to Send
6, 1  4  Data Set Ready and Carrier Detect
5  5  Signal Ground
4  6, 1  Data Terminal Ready
25-Pin NULL Modem Cabling

Configuring the Host computer for a Remote.exe session

The concept here is that the host computer is being set up to do a debug of the target computer as if there were a technical support person sitting at the host computer. The Remote.exe will be run to allow this debug session to be accessed remotely through a RAS connection.

The Remote.exe is a command line utility which allows you to run command-line programs on remote computers. Remote.exe uses two parts, the server component and the client component. To use Remote.exe, you must first start the server end on the computer where you are debugging from. This allows other users to connect to your debugging session using the client portion of Remote. The Remote.exe comes with the Resource Kit. For more details and the command syntax of this utility, refer to the Resource Kit online help.

The Windows NT installation compact disc and the Windows NT Resource Kit compact disc should be available to accomplish this setup.
  1. The first thing should be done is to set up a RAS account on the network for the technical support person. This information, including the name of the host system, should be documented and ready to give to the support person.


  2. The host computer should be configured as described in the following Microsoft Knowledge Base article Q148659. This article describes in detail how to set up the symbols on the host computer for debugging the target computer.

    Q148659How to Set Up Windows NT Debug Symbols



  3. The recommended way to run I386KD is to copy all of the files located in the \Support\Debug\I386\ directory (assuming that you are debugging an I386 computer) to a directory on the hard drive, and change to that directory.


  4. Copy the Remote.exe from the Resource Kit to the same directory.


  5. The I386KD relies on environment variables for information necessary to run successfully. I386KD uses many environment variables, the minimal needed in our case is the _NT_SYMBOL_PATH and the _NT_DEBUG_PORT variable. The set command is used to set the environment variables. The _NT_SYMBOL_PATH variable points to the path of the symbols file that the debugger will use for the debug session. The _NT_DEBUG_PORT point to the active serial port being used on the host computer for debugging. If this is anything other than serial port 2 then the _NT_DEBUG_PORT environment variable must be used.

    Example:
    The host computer Null modem cable will be connected to serial port 1 and has copied the symbols to the c:\debug\symbols directory. At a command prompt type the following.

    SET _NT_DEBUG_PORT=COM1
    SET _NT_SYMBOL_PATH=C:\DEBUG\SYMBOLS
    SET _NT_DEBUG_LOG_FILE_APPEND=c:\temp\DEBUG.LOG



  6. The only thing left to do at this point is to start the debug session and remote it out. At a command prompt type the following:

    remote /s "i386kd -v" debug



Additional Information Resources

Additional query words: 3.50 3.51 prodnt debugref 


Keywords          : NTSrvWkst 
Version           : winnt:3.50,3.51,4.0
Platform          : winnt 
Issue type        :

Last Reviewed: July 20, 1999