IDA v5.4 supports Windbg as a debugger backend

A new release of IDA, the Interactive Disassembler, has been recently released featuring new debugger integration capabilities. IDA’s existing built-in debugger often proved lackluster, but IDA’s static analysis and navigation features are, of course, unrivaled by anything else. I always wished IDA would address the weakness of its debugging features and now they have done so in the v5.4 release. The new version can drive a gdb debugging server (as embedded platforms often provide), a Bochs virtual machine (great for BIOS and boot loader debugging) and most importantly DbgEng, the Microsoft debugging engine used by Windbg. Since Windbg sessions often involve heavy use of PDBs, IDA v5.4 has improved its support for importing data from PDBs and now uses more of their embedded type information (previously the third party Determina PDB plugin attempted to improve IDA’s PDB support). To top things off, the Python plugin is now bundled with IDA, as well.

I haven’t had the chance to use the new version yet, but Hex Rays have a great demo video posted here. The only thing notable that appears missing is a nice UI for examining the stack trace, but if push comes to shove the Windbg command line can be used to invoke “k”, as demonstrated.

Advertisements

Installing filter drivers with DIFxApp and a WiX v3 MSI

The designers of Windows Installer probably did not have driver installation in mind. Indeed, the ServiceInstall MSI database table does not even allow for service types SERVICE_KERNEL_DRIVER and SERVICE_FILE_SYSTEM_DRIVER in the ServiceType column. Originally, Microsoft encouraged driver installation by means of a “setup application”, custom code invoking the SetupAPI. The Setup API functions would be used to examine an .INF file describing required steps for the driver’s installation. Back in Windows NT 4.0, even .INF files were often avoided in lieu of directly setting up the legacy driver service in the system registry with C code.

Enter Windows 2000 with Plug and Play. Now the system uses .INF files it maintains in a store to locate and install drivers for devices on demand. When a new device is inserted, the system looks for an .INF file with a matching hardware identifier. For most hardware devices, .INF based installation becomes a de-facto requirement. However, for other driver categories, such as file system filters, device class filters, etc., an .INF file is but one way of performing the driver installation process.

Microsoft, when confronted by driver setup authors who denounce .INF files as having cryptic, old-fashioned syntax, non-ideal diagnosibility and debuggability and other shortcomings, persists in encouraging their use. They are a WHQL logo requirement and in some cases it is claimed they will be the only way to install drivers in future Windows releases.

Often driver installation is but a part of the installation process of a larger application program. For applications, Microsoft encourages (and mandates as a logo requirement) the use of MSI packages and the Windows Installer service. In order to avoid having every MSI setup author who needs to perform driver installation roll his own custom invocation of the driver’s .INF installation and in a bid to improve driver installation’s synergy with Windows Installer in general, Microsoft created DIFxApp – Driver Install Frameworks for Applications.

MSI and DIFxApp’s .INF based installation make for an odd couple. The Windows Installer model is transactional – describe what changes should be made to the system and let Windows Installer determine how to execute them and importantly, how to roll them back. After having encouraged COM DLLs to encapsulate the details of their installation and registration with the Self-Registration DllRegisterServer export for years, describing the individual elements of the COM registration using an MSI’s Registry table is now encouraged instead. DllUnregisterServer can fail in mysterious ways due to missing dependencies, etc., leaving cruft behind during uninstallation. By contrast, rolling back insertions made from the Registry table is always possible for the MSI service. If COM’s long standing installation architecture was set aside to better align with MSI’s model, one would expect a similar occurance for drivers. However, DIFxApp, the solution for installing drivers with MSI, is the moral equivalent of COM Self-Registration. If a driver’s .INF does not remove all changes it made during installation, the uninstallation process is bound to be imperfect. MSI’s ability to diagnose whether a driver’s installation has been damaged is limited and thwarted.

Be that as it may, .INF files’ tight integration with the Plug and Play system, the fact they are a logo requirement and uniformity considerations are compelling arguments for their use. From this point forward, it shall be assumed that .INF based installation and MSI integration have been chosen.

If you’ve developed a common PnP WDM driver, you probably already have an .INF for installation purposes. That .INF can usually be used as is with DIFxApp, which assumes PnP drivers by default. It is for other driver categories that the process becomes more tricky. I’ll discuss filter drivers of three varieties in particular – file system filters, file system minifilters and device class filters.

For illustration purposes, let’s consider a typical file system minifilter driver, DirFilter, which is presently a pet project of mine that will eventually perform interesting modifications to directory listing I/O operations. The following is DirFilter’s .INF file:


[Version]
Signature = "$Windows NT$"
Class = "ActivityMonitor"
ClassGuid = {b86dff51-a31e-4bac-b3cf-e8cfe75c9fc2}
Provider = %Koby%
DriverVer = 09/12/2008,1.0.0.0
CatalogFile = dirfilter.cat
DriverPackageType = FileSystemMinifilter
[DestinationDirs]
DefaultDestDir = %DIRID_DRIVERS%
DirFilter.DriverFiles = %DIRID_DRIVERS%
[DefaultInstall]
OptionDesc = %ServiceDescription%
CopyFiles = DirFilter.DriverFiles
[DefaultInstall.Services]
AddService = %ServiceName%,,DirFilter.Service
[DefaultUninstall]
DelFiles = DirFilter.DriverFiles
[DefaultUninstall.Services]
DelService = %ServiceName%,0x200 ;Ensure service is stopped before deleting
[DirFilter.Service]
DisplayName = %ServiceName%
Description = %ServiceDescription%
ServiceBinary = %12%\%DriverName%.sys ; DIRID_DRIVERS
Dependencies = "FltMgr"
ServiceType = %SERVICE_FILE_SYSTEM_DRIVER%
StartType = %SERVICE_DEMAND_START%
ErrorControl = %SERVICE_ERROR_NORMAL%
LoadOrderGroup = "FSFilter Activity Monitor"
AddReg = DirFilter.AddRegistry
[DirFilter.AddRegistry]
HKR,"Instances","DefaultInstance",0x00000000,%Instance1.Name%
HKR,"Instances\"%Instance1.Name%,"Altitude",0x00000000,%Instance1.Altitude%
HKR,"Instances\"%Instance1.Name%,"Flags",0x00010001,%Instance1.Flags%
[DirFilter.DriverFiles]
%DriverName%.sys
[SourceDisksFiles]
dirfilter.sys = 1
[SourceDisksNames]
1 = %DiskId1%
[Strings]
Koby = "Koby Kahane"
DIRID_DRIVERS = 12
SERVICE_FILE_SYSTEM_DRIVER = 2
SERVICE_DEMAND_START = 2
SERVICE_ERROR_NORMAL = 1
ServiceDescription = "DirFilter minifilter driver"
ServiceName = "DirFilter"
DriverName = "DirFilter"
DiskId1 = "DirFilter minifilter installation media"
; Instance specific information.
Instance1.Name = "DirFilter Instance"
Instance1.Altitude = "370021" ; Real world minifilters should use Microsoft allocated altitude
Instance1.Flags = 0x0 ; Allow automatic attachments

This is, I believe, a pretty typical .INF for a minifilter. A default instance of it is set up at an altitude in an appropriate filter load order group. This is as good a place as any to note the importance of requesting a Minifilter Altitude Allocation from Microsoft for public drivers rather than using some random number which may end up causing conflicts and interoperability issues. From my experience, it takes Microsoft two months or so to respond to an allocation request so make sure to take care of it early in your release cycle.

As a first step towards a DIFxApp-based MSI driver installation, write an .INF file that will work when right clicking it and selecting Install from Explorer’s popup menu. This calls SetupAPI’s InstallHinfSection on the .INF file’s DefaultInstall section. Further, ensure that the uninstallation process works as expected, and leaves no trails behind, by using InstallHinfSection to invoke the DefaultUninstall section of the file.

You may discover that you are encountering difficulty at this stage. I recommend running the ChkINF utility, which analyzes .INF files for errors and mishaps, before testing the .INF file and indeed after every modification to an existing file. ChkINF, while sometimes terse, is nowhere near as cryptic as errors you will receive from SetupAPI. However, if you encounter setup errors even after ChkINF has verified your file, you can utilize verbose SetupAPI logging to further diagnose setup issues.

With a working “right-click Install” .INF file, the time is ripe for integration into an MSI installer. First of all, DIFxApp requires an additional directive in the Version section of your .INF file, the DriverPackageType. The driver package type is used by DIFx to determine what steps are required for the installation of your specific type of driver. For example, if you fail to specify the type of your file system filter, DIFx assumes it is a PnP driver and the “installation” will just import the .INF into the system driver store. No service will be created and success will not be achieved. Unfortunately, the documentation for the DriverPackageType is sorely lacking at best. You will see it only mentions ClassFilter and PlugAndPlay as possible types in MSDN. MSDN proves a disappointment. However, another Microsoft information source, their Requirements for Driver Packages that are Used with the DIFx Tools document, calls for the use of DIFx for Class Filters, Plug and Play drivers, File System & File System Filter drivers, Network drivers, Kernel Service Drivers and Export Drivers. The DriverPackageType values ClassFilter, FileSystem, FileSystemFilter, KernelModule, KernelService, Network and PlugAndPlay are documented there. You may notice that a special type for minifilters is conspicuously absent from the list. At first, I figured perhaps DIFx has no need for a distinction between legacy filters and minifilters. Later, an old post (free registration required) on OSR’s NTFSD mailing list turned my attention to the fact DIFx will require a system restart for a minifilter, even though it can be loaded and unloaded without one. Neal Christiansen, Microsoft’s File System Filter Group Lead, replies (back in 2004) that appropriate support for minifilter installation is forthcoming. I therefore set out to discover whether DIFxApp as available today provides special support for minifilters.

When the Windows Server 2008 WDK is installed to the default location, the x86 versions of the DIFxApp MSI custom action DLLs reside in C:\WINDDK\6001.18000\redist\DIFx\DIFxApp\English\WixLib\x86. I examined them with the Interactive Disassembler and was happy to see Microsoft made available public symbols (.PDBs) for them. To make a long story short, the DriverPackageType field is examined by DIFxApp in the SetupAPI::CDriverPackage::GetDriverType(CRefPtr<SetupAPI::CInf>, DRVPKG_TYPE&) function in DIFxAppA.dll, which references a global table, dubbed DriverTypeStringList, for possible package types:

It is now evident that available driver package types as of DIFx 2.1 are ClassCoInstaller, ClassFilter, DeviceFilter, FileSystem, FileSystemFilter, FileSystemMinifilter, KernelModule, KernelService, NdisImMiniport, Network and PlugAndPlay. With this information, I opted for the FileSystemMinifilter type for DirFilter’s installation .INF file.

Next, a catalog file is required by DIFx. Even though the purpose of the catalog is to be WHQL signed by Microsoft, DIFx can be used to install drivers without WHQL certification. Nevertheless, the catalog still needs to be there. The easy way is to use the Inf2Cat tool for the catalog’s generation. Place the .INF file and the driver files it references in a staging area and you can run it like so:

E:\Projects\dirfilter\staging>inf2cat /driver:E:\Projects\dirfilter\staging /os:Server2008_X86,Vista_X86,Server2003_X86,XP_X86,2000 /verbose

Specifying, of course, the operating systems and platforms appropriate for your driver.

An .INF file, a .CAT file and a .SYS file are the three files required for a DIFxApp driver package. The DIFxApp model requires your MSI to install these files to a source directory, per driver package (i.e., if you have multiple drivers, they’ll need multiple .INFs in multiple directories) and performs the installation from there to the Windows driver store and eventually the drivers directory under system32.

Microsoft distributes a WiX library and documentation for using it with DIFxApp. However, that library targets the older WiX v2.0 release. WiX v3.0 is the actively developed version, and has different schema, semantics and a slightly different integration model with DIFxApp. If you use WiX v2.0, consult Microsoft’s documentation. Otherwise, let’s proceed. The following .wxs file is the sample installer I wrote for installing DirFilter with DIFxApp:


<?xml version='1.0' encoding='windows-1252'?>
<Wix xmlns='http://schemas.microsoft.com/wix/2006/wi'
     xmlns:difx='http://schemas.microsoft.com/wix/DifxAppExtension'>
    <Product Name='DirFilterInstaller' Id='59486bf1-77ad-460f-802d-29dbab0f78be' Language='1033' Codepage='1252' Version='1.0' Manufacturer='KK' UpgradeCode='96fe2ba0-fe8b-45b4-85ef-ea9aca103e6f'>
        <Package Id='*' Keywords='DirFilter' Description='DirFilter Installer' Comments='Installs DirFilter' Manufacturer='KK' InstallerVersion='100' Languages='1033' Compressed='yes' SummaryCodepage='1252' />
        <Media Id='1' Cabinet='DirFilterInstaller.cab' EmbedCab='yes' DiskPrompt='DirFilter Media' />
        <Property Id='DiskPrompt' Value='DirFilter Install Media' />
        <Directory Id='TARGETDIR' Name='SourceDir'>
            <Directory Id='ProgramFilesFolder'>
                <Directory Id='INSTALLDIR' Name='DirFilterApp'>
                    <Directory Id='DirFilterAppDrivers' Name='Drivers'>
                        <Directory Id='DirFilterDir' Name='DirFilter'>
                            <Component Id='DirFilterDriver' Guid='8c64e674-5476-46e4-93cd-ba1ae78622df'>
                                <File Id='DirFilterSYS' Name='DirFilter.sys' DiskId='1' Source='DirFilter.sys' KeyPath='yes' />
                                <File Id='DirFilterINF' Name='DirFilter.inf' DiskId='1' Source='DirFilter.inf' />
                                <File Id='DirFilterCAT' Name='DirFilter.cat' DiskId='1' Source='DirFilter.cat' />
                                <difx:Driver Legacy='yes' />
                            </Component>
                        </Directory>
                    </Directory>
                </Directory>
            </Directory>
        </Directory>
        <Feature Id='Complete' Level='1'>
            <ComponentRef Id='DirFilterDriver' />
        </Feature>
        <UIRef Id="WixUI_Minimal" />
    </Product>
</Wix>

Several points are worth mentioning regarding this installer:

  • The XML schema for the WiX v3 DIFxApp extension is referenced at the top of the file. This is required for the Driver element to work. Note that in WiX v3, the Driver element is used instead of driver-specific attributes that were available for the Component element in the WiX v2 DIFxApp extension.
  • As said before, DIFx requires one driver per directory and one driver per .INF file. The driver component is placed in a directory under the application’s directory. Additional drivers will be placed in additional components under additional directories.
  • The “Legacy” attribute is used in the DIFx Driver element because my driver package is not WHQL signed. Of course the best solution for this is to ensure your driver is up to par and actually get WHQL certification. You should also note that Legacy mode disables integrity checks, allowing the driver installation to appear successful even when critical files are missing, etc.
  • Do NOT copy the GUIDs I used if you end copying and pasting from this example. Make sure you generate your own with uuidgen and replace them.

The .wxs source for this WiX installer is straightforward enough. However, actually compiling it and linking it requires referencing several external dependencies. When running Candle, the WiX compiler, the WixDifxAppExtension should be referenced (and others you may depend on, such as WixUIExtension). When running Light, the WiX linker, you should reference the extension as before. Additionally, in the newer builds of WiX v3.0, multiplatform support for DIFx requires you reference the .wixlib with the DIFxApp custom action DLLs for the appropriate platform:

E:\Projects\dirfilter\staging>%WIX_PATH%\candle -ext %WIX_PATH%\WixUIExtension.dll -ext %WIX_PATH%\WixDifxAppExtension.dll DirFilterInstaller.wxs
Microsoft (R) Windows Installer Xml Compiler version 3.0.4617.0
Copyright (C) Microsoft Corporation. All rights reserved.

DirFilterInstaller.wxs

E:\Projects\dirfilter\staging>%WIX_PATH%\light -ext %WIX_PATH%\WixUIExtension.dll -
ext %WIX_PATH%\WixDifxAppExtension.dll DirFilterInstaller.wixobj %WIX_PATH%\difx
app_x86.wixlib
-o DirFilterInstaller.msi
Microsoft (R) Windows Installer Xml Linker version 3.0.4617.0
Copyright (C) Microsoft Corporation. All rights reserved.

(The WIX_PATH environment variable is set to WiX 3.0’s bin directory on my system.)

Failing to reference WixDifxAppExtension or difxapp_[platform].wixlib will result in assorted compilation or linkage errors during installer build.

When debugging DIFxApp driver installation, usually the interesting things are performed by the SetupAPI directly and therefore the SetupAPI log should be examined first. However, DIFxApp’s custom actions write diagnostic information into the MSI log (which can be enabled with a command-line switch to msiexec) and can provide additional insight into the driver installation process. Here’s an excerpt from the MSI log of a successful DirFilter installation:

MSI (s) (2C:44) [20:22:39:061]: Executing op: ActionStart(Name=MsiInstallDrivers,,)
Action 20:22:39: MsiInstallDrivers.
MSI (s) (2C:44) [20:22:39:077]: Executing op: CustomActionSchedule(Action=MsiInstallDrivers,ActionType=3073,Source=BinaryData,Target=InstallDriverPackages,CustomActionData=2.15{8C64E674-5476-46E4-93CD-BA1AE78622DF}C:\Program Files\DirFilterApp\Drivers\DirFilter\82DirFilterInstallerKK)
MSI (s) (2C:90) [20:22:39:092]: Invoking remote custom action. DLL: C:\WINDOWS\Installer\MSI29.tmp, Entrypoint: InstallDriverPackages
DIFXAPP: ENTER: InstallDriverPackages()
DIFXAPP: INFO: 'CustomActionData' property 'DIFxApp Version' is '2.1'.
DIFXAPP: INFO: 'CustomActionData' property 'UI Level' is '5'.
DIFXAPP: INFO: 'CustomActionData' property 'componentId' is '{8C64E674-5476-46E4-93CD-BA1AE78622DF}'.
DIFXAPP: INFO: 'CustomActionData' property 'componentPath' is 'C:\Program Files\DirFilterApp\Drivers\DirFilter\'.
DIFXAPP: INFO: 'CustomActionData' property 'flags' is 0x8.
DIFXAPP: INFO: 'CustomActionData' property 'installState' is '2'.
DIFXAPP: INFO: 'CustomActionData' property 'ProductName' is 'DirFilterInstaller'.
DIFXAPP: INFO: 'CustomActionData' property 'ManufacturerName' is 'KK'.
DIFXAPP: INFO: user SID of user performing the install is 'S-1-5-21-1417001333-651377827-725345543-1003'.
DIFXAPP: INFO: opening HKEY_USERS\S-1-5-21-1417001333-651377827-725345543-1003\Software\Microsoft\Windows\CurrentVersion\DIFxApp\Components\{8C64E674-5476-46E4-93CD-BA1AE78622DF} (User's SID: 'S-1-5-21-1417001333-651377827-725345543-1003') ...
DIFXAPP: INFO: ENTER: DriverPackageInstallW
DIFXAPP: INFO: Copied 'DirFilter.inf' to driver store...
DIFXAPP: INFO: Copied 'dirfilter.cat' to driver store...
DIFXAPP: INFO: Commiting queue...
DIFXAPP: INFO: Copied file: 'C:\Program Files\DirFilterApp\Drivers\DirFilter\dirfilter.sys' -> 'C:\WINDOWS\system32\DRVSTORE\DirFilter_11E71DD96C3AADF7CDEC882620E8410DECE9B5EF\dirfilter.sys'.
DIFXAPP: INFO: Installing INF file "C:\WINDOWS\system32\DRVSTORE\DirFilter_11E71DD96C3AADF7CDEC882620E8410DECE9B5EF\DirFilter.inf" of Type 4.
DIFXAPP: INFO: Installing File System Driver 'C:\WINDOWS\system32\DRVSTORE\DirFilter_11E71DD96C3AADF7CDEC882620E8410DECE9B5EF\DirFilter.inf'
DIFXAPP: INFO: Service 'DirFilter' was started
DIFXAPP: SUCCESS:Installation completed with code 0x0.
DIFXAPP: INFO: RETURN: DriverPackageInstallW (0x0)

DIFXAPP: INFO: ENTER: DriverPackageGetPathW
DIFXAPP: SUCCESS:Found driver store entry.
DIFXAPP: INFO: RETURN: DriverPackageGetPathW (0x7A)
DIFXAPP: INFO: ENTER: DriverPackageGetPathW
DIFXAPP: SUCCESS:Found driver store entry.
DIFXAPP: INFO: RETURN: DriverPackageGetPathW (0x0)
DIFXAPP: INFO: driver store entry for 'C:\Program Files\DirFilterApp\Drivers\DirFilter\DirFilter.inf' is 'C:\WINDOWS\system32\DRVSTORE\DirFilter_11E71DD96C3AADF7CDEC882620E8410DECE9B5EF\DirFilter.inf'.
DIFXAPP: INFO: The component Id '{8C64E674-5476-46E4-93CD-BA1AE78622DF}' is now set to point to driver store: 'C:\WINDOWS\system32\DRVSTORE\DirFilter_11E71DD96C3AADF7CDEC882620E8410DECE9B5EF\DirFilter.inf'
DIFXAPP: INFO: A reboot is not needed to install the component '{8C64E674-5476-46E4-93CD-BA1AE78622DF}'.
DIFXAPP: RETURN: InstallDriverPackages() 0 (0x0)

Notice how DIFxApp places a copy of your driver package in the system driver store, a behavior which is distinct from “right-click Install” .INF invocation. Notice also it reports whether the installation operation with SetupAPI was successful and will additionally take care of scheduling a reboot if the driver package so requires.

DIFxApp works equally well for installing a legacy file system filter. Simply use the FileSystemFilter DriverPackageType value instead.

Device class filters require special consideration. MSDN has a page about Installing a Filter Driver. For device filters, the process is straightforward – your .INF simply adds an UpperFilter under the appropriate device’s hardware key, using the special “HKR” relative root. However, a device class filter has no hardware key to refer to and indeed may wish to filter several distinct device classes. The MSDN page says one ought to write a custom “setup application” that shall invoke the .INF file for the several device classes of interest, as desired. However, DIFxApp offers no such flexibility and it does not appear that it can be used to drive such an .INF file. However, the fact it has a ClassFilter type clearly implies it has device class filters in mind.

As is often the case, the answer to this dillemma lies in the WDK. The WDK features a sample WiX v2.0 project for illustrating DIFxApp’s use, which in fact installs a class filter. In a default installation, you can find it in C:\WINDDK\6001.18000\src\setup\DIFxApp\WiXLib\ClassFilter.wxs. Nearby, in C:\WINDDK\6001.18000\src\setup\infs\clasfilt\ClasFilt.Inf, resides the sample .INF file. Among other things, the file includes an explicit registration as a class UpperFilter:

[ClassFilter_AddReg]
;
; Change {setup-ClassGUID} to the string form of the ClassGUID that you are installing the filter on.
;
; Change UpperFilters to LowerFilters if this is a lower class filter.
;
HKLM, System\CurrentControlSet\Control\Class\{setup-ClassGUID}, UpperFilters, 0x00010008, clasfilt

Fair enough. Multiple entries can be included in the registry section for UpperFilter registration directly under the class keys of interest. However, if you opt for this approach you may find to your dismay you no longer fall in ChkINF’s favor:

(W22.1.2213) INF files should not set registry entries under 'HKLM,System\CurrentControlSet\Control\Class'.

I guess you’re damned if you do and damned if you don’t. I suggest not drinking ChkINF’s Kool-Aid, setting up the device class filter registration directly and using DIFxApp rather than iterating with a custom setup application, obviously.

Hopefully this lengthy writeup will prove useful to other authors of MSI-based driver installations.

Replacing boot load drivers with the Windows Boot Debugger

Recently, I’ve been assigned to work on fixing several bugs in a Windows file system filter driver. Debugging native code has always been characterized by the tedious and cumbersome modify, compile and link, copy, run, repeat… cycle, but in the case of kernel-mode development, the overhead of that cycle is even more acute.

I’ve found that booting the target system or virtual machine every time you want to replace a driver file with an updated build and then rebooting to have the new driver loaded significantly prolongs the cycle. Therefore, I was happy to discover Windbg’s .kdfiles command.

The .kdfiles command configure’s the kernel debugger’s driver replacement map. Whenever the NT Memory Manager attempts to load a driver image, it consults the kernel debugger, if attached, asking it for an alternative driver image. If the debugger has one, it is transmitted over the kernel debugging connection from the host to the target, and used in lieu of the target’s local driver image.

Using the driver replacement map makes it easier to replace a driver with an updated version. However, in its usual form, the replacement map feature has a significant limitation – it cannot replace boot load drivers.

To understand the logic behind this restriction, one must consider the nature of boot driver loading. While demand-start drivers are started by the user-mode Service Control Manager (SCM) and system-start drivers are loaded by NTOSKRNL’s IoInitSystem function, boot drivers are, as their name suggests, required for the system to boot and are therefore loaded by osloader, a part of ntldr (this description is for pre-Vista systems).

By the time the NT kernel is up and its Memory Manager consults the kernel debugger and its driver replacement map, it is far too late to do anything about those drivers which have been pre-loaded by the OS loader. The initial breakpoint offered by the kernel debugger is simply too late.

Fortunately, Microsoft recognized the importance of providing a driver replacement map for boot load drivers and provides a somewhat esoteric solution in the form of the debug version of NTLDR.

The debug version of NTLDR expects the kernel debugger to attach to it during system startup. Unlike the kernel debugger, it is not configured with the boot.ini file and is always configured to a 115,200 baud connection on the COM1 serial port.

The documentation for .kdfiles points out that the Windows Driver Kit (WDK) bundles a debug version of NTLDR in the debug subdirectory. However, such a file is nowhere to be found there, probably because the WDK now contains the Vista checked kernel in its debug directory and the modern Vista boot loader is distinct from NTLDR. More on Windows Vista later, but for now let’s concentrate on Windows XP.

Failing to locate the debug NTLDR in the WDK, I turned back in time to the Windows Server 2003 SP1 IFS Kit, a variant of the Windows Server 2003 SP1 DDK for file system and file system filter developers. I was glad to find the ntldr_dbg file in its debug subdirectory.

However, my happiness quickly turned to disappointment when I replaced the original NTLDR with ntldr_dbg in a Windows XP virtual machine. The system refused to boot, claiming that NTLDR was corrupt. Since the debug directory in the IFS kit contains checked kernel binaries for Windows Server 2003 SP1, I figured that the provided version of ntldr_dbg is a match for that version, as well.

I turned to the archives, so to speak, and dusted off old MSDN Subscription CDs. I eventually turned up the rather antiquated Windows XP SP1 DDK. In there, I found another version of ntldr_dbg. I placed it as required and this time the system booted successfully.

It is unfortunate that one has to dig up the DDK of yore to locate the boot debugger. It really ought to be more accessible.

With the debug version of NTLDR is in place, when you boot the system, right before the OS loader menu appears, you see the following message:
Boot Debugger Using: COM1 (Baud Rate 115200)

Once the message is displayed, NTLDR blocks waiting for a kernel debugger to connect. I start the kernel debugger the way I’d usually start it:
windbg -b -k com:pipe,port=\\.\pipe\com_1

Soon enough, however, it is evident that this is no ordinary kernel debugging session:

Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
Copyright (c) Microsoft Corporation. All rights reserved.


Opened \\.\pipe\com_1
Waiting to reconnect...
BD: Boot Debugger Initialized
Connected to Windows Boot Debugger 2600 x86 compatible target, ptr64 FALSE
Kernel Debugger connection established. (Initial Breakpoint requested)
Symbol search path is: C:\WINDOWS\Symbols;SRV*E:\SymStore*http://referencesource.microsoft.com/symbols;SRV*E:\SymStore*http://msdl.microsoft.com/download/symbols
Executable search path is:
Module List address is NULL - debugger not initialized properly.
WARNING: .reload failed, module list may be incomplete
KdDebuggerData.KernBase < SystemRangeStart
Windows Boot Debugger Kernel Version 2600 UP Checked x86 compatible
Primary image base = 0x00000000 Loaded module list = 0x00000000
System Uptime: not available
Break instruction exception - code 80000003 (first chance)
0041cf70 cc int 3
kd>

Windbg has attached to the Windows Boot Debugger, a debugging environment provided by the debug version of NTLDR at a very early stage of system startup, well before the NT kernel has been loaded. Indeed, the initial breakpoint at the boot debugger occurs before an OS to start has been selected at the loader boot menu.

With the boot debugger at its initial breakpoint, we can set up the driver replacement map as desired. For instance, we can replace NTFS and NDIS with their counterparts from the checked build of Windows XP:

kd> .kdfiles -m \WINDOWS\system32\drivers\Ntfs.sys C:\Stuff\xpsp3checked\Ntfs.sys
Added mapping for '\WINDOWS\system32\drivers\Ntfs.sys'
kd> .kdfiles -m \WINDOWS\system32\drivers\Ndis.sys C:\Stuff\xpsp3checked\Ndis.sys
Added mapping for '\WINDOWS\system32\drivers\Ndis.sys'
kd> g
BD: osloader.exe base address 00400000
BD: \WINDOWS\system32\NTKRNLMP.CHK base address 80A02000
BD: \WINDOWS\system32\HALMACPI.CHK base address 80100000
BD: \WINDOWS\system32\KDCOM.DLL base address 80010000
BD: \WINDOWS\system32\BOOTVID.dll base address 80001000
BD: \WINDOWS\system32\DRIVERS\ACPI.sys base address 8014C000
BD: \WINDOWS\system32\DRIVERS\WMILIB.SYS base address 80007000
BD: \WINDOWS\system32\DRIVERS\pci.sys base address 80062000
BD: \WINDOWS\system32\DRIVERS\isapnp.sys base address 80012000
BD: \WINDOWS\system32\DRIVERS\compbatt.sys base address 80009000
BD: \WINDOWS\system32\DRIVERS\BATTC.SYS base address 8000C000
BD: \WINDOWS\system32\DRIVERS\intelide.sys base address 8001C000
BD: \WINDOWS\system32\DRIVERS\PCIIDEX.SYS base address 8017A000
BD: \WINDOWS\System32\Drivers\MountMgr.sys base address 80181000
BD: \WINDOWS\system32\DRIVERS\ftdisk.sys base address 8018C000
BD: \WINDOWS\System32\drivers\dmload.sys base address 8001E000
BD: \WINDOWS\System32\drivers\dmio.sys base address 801AB000
BD: \WINDOWS\System32\Drivers\PartMgr.sys base address 801D1000
BD: \WINDOWS\System32\Drivers\VolSnap.sys base address 801D6000
BD: \WINDOWS\system32\DRIVERS\atapi.sys base address 801E3000
BD: \WINDOWS\system32\DRIVERS\vmscsi.sys base address 80073000
BD: \WINDOWS\system32\DRIVERS\SCSIPORT.SYS base address 801FB000
BD: \WINDOWS\system32\DRIVERS\disk.sys base address 80213000
BD: \WINDOWS\system32\DRIVERS\CLASSPNP.SYS base address 8021C000
BD: \WINDOWS\system32\drivers\fltmgr.sys base address 80229000
BD: \WINDOWS\system32\DRIVERS\sr.sys base address 802A7000
BD: \WINDOWS\System32\Drivers\KSecDD.sys base address 802B9000
KD: Accessing 'C:\Stuff\xpsp3checked\Ntfs.sys' (\WINDOWS\System32\Drivers\Ntfs.sys)
File size 814K.... ....BD: Loaded remote file \WINDOWS\System32\Drivers\Ntfs.sys

BlLoadImageEx: Pulled \WINDOWS\System32\Drivers\Ntfs.sys from Kernel Debugger
BD: \WINDOWS\System32\Drivers\Ntfs.sys base address 802D0000
KD: Accessing 'C:\Stuff\xpsp3checked\Ndis.sys' (\WINDOWS\System32\Drivers\NDIS.sys)
File size 424K.... ....BD: Loaded remote file \WINDOWS\System32\Drivers\NDIS.sys

BlLoadImageEx: Pulled \WINDOWS\System32\Drivers\NDIS.sys from Kernel Debugger
BD: \WINDOWS\System32\Drivers\NDIS.sys base address 804DC000
Shutdown occurred...unloading all symbol tables.
Waiting to reconnect.

We can see that the boot debugger picked up our driver replacements and transferred them from the host to the target through the kernel debugger connection. Alas, this can be a lengthy process for an obese driver over the 115,200 baud link…

Beyond being useful for replacing your own drivers, which is what I had in mind when I looked into this feature, the boot debugger can be used to easily go back and forth between Windows free build and checked build operating system components, as illustrated above. However, such use is not without its problems.

For one, replacing the kernel and the HAL with their checked counterparts through the driver replacement map does not work. An error citing kernel corruption results from such an attempt. The traditional way of using a checked kernel, by placing an appropriate entry in boot.ini, is still required.

When testing a file system filter driver, apart from using the checked version of the I/O Manager through the use of the checked NT kernel, it is advantageous to use checked versions of underlying file system drivers such as NTFS. The checked versions can assert when you pass on requests to them in a way which violates the file system’s locking hierarchy and which may lead to deadlocks. Replacing the NTFS driver with the driver replacement map feature worked as expected, apart from causing NDIS to bugcheck during system boot with some sort of paging error. The issue was resolved by replacing NDIS with its checked counterpart through the driver replacement map, as well.

However, for a reason I do not understand, when placing the checked build of the Filter Manager, useful for debugging file system minifilters, there was no such luck. The boot loader complained after transferring the checked Filter Manager that the NTFS driver was corrupt. I disabled System File Protection and replaced the free drivers with the checked drivers on disk, the traditional way and the system booted with the checked NTFS and Filter Manager successfully. So it appears that the boot-time driver replacement map feature can be a bit flaky…

It is probably best to place checked operating system components the traditional way and only replace your own, frequently modified drivers with the boot debugger and the driver replacement map.

So much for Windows XP and the legacy NTLDR. But what about Windows Vista?

At first, the situation looked promising. In Windows Vista, the boot debugger is built-in. It can, for instance, be enabled for an existing boot entry with the Boot Configuration Database editor from an elevated command prompt:

C:\Windows\system32>bcdedit /enum

Windows Boot Manager
--------------------
identifier {bootmgr}
device partition=C:
description Windows Boot Manager
locale en-US
inherit {globalsettings}
default {current}
displayorder {current}
{5761b19a-1e8a-11dd-bcd4-000c29797dc6}
toolsdisplayorder {memdiag}
timeout 30

Windows Boot Loader
-------------------
identifier {current}
device partition=C:
path \Windows\system32\winload.exe
description Microsoft Windows Vista
locale en-US
inherit {bootloadersettings}
osdevice partition=C:
systemroot \Windows
resumeobject {694d30db-e737-11dc-814f-e01223f3682a}
nx OptIn

Windows Boot Loader
-------------------
identifier {5761b19a-1e8a-11dd-bcd4-000c29797dc6}
device partition=C:
path \Windows\system32\winload.exe
description Debugging
locale en-US
inherit {bootloadersettings}
osdevice partition=C:
systemroot \Windows
resumeobject {694d30db-e737-11dc-814f-e01223f3682a}
nx OptIn
debug Yes

C:\Windows\system32>bcdedit /bootdebug {5761b19a-1e8a-11dd-bcd4-000c29797dc6} ON

The operation completed successfully.

Unlike the XP boot debugger, the Vista boot debugger is set for a specific boot loader menu entry. Once we reboot and pick the entry for which boot debugging is enabled, we can attach:

Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
Copyright (c) Microsoft Corporation. All rights reserved.

Opened \\.\pipe\com_1
Waiting to reconnect...
BD: Boot Debugger Initialized
Connected to Windows Boot Debugger 6001 x86 compatible target, ptr64 FALSE
Kernel Debugger connection established. (Initial Breakpoint requested)
Symbol search path is: C:\WINDOWS\Symbols;SRV*E:\SymStore*http://referencesource.microsoft.com/symbols;SRV*E:\SymStore*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows Boot Debugger Kernel Version 6001 UP Free x86 compatible
Primary image base = 0x00584000 Loaded module list = 0x00684e78
System Uptime: not available
Break instruction exception - code 80000003 (first chance)
winload!RtlpBreakWithStatusInstruction:
005bce88 cc int 3
kd> k
ChildEBP RetAddr
00120c6c 005b0862 winload!RtlpBreakWithStatusInstruction
00120e84 005b0760 winload!vDbgPrintExWithPrefixInternal+0x100
00120e94 0058bdaf winload!DbgPrint+0x11
00120eb0 0058bf6d winload!BlBdStart+0x81
00120f48 005a2f88 winload!BlBdInitialize+0x172
00120f64 005a28c2 winload!InitializeLibrary+0x168
00120f7c 0058513a winload!BlInitializeLibrary+0x42
00120fe8 0044646a winload!OslMain+0x13a
WARNING: Frame IP not in any known module. Following frames may be wrong.
00000000 f000ff53 0x44646a
00000000 00000000 0xf000ff53

We can see that in Vista, the boot debugger’s initial break is in the new winload.exe, replacing the osloader.exe embedded in ntldr of yesteryear. At this point the boot load drivers have yet to be loaded, so it would be perfect to set the .kdfiles driver replacement map at this point.

Alas, no such luck. It turns out the boot load driver replacement map feature is MIA in Windows Vista. This is confirmed by Microsoft’s Doron Holan in a reply to a post (free registration required) in OSR’s WINDBG mailing list. It is unclear what is the point of bundling the boot debugger with the regular operating system, unlike in the case of the hard to find ntldr_dbg for XP, only for it to be completely useless… Anyone using the boot debugger for purposes other than boot load driver replacement is probably working for Microsoft, so why should the boot debugger be a part of the OS if it is now missing what seems to be its most important functionality?

Hopefully the boot load driver replacement map will make a comeback in the Windows 7 boot debugger…