Debugging user-mode BootExecute native applications with kd

Debugging code executing during system startup always poses a unique challenge. One may need to debug a custom or built-in Windows service right from the start, when attaching to it after it has initialized proves insufficient or inappropriate. When developing a GINA hook or GINA stub, the need to debug the Winlogon process before the logon process is performed arises. The inability of the Visual Studio Debugger to be useful in these situations is one of the reasons people turn to Windbg.

For debugging Windows services or the Winlogon process during startup, Image File Execution Options provides a workable solution. As soon as a process of the name specified under the Image File Execution Options registry key is created, the debugger command-line specified in the Debugger value is executed in lieu of the original command-line, which is appended to the debugger command-line. The debugger started might be Visual Studio’s, if appropriate, an interactive Windbg in other cases or an NTSD remote debugging server when you will not or cannot do things like make the service process interactive.

For the vast majority of startup applications, the aforementioned technique is both quite sufficient and convenient. However, there is another, perhaps esoteric, category of startup processes. These run a very early stage of the boot process. They are the BootExecute applications.

BootExecute applications are started by the Session Manager (smss.exe) before invoking the “initial command” (Winlogon in XP) and before the various subsystems are started. As far as user-mode goes, it doesn’t get much earlier than this. Because of their early nature, a significant constraint is in place for BootExecute applications: they are native applications.

Do not confuse this usage of “native” with native code vs. .NET managed code. In this context, native means that only the Windows NT Native API, resident in ntdll.dll, is available. At this stage, the Win32 subsystem, composed of the kernel-mode win32k.sys component and the user-mode client/server runtime, CSRSS, have not yet been started by SMSS. Not even the Kernel32 library is usable by BootExecute applications.

What are these useful for? Those special tasks that must be performed before everything else has started in the system, yet remain in the domain of user-mode work. Consider these two typical examples:

  • AutoCheck, the BootExecute variant of the CHKDSK tool, used to examine the boot volume before it is locked and to fix critical file-system errors.
  • Sysinternals PageDefrag, a BootExecute utility that defragments the Paging File, registry hives and other files inaccessible to defragging by the normal Win32 Disk Defragmentation tool.

We can confirm that AutoCheck is indeed a native application by examining it with Visual C++’s DUMPBIN utility:

C:\WINDOWS\system32>dumpbin /headers autochk.exe
Microsoft (R) COFF/PE Dumper Version 9.00.21022.08
Copyright (C) Microsoft Corporation. All rights reserved.

Dump of file autochk.exe

PE signature found

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES
14C machine (x86)
4 number of sections
48025203 time date stamp Sun Apr 13 21:33:39 2008
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
10E characteristics
Executable
Line numbers stripped
Symbols stripped
32 bit word machine

OPTIONAL HEADER VALUES
10B magic # (PE32)
7.10 linker version
5B800 size of code
34200 size of initialized data
0 size of uninitialized data
D6B9 entry point (0100D6B9) _NtProcessStartupForGS@4
1000 base of code
5D000 base of data
1000000 image base (01000000 to 01091FFF)
1000 section alignment
200 file alignment
5.01 operating system version
5.01 image version
5.01 subsystem version
0 Win32 version
92000 size of image
400 size of headers
96A6F checksum
1 subsystem (Native)
0 DLL characteristics
40000 size of stack reserve
1000 size of stack commit
100000 size of heap reserve
1000 size of heap commit
0 loader flags
10 number of directories
... snipped ...

Notice the subsystem specified for AutoChk is the native subsystem. Notice further that the application’s entrypoint is NtProcessStartup (in its /GS compiler stack buffer overflow protection stub form).

As for PageDefrag, it takes advantage of the Session Manager running its BootExecute application before it has enabled use of the Paging File.

You may find reasons of your own to develop a BootExecute native application, or you may find yourself in a situation requiring debugging of an existing BootExecute application. For instance, you may wish to debug the interactions of AutoChk’s volume locking attempts with your file system filter driver.

Unfortunately, these native applications pose a special difficulty to the user-mode debugger. NTSD is a Win32 application and must be invoked only after the Win32 subsystem has been initialized. Therefore, invocation of NTSD for debugging BootExecute applications is out of the question. Indeed, it is quite likely the Image File Execution Options registry key is not even consulted for BootExecute invocations, as that would be quite pointless.

Theoretically, this problem could be addressed by the development of a native subsystem user mode debugger, in lieu of the Win32-based NTSD. Alex Ionescu, most recently contributing to the eagerly awaited 5th edition of the Windows Internals book, has discussed the specifics of the NT Native Debugging API (DbgUi, etc.) in a series of articles titled Windows Native Debugging Internals.

At the moment, however, I am unaware of any available native subsystem user mode debugger. Such a tool may or may not be available internally in Microsoft. Presumably the Windows developers would benefit from such functionality, but they might also be content with using the kernel debugger for those purposes.

Be that as it may, the rest of us must turn to the kernel debugger for resolution. The kernel debugger can be used for source-level debugging of user-mode applications, including native subsystem applications. The special difficulty with using it is getting to break in the right place at the right time. In lieu of a Image File Execution Options-style apparatus, an alternative approach is required.

When modifying the native BootExecute application in question is feasible, the simple approach of adding an invocation of ntdll’s DbgBreakPoint API to the top of the NtProcessStartup process entrypoint is probably the quickest way to get the desired effect. In the absence of a user-mode debugger, the debug break will make its way to the kernel debugger. The debugger will notice the presence of the user-mode module, load symbols and source and the usual debugger functions will be accessible. If source is not available, in many cases the image can be patched to contain either an invocation of DbgBreakPoint or just an inline INT 3, as appropriate.

Such an approach, however, may not be feasible at all times and has the significant disadvantage of making the modified native application hang when a kernel debugger is not attached to the system at boot. Ideally, we’d like to break at process startup without modifying the native application at all.

When using the user-mode debugger, “sxe ld” can break when user-mode modules are mapped by the loader, as documented in Controlling Exceptions and Events. Normally, the kernel debugger does not provide that capability. However, it turns out that it can do so, once appropriately configured.

Before booting with the kernel debugger, turn on the “Enable loading of kernel debugger symbols” Global Flag, using the GFlags utility bundled with the Debugging Tools for Windows:

C:\Program Files\Debugging Tools for Windows>gflags /r +ksl
Current Boot Registry Settings are: 00040000
ksl - Enable loading of kernel debugger symbols

Although the name and description of this Global Flag appear to have nothing to do with user-mode module load events in the kernel debugger, they acheive the desired effect. Once enabled, we can reboot with the kernel debugger attached and ask for the kernel debugger to break once the desired native application is mapped:

Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
Copyright (c) Microsoft Corporation. All rights reserved.

Connected to Windows XP 2600 x86 compatible target, ptr64 FALSE
Kernel Debugger connection established. (Initial Breakpoint requested)
Symbol search path is: C:\WINDOWS\Symbols;SRV*E:\SymStore*http://referencesource.microsoft.com/symbols;SRV*E:\SymStore*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows XP Kernel Version 2600 MP (1 procs) Free x86 compatible
Built by: 2600.xpsp.080413-2111
Kernel base = 0x804dc000 PsLoadedModuleList = 0x805684c0
System Uptime: not available
Break instruction exception - code 80000003 (first chance)
... snipped breakpoint warning message ...
nt!RtlpBreakWithStatusInstruction:
804e7a42 cc int 3
kd> sxe ld:autochk
kd> g
nt!DebugService2+0x10:
8050ae56 cc int 3

Setting the kernel debugger to break on the load of the AutoChk native BootExecute application resulted in our desired break. Let us consider the context of this break:

0: kd> kb
ChildEBP RetAddr Args to Child
f738d9fc 8050b2f9 f738da40 f738da10 00000003 nt!DebugService2+0x10
f738da20 805c533a f738da40 01000000 82953020 nt!DbgLoadImageSymbols+0x42
f738da70 805c51f0 82ab9c28 01000000 82953020 nt!MiLoadUserSymbols+0x169
f738dab4 8058d013 82ab9c28 01000000 f738db5c nt!MiMapViewOfImageSection+0x4b6
f738db10 80504e27 00000004 82953110 f738db5c nt!MmMapViewOfSection+0x13c
f738db6c 80590520 e165ec14 00000000 e1412398 nt!MmInitializeProcessAddressSpace+0x33d
f738dcbc 8059082f 0015f870 001f0fff 0015f7d8 nt!PspCreateProcess+0x333
f738dd10 805b54b2 0015f870 001f0fff 0015f7d8 nt!NtCreateProcessEx+0x7e
f738dd3c 804e298f 0015f870 001f0fff 0015f7d8 nt!NtCreateProcess+0x3d
f738dd3c 7c90e4f4 0015f870 001f0fff 0015f7d8 nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
0015f830 00000000 00000000 00000000 00000000 0x7c90e4f4
0: kd> !process 0 0
**** NT ACTIVE PROCESS DUMP ****
PROCESS 82bc9830 SessionId: none Cid: 0004 Peb: 00000000 ParentCid: 0000
DirBase: 02f40000 ObjectTable: e1002e40 HandleCount: 46.
Image: System

PROCESS 82935128 SessionId: none Cid: 0218 Peb: 7ffd9000 ParentCid: 0004
DirBase: 0899b000 ObjectTable: e13fbc68 HandleCount: 7.
Image: smss.exe

Although AutoChk has been mapped into memory, the AutoChk process is still in the process of being created. Indeed, the AutoChk process is as of yet absent from the system process list displayed by the !process debugger extension command.

However, AutoChk’s pseudo-created state does not prevent us from taking this opportunity to set up a debug breakpoint at the top of user code:

1: kd> lm m autochk
start end module name
01000000 01092000 autochk (deferred)
1: kd> bp autochk!NtProcessStartup
1: kd> bl
0 e 0100dd3d 0001 (0001) autochk!NtProcessStartup

Beware that if you perform a symbol reload with the .reload command after the module load event for autochk has fired off, you may find that it has disappeared from the debugger’s loaded module list… Just make sure you set up your breakpoint immediately after the event break.

It is easy enough to set a breakpoint at the application’s NtProcessStartup entrypoint before the EPROCESS is available, but we may wish to to set early breakpoints in process context elsewhere. To that end, we may proceed to the return from the process creation API from the module load event break, until the process is listed in the system process list:

1: kd> k
ChildEBP RetAddr
f7b619fc 8050b2f9 nt!DebugService2+0x10
f7b61a20 805c533a nt!DbgLoadImageSymbols+0x42
f7b61a70 805c51f0 nt!MiLoadUserSymbols+0x169
f7b61ab4 8058d013 nt!MiMapViewOfImageSection+0x4b6
f7b61b10 80504e27 nt!MmMapViewOfSection+0x13c
f7b61b6c 80590520 nt!MmInitializeProcessAddressSpace+0x33d
f7b61cbc 8059082f nt!PspCreateProcess+0x333
f7b61d10 805b54b2 nt!NtCreateProcessEx+0x7e
f7b61d3c 804e298f nt!NtCreateProcess+0x3d
f7b61d3c 7c90e4f4 nt!KiFastCallEntry+0xfc
0015f830 00000000 ntdll!KiFastSystemCallRet
1: kd> gu; gu; gu; gu; gu; gu; gu
nt!NtCreateProcessEx+0x7e:
8059082f e87a76f5ff call nt!_SEH_epilog (804e7eae)
1: kd> !process 0 0
**** NT ACTIVE PROCESS DUMP ****
PROCESS 82bc9830 SessionId: none Cid: 0004 Peb: 00000000 ParentCid: 0000
DirBase: 02f40000 ObjectTable: e1002e40 HandleCount: 46.
Image: System

PROCESS 829b4128 SessionId: none Cid: 0228 Peb: 7ffd7000 ParentCid: 0004
DirBase: 08bbb000 ObjectTable: e1468f58 HandleCount: 8.
Image: smss.exe

PROCESS 8294a3d0 SessionId: none Cid: 0238 Peb: 7ffd6000 ParentCid: 0228
DirBase: 08d40000 ObjectTable: e13fd408 HandleCount: 0.
Image: autochk.exe

By examining our location in the call stack after the module load event fires, we can see that returning from the Process Manager’s process creation routine PspCreateProcess would require going up 7 times. With that routine’s execution completed, the EPROCESS for autochk is now listed in the system process list and its value can be used as a context parameter for breakpoint commands, etc.

With the breakpoint on the native entrypoint in place, we can resume system execution and have the kernel debugger land right where we want it:

1: kd> g
Breakpoint 0 hit
autochk!NtProcessStartup:
001b:0100dd3d 8bff mov edi,edi
1: kd> kb
ChildEBP RetAddr Args to Child
0006fff4 00000000 7ffde000 000000c8 0000010a autochk!NtProcessStartup
1: kd> .process
Implicit process is now 82935020
1: kd> .thread
Implicit thread is now 8293f020

From this point, convenient source debugging of the native application is also possible if it’s your own custom written application. The various features such as Locals, Watches, single stepping, etc., work as expected. Some quirks of kernel debugging of a user process should be taken into consideration (make sure breakpoints have an EPROCESS and ETHREAD context specified when appropriate to avoid venturing into other processes by accident, etc.) and the inaccessibility of some user-mode debugger extension commands may prove inconvenient.

Sure beats DbgPrints, though!

Advertisements

One thought on “Debugging user-mode BootExecute native applications with kd

  1. I’m trying to mount a volume at startup, and am experiencing exorbitant issues.

    The problem is (I think) because I’m attempting execution either too early, or with insufficient ‘credentials’.

    analysing event viewer logs I see that, naturally, VHD Attach performs it’s mount operation AFTER ‘server’ and BEFORE ‘wmi’

    svc boot order (sorted by newest events)
    wia
    wmi
    vhd
    server
    protected storage
    intel clock
    daemon
    browser
    winhttp

    I tried adding VHD Attach to a ServiceGroup, and stipulating that this load eary (after FileSystem group), but this is not honoured.

    I tried (for a day) with sc tweaks, discovering sdshow and sdet, but not understanding.

    I’ve tried BootExecute entries, but (perhaps as the exe is non-native?) nothing happens (well, a boot delay is experienced, but no joy, no mounted volume).

    Am I being unrealistic, to expect to be able to begin to mount a volume earlier than currently possible? Adventures in Scheduled Tasks have met similar fates.

    Most disconcerting

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s