Modifying a Visual C++ 2010 project’s Platform Toolset programmatically with IVCRulePropertyStorage

Back from a long hiatus.

Lately I’ve been migrating a large legacy code base from Visual C++ 2008 to Visual C++ 2010. Visual C++ 2010 supports Native Multi-Targeting, which lets you use the Visual Studio 2010 IDE and the new MSBuild-based C++ project system (*.vcxproj files, etc.) while invoking a legacy compiler like Visual C++ 2008 for the actual build, or for invoking the modern compiler with special INCLUDE and LIB directory settings (e.g., using the Windows Driver Kit’s headers and import libraries).

The “Platform Toolset” is the project setting which specifies which baseline compiler settings apply. Visual Studio 2010 bundles the “v100” platform toolset for using the Visual C++ 2010 compiler and the “v90” platform toolset for using the Visual C++ 2008 compiler. The Windows SDK 7.1 includes the “Windows7.1SDK” platform toolset which invokes the Visual C++ 2010 compiler, but with the SDK’s build environment. The latest Intel C++ Compiler includes a “Intel C++ Compiler XE 12.0” toolset, as well. A third party has even created a platform toolset for the antiquated Visual C++ 2003 compiler.

The IDE GUI exposes the Platform Toolset project setting in the General property sheet. By right clicking a project in Solution Explorer, clicking Properties and going to General under Configuration Properties, you can change the setting on a per-build configuration basis. Solution Explorer is clever enough to let you group select several or all of the projects in the open solution and group-modifying the platform toolset in the properties dialog. For many cases, this may be sufficient.

However, if you want to modify the platform toolset en masse, but based on more complicated criteria, or just wish to achieve a higher degree of automation in the migration process, you may prefer to modify the setting programmatically like I did.

Visual Studio has an automation model for modifying project settings and for many other IDE tasks. This object model is accessible from Visual Studio Macros (which have unfortunately been removed from the Visual Studio 11 Developer Preview) and from other forms of Visual Studio extensibility such as add-ins. After some browsing of the documentation and searching the Web, it is clear how to modify things like the project’s Linker OutputFile property. However, it took me a while to figure out how to access other properties like the PlatformToolset.

It turns out that project settings that have appeared recently are not accessible in straightforward way like the older ones. That is, they are not properties of the VCProject object or its child objects. Instead, there is a new interface for manipulating these properties, IVCRulePropertyStorage.

With this interface, the PlatformToolset property in the General property sheet becomes easily accessible. To illustrate, the following is an example Visual Studio macro that changes the platform toolset of any project in the solution using the “v100” toolset to the “Windows7.1SDK” toolset:


Option Strict On
Option Explicit On

Imports System
Imports EnvDTE
Imports EnvDTE80
Imports EnvDTE90
Imports EnvDTE90a
Imports EnvDTE100
Imports System.Diagnostics
Imports Microsoft.VisualStudio.VCProject
Imports Microsoft.VisualStudio.VCProjectEngine

Public Module Module1
Sub RetargetProjects()
  For Each p As Project In DTE.Solution.Projects
    If p.Kind = vcContextGuids.vcContextGuidVCProject Then
      Dim prj As VCProject = CType(p.Object, VCProject)

      For Each cfg As VCConfiguration In CType(prj.Configurations, IVCCollection)
        Dim generalRule As IVCRulePropertyStorage = CType(cfg.Rules.Item("ConfigurationGeneral"), IVCRulePropertyStorage)

        Dim currentPlatformToolset = generalRule.GetUnevaluatedPropertyValue("PlatformToolset")

        If currentPlatformToolset = "v100" Then
          generalRule.SetPropertyValue("PlatformToolset", "Windows7.1SDK")
        End If
      Next
    End If
  Next
End Sub
End Module

You need to add references to the VCProject and VCProjectEngine assemblies in your macro project to access the types of the Visual C++ Project Model used in this macro.

Stay tuned for more useful functionality of IVCRulePropertyStorage.

Advertisements

Visual Studio 2010 Beta 2 debugger may be confused by your symbol path

I’ve been evaluating Microsoft’s Visual Studio 2010 Beta 2 release recently on my Windows 7 x64 system. As can be expected, the beta has quite a few rough edges, but overall I like the new WPF-based IDE GUI and the refreshed code Editor in particular (I like how selecting a block of code now preserves syntax highlighting while the block is highlighted, for instance).

The new GUI does have some badness. I had particular distaste for the poor aliasing of tooltips in the editor. This can be seen, for example, when hovering over a function name like “printf”. The menu bar’s dark blue color scheme also appears rather peculiar.

Besides the “bling”, notable changes include the Visual C++ 2010 CRT reverting to the traditional deployment model used by the Visual C++ 2003 CRT. More specifically, the CRT DLLs no longer use SxS binding (“Fusion”) and are now simply deployed to the “system32” directory or to the application’s directory, as desired. Dropping SxS has some obvious disadvantages (SxS binding redirects would no longer be able to redirect applications that load a private copy of the CRT DLLs to updated versions with bug fixes and security updates) but presumably the pain of integrating SxS deployment into the setup process, which required either an MSI installation or pseudo-documented use of the SxS API, resulted in too much negative feedback and they chose to revert to the legacy approach.

Visual C++ projects are now built with MSBuild, like their C# and other .NET counterparts. This should have several benefits. One that comes to mind is the the Windows Installer XML Toolset’s Votive (its VS IDE integration component) should be able to support C++ projects as References in addition to .NET projects.

A more important update to the project system is support for multi-targeting. Most of Microsoft’s discussions on the subject mainly deal with said support for .NET projects, with the new Visual Studio being able to target .NET 2.0 through .NET 4.0 on a per-project basis. However, similar support is offered for native multi-targeting. A per-project setting specifies the “toolset” with which it is to be built. The product comes built-in with toolset definitions for the VC++ 10 and the VC++ 9 compilers, but since toolset definitions are simple XML files describing tool paths, older compilers and custom definitions are easy to define. Indeed, a toolset definition can be found for the Windows 7 SDK build environment. I foresee using this functionality to build user-space applications with headers and tools from the Windows Driver Kit build environment, resulting in being able to link with the OS CRT (msvcrt.dll) in a clean way, without modifying global Visual C++ directory settings, but rather keeping the changes contained to specific projects.

My enthusiasm for testing the product was struck a severe blow when my first attempt to run the Visual C++ debugger on a “Hello, World!” console application went awry. The IDE was hung for a good 15-20 seconds. The IDE sat frozen for quite a bit after F10 was pressed to initiate the debugging session, finally presented the console window for the test application and then spent some more time being frozen. After a lengthy wait, the session was finally ready.

But the worst part was that when the debugging session was finally ready, debugging symbols for all modules except the application .exe and the VC 10 CRT were NOT loaded! It appeared as though the lengthy wait was all for naught.

The experience was sufficiently poor for me to report it to Microsoft through one of the feedback channels. I was eventually contacted by helpful folks from the Visual Studio Debugger team and we analyzed the problem in an e-mail exchange. The performance issue is the result of problematic contents of the symbol path I configured for the debugger.

As I mentioned, I’m evaluating the beta on a Windows 7 machine. For this reason, the Windows 7 symbol packages, available from Microsoft’s public symbols download page, are deployed in my system and are a part of the symbol path defined by the _NT_SYMBOL_PATH environment variable. Since this is an x64 machine and I find myself debugging 32-bit processes quite often, both the 32-bit and 64-bit symbol packages are installed. My initial symbol path was:

C:\Symbols;C:\Symbols32;CACHE*C:\websymbols;SRV*http://msdl.microsoft.com/download/symbols

The first issue with this symbol path is that the x64 symbols package (extracted to C:\Symbols) and the x86 symbols package (extracted to C:\Symbols32) are specified as directories in the symbol path, rather than symbol stores. This is what you’d expect from symbol packages designed for local deployment, but it turns out that the Windows 7 symbol packages, unlike the PDB packages for previous versions of Windows, come in the symbol store directory layout rather than the flat directory layout. This means, for example, that the symbols for ntdll.dll are in a path like C:\Symbols\ntdll.pdb\CFF40300FD804691B73E12CF2A150EE02\ntdll.pdb rather than the simpler C:\Symbols\dll\ntdll.pdb.

I did not notice this issue, however, before installing Visual Studio 2010 Beta 2, because apparently Windbg doesn’t mind when stores are specified in this syntax, as it exercises some sort of heuristic to determine the layout of a symbol directory. However, Visual Studio 2010 Beta 2 is not as liberal. Examining its I/Os with Sysinternals Process Monitor determined that it wasn’t trying to find PDBs under the symbol directories except directly under them or in a “dll” subdirectory, rather than looking for the appropriate hash as it would in a symbol store. The resolution for this issue is simple enough, refer to the Windows 7 symbol packages with SRV* syntax in the symbol path. Therefore, the symbol path is updated to something like:

SRV*C:\Symbols;SRV*C:\Symbols32;CACHE*C:\websymbols;SRV*http://msdl.microsoft.com/download/symbols

With this change in place, the Visual Studio 2010 Beta 2 debugger was able to pick up symbols for system DLLs from the local stores, and now the debugging session started instantly. But the question remained: even if the debugger didn’t know how to look for symbols under C:\Symbols and C:\Symbols32 when they were not specified with a srv* directive, why did it download symbols from the HTTP public store, only to end up starting with symbols not being loaded for any of the system DLLs in the debugged process?

To get to the bottom of this, the local symbol caches were removed from the symbol path. At this point, it was

CACHE*C:\websymbols;SRV*http://msdl.microsoft.com/download/symbols

Running the debugger with this stripped down symbol path reproduced the poor debugger startup experience and the worse issue of symbols being downloaded only to end up not being used. At this point, Sysinternals Process Monitor was used to examine the actions of the debugger. Two curious facts were revealed.

The first was that the Visual Studio debugger was literally examining a directory called “cache*C:\websymbols” under its path in a vain attempt to find symbols. Since the “cache*” string made it to a file open request, obviously the cache* directive in the _NT_SYMBOL_PATH variable was not being correctly parsed or understood by the debugger.

The result of this deficiency is that the Visual Studio debugger should be using some default local cache directory for the downloaded symbols, instead of the one explicitly specified by the cache* directive. Therefore, the same behavior would be expected with the following symbol path:

SRV*http://msdl.microsoft.com/download/symbols

And indeed, a quick check revealed that the same peculiarity reproduced with this symbol path setting: symbols were being downloaded from the HTTP server, but in the end of the process, symbols were not loaded for any of the system modules in the debugged process. Whatever the default directory is, attempts to download symbols there resulted in their loss into oblivion.

Specifying the local symbol cache using the SRV* directive is also possible. This is the legacy approach, before Windbg recommend using CACHE* instead. A symbol path of this form is

SRV*C:\websymbols*http://msdl.microsoft.com/download/symbols

With this symbol path in place, the Visual Studio 2010 Beta 2 debugger both downloaded symbols from the HTTP store and actually ended up using them. Specifying the cache directory with CACHE* or not at all triggered the bug, while specifying it the old fashioned way in the SRV* directive satisfied the debugger.

As a result, my guidance for the Visual Studio 2010 Beta 2 debugger users experiencing performance issues or other symbol problems that have no issue with their symbol path when used with Windbg is:

  1. For each directory in your _NT_SYMBOL_PATH, determine whether it is in the “flat” format or in symbol store format. Prefix symbol stores with SRV*, changing “C:\Symbols” to “SRV*C:\Symbols”. Windows 7 users in particular should be aware that the symbol packages for their platforms should be specified with SRV* syntax for Visual Studio 2010 Beta 2.
  2. Specify local caches directories for remote symbol stores (SMB or HTTP) directly in the SRV* directive, to have Visual Studio 2010 Beta 2 pick them up. It is OK to keep the CACHE* directive in your symbol path as well, but for the time being, Visual Studio 2010 Beta 2 does not seem to use it correctly.

The Visual Studio Debugger team is addressing issues revealed by the investigation of this behavior for the forthcoming RTM release of Visual Studio 2010. However, several significant deficiencies in debugger symbol support will not be addressed in the 2010 release. One is that symbol loading is done synchronously rather than asynchronously to the debugging session. The other being that unless manual symbol loading is used, no progress indication nor cancellation UI is presented as the symbols are being transferred from a remote store. Therefore, the perceived performance of symbol support in the debugger will leave something to be desired for the time being.

Implementing an LSA proxy authentication package

LSA authentication packages are a part of the core security ecosystem in Windows NT. LSA APs are provided with credentials by logon applications, such as Winlogon, authenticate these credentials and provide the logon application with a logon session handle if authentication was successful. Two authentication packages provided with Windows are of special interest. MSV1_0 authenticates user credentials against the local SAM database or against a domain controller through the legacy logon protocols. Kerberos.dll uses the Kerberos protocol with modern domain controllers (or third party KDCs, etc.) for establishing logon sessions.

Applications that want to provide value-added logon functionality wish to become involved in the logon process. Winlogon provides the GINA facility before Windows Vista and the new Credential Providers in Windows Vista and later to allow involvement in the end-user logon experience.

While customizing the logon process is done at the logon application side with GINAs or CPs, sometimes customization is required at the authentication side, in the Local Security Authority subsystem. For instance, a fingerprint reader may customize the logon UI to allow fingerprint scanning as a means of authentication. The customized logon application will call LsaLogonUser with the fingerprint scanner’s custom LSA authentication package as the desired authenticator. The fingerprint LSA AP will scan a fingerprint database it maintains to perform the authentication process. That would be an independent LSA authenticaion package.

There are other scenarios in which the regular MSV1_0/Kerberos based logon authentication process is desired, but special pre or post-processing is required. In Windows XP, the problematic GINA architecture makes replacing the GINA cumbersome, as multiple third-party applications may attempt to install conflicting GINA replacements. Indeed, Microsoft documentation has been modified retroactively to recommend GINA hooks and stubs in lieu of outright replacement of the MSGINA implementation previously suggested.

With the ability to instrument the logon application at the GINA side limited and version dependent (a totally different approach is required for Vista’s CPs), the alternative of instrumentation at the LSA side warrants exploration. We can implement an LSA proxy authentication package. Such a package would appear to the LSA server as an ordinary package, but would be implemented by delegating to original APs like MSV1_0 and Kerberos, providing it with monitoring and instrumentation capabilities.

LSA authentication packages are loaded by LSASS at system startup by enumerating them from the registry value HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\Security Packages. Both LSA authentication packages (APs) and security providers (SSPs) are registered at that location. For the purposes of this post, we’ll ignore pure SSPs and focus only on APs. We will register our proxy authentication package instead of MSV1_0 (and possibly Kerberos) at this location, and leave loading of the original APs to our proxy AP implementation.

Let us examine the system’s core APs, MSV1_0 and Kerberos:

C:\WINDOWS\system32>dumpbin /exports msv1_0.dll
Microsoft (R) COFF/PE Dumper Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file msv1_0.dll
File Type: DLL
Section contains the following exports for msv1_0.dll
00000000 characteristics
48025C84 time date stamp Sun Apr 13 22:18:28 2008
0.00 version
1 ordinal base
32 number of functions
17 number of names
ordinal hint RVA name
5 0 0000175C LsaApCallPackage = _LsaApCallPackage@28
6 1 00014BC8 LsaApCallPackagePassthrough = _LsaApCallPackagePassthr
ough@28
7 2 00014A59 LsaApCallPackageUntrusted = _LsaApCallPackageUntrusted
@28
8 3 0000BDBB LsaApInitializePackage = _LsaApInitializePackage@20
9 4 0000F7FE LsaApLogonTerminated = _LsaApLogonTerminated@4
10 5 00007498 LsaApLogonUserEx2 = _LsaApLogonUserEx2@64
11 6 0001A7E5 Msv1_0ExportSubAuthenticationRoutine = _Msv1_0ExportSu
bAuthenticationRoutine@40
12 7 0001A7C6 Msv1_0SubAuthenticationPresent = _Msv1_0SubAuthenticat
ionPresent@4
13 8 00016E63 MsvGetLogonAttemptCount = _MsvGetLogonAttemptCount@0
2 9 0001B704 MsvIsLocalhostAliases = ?MsvIsLocalhostAliases@@YGHPAU
_UNICODE_STRING@@@Z (int __stdcall MsvIsLocalhostAliases(struct _UNICODE_STRING
*))
14 A 00016E59 MsvSamLogoff = _MsvSamLogoff@12
15 B 0000A7BA MsvSamValidate = _MsvSamValidate@52
16 C 00016E6E MsvValidateTarget = _MsvValidateTarget@12
1 D 0000E415 SpInitialize = _SpInitialize@12
32 E 00006BC2 SpInstanceInit = _SpInstanceInit@12
3 F 0000F08F SpLsaModeInitialize = ?SpLsaModeInitialize@@YGJKPAKPAP
AU_SECPKG_FUNCTION_TABLE@@0@Z (long __stdcall SpLsaModeInitialize(unsigned long,
unsigned long *,struct _SECPKG_FUNCTION_TABLE * *,unsigned long *))
4 10 00006AE0 SpUserModeInitialize = ?SpUserModeInitialize@@YGJKPAKP
APAU_SECPKG_USER_FUNCTION_TABLE@@0@Z (long __stdcall SpUserModeInitialize(unsign
ed long,unsigned long *,struct _SECPKG_USER_FUNCTION_TABLE * *,unsigned long *))
Summary
2000 .data
2000 .reloc
2000 .rsrc
1D000 .text
C:\WINDOWS\system32>dumpbin /exports kerberos.dll
Microsoft (R) COFF/PE Dumper Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file kerberos.dll
File Type: DLL
Section contains the following exports for Kerberos.dll
00000000 characteristics
48025C4A time date stamp Sun Apr 13 22:17:30 2008
0.00 version
1 ordinal base
32 number of functions
10 number of names
ordinal hint RVA name
5 0 00026F8A KerbCreateTokenFromTicket = _KerbCreateTokenFromTicket
@44
2 1 00025773 KerbDomainChangeCallback = ?KerbDomainChangeCallback@@
YGXW4_POLICY_NOTIFICATION_INFORMATION_CLASS@@@Z (void __stdcall KerbDomainChange
Callback(enum _POLICY_NOTIFICATION_INFORMATION_CLASS))
6 2 000018B0 KerbFree = _KerbFree@4
7 3 00020A71 KerbIsInitialized = _KerbIsInitialized@0
8 4 00020A7C KerbKdcCallBack = _KerbKdcCallBack@0
9 5 0000380B KerbMakeKdcCall = _KerbMakeKdcCall@32
1 6 00013CAD SpInitialize = _SpInitialize@12
32 7 0000EDDF SpInstanceInit = _SpInstanceInit@12
3 8 000151DE SpLsaModeInitialize = ?SpLsaModeInitialize@@YGJKPAKPAP
AU_SECPKG_FUNCTION_TABLE@@0@Z (long __stdcall SpLsaModeInitialize(unsigned long,
unsigned long *,struct _SECPKG_FUNCTION_TABLE * *,unsigned long *))
4 9 0000ED1E SpUserModeInitialize = ?SpUserModeInitialize@@YGJKPAKP
APAU_SECPKG_USER_FUNCTION_TABLE@@0@Z (long __stdcall SpUserModeInitialize(unsign
ed long,unsigned long *,struct _SECPKG_USER_FUNCTION_TABLE * *,unsigned long *))
Summary
2000 .data
3000 .reloc
3000 .rsrc
43000 .text

Both DLLs export the SpInitialize function used for SSP initialization and for AP initialization export SpLsaModeInitialize and SpUserModeInitialize, called in LSASS context and in logon application context by secur32.dll, respectively.

As can be seen in MSV1_0’s export table, it provides a subauthentication package facility for those interested in becoming involved in the logon process. Clearly, this is the facility Microsoft intended third-parties to use, rather than creating proxy authentication packages. Unfortunately, subauthentication packages only provide a subset of authentication package functionality, primarily failing to provide access to the interactive logon (as opposed to the network logon) process. You should carefully consider your requirements. If they can be met with a subauthentication package, it is best to opt for that approach as it is far more likely to remain intact in Windows 7 and beyond. That said, the proxy authentication package model is given some legitimacy by this diagram, illustrating delegation to MSV1_0 by a hypothetical third-party custom authentication package.

It is interesting to note that MSV1_0 chooses to export the various LSA AP functions, such as LsaApLogonUserEx2, etc., via its DLL export table, while Kerberos opts not to export them, though quick examination of Kerberos.dll’s public symbols clearly illustrates the presence of the AP functions. So how does the LSA server know how to find and call the various AP functions? They are provided in a function dispatch table filled by the AP at initialization time, when SpLsaModeInitialize or SpUserModeInitialize are invoked (depending on the context of the AP’s use).

Keeping in mind that MSV1_0 was around before Kerberos was added to Windows NT in the Windows 2000 release and therefore may have remaining legacy cruft, reviewing the disassembly of the LSA server DLL, LSASRV, reveals that LSA first calls the Initialize functions to let the AP fill the function dispatch table, but then promptly examines the AP’s export table with GetProcAddress in the aptly named lsasrv!BindOldPackage function. This function appears to be invoked for the AP even when the function dispatch table has been filled. However, empirical evidence appears to suggest that the function dispatch table takes precedence over the contents of the export table. Indeed, Kerberos opts to do away with the export table entirely.

It appears that the export-based model was replaced with the dispatch table based model at some point during the lifetime of Windows NT. The export model has an obvious disadvantage of allowing a single DLL to implement only a single authentication package, a restriction will come back to later. The dispatch table model, on the other hand, allows the initialization function to return multiple dispatch tables, for multiple authentication packages, from a single DLL. This may yet prove useful for a proxy package.

In order to explore the feasibility of creating a proxy LSA authentication package, I wrote a test DLL, dubbed “LsaPrxAp”. This DLL provides the SpInitialize, SpLsaModeInitialize and SpUserModeInitialize exports. When invoked, it loads msv1_0 if it is not yet loaded. The DLL’s Initialize functions call the original msv1_0’s Initialize functions and then hook the authentication package by replacing function pointers with pointers to implementations of the LSA AP functions residing in LSAPrxAp. The original function pointers are saved in a global data structure so that LSAPrxAp’s stubs can invoke them while having value-added processing during the logon sequence. Since the name of the authentication package is the one provided in the SpLsaModeInitialize function by the AP, rather than the DLL’s name, it is easy to proxy the MSV1_0 module and become involved in the logon process. I did not bother with supporting Kerberos in my test, but hypothetically the proxy AP can call the Initialize functions for both MSV1_0 and Kerberos and return two dispatch tables as the parameter model for it allows.

Proxying a known authentication package of a known revision is one thing, but a more generic approach warrants special consideration. In both XP and Vista, the APs provide the latest LsaApLogonUserEx2 function. However, it is preceeded by LsaApLogonUserEx and LsaApLogonUser. The documentation for AP developers seems to indicate it is still legitimate to only support one of the older variants. Therefore, the proxy should take note that NULL function pointers in the dispatch table provided by the original APs are possible and should react accordingly.

As I investigated the behavior of the LSA server with respect to the export-based model for locating LSA AP functions vs. the function dispatch table model, I considered the problem of implementing a generic proxy if the dispatch table model was not preferred over the export table. The proxy AP would have to load the original AP and only have an LsaApLogonUserEx2 function if the original AP has. If the original AP has no LsaApLogonUser function exported, neither should the proxy AP. This presents special complication since compiling a different proxy AP for every possible set of exports is not feasible. I reiterate that the problem is theoretical since the function dispatch table model has no such fallacy.

In order to deal with this issue, I devised an approach where the proxy AP would initially export all possible LSA AP functions. At runtime, as the actual exports of the original AP is detected, functions that are missing from the original AP can be unexported from the proxy AP. I discuss the technique of unexporting functions through name obfuscation in the previous post.

Successfully compiling an LSA AP DLL required numerous acts of header juggling. For reference purposes, I provide the sequence I used:

#include <ntstatus.h>
#define WIN32_NO_STATUS
#include <Windows.h>
#include <NTSecAPI.h>
#define SECURITY_WIN32
#include <SSPI.h>

#ifndef NT_SUCCESS
#define NT_SUCCESS(Status) ((NTSTATUS)(Status) >= 0)
#endif

// LsaApCallPackageUntrusted is partially missing from NTSecPkg.h
typedef NTSTATUS
(NTAPI LSA_AP_CALL_PACKAGE_UNTRUSTED)(
	__in PLSA_CLIENT_REQUEST ClientRequest,
	__in PVOID ProtocolSubmitBuffer,
	__in PVOID ClientBufferBase,
	__in ULONG SubmitBufferLength,
	__out PVOID* ProtocolReturnBuffer,
	__out PULONG ReturnBufferLength,
	__out PNTSTATUS ProtocolStatus
);

The WIN32_NO_STATUS definition is required to prevent Windows.h from defining the NTSTATUS values. Letting ntstatus.h define them is required because Windows.h only defines a subset of the required values. SSPI.h requires choosing between Win32, Kernel-mode and Macintosh Classic (heh…) usage, resulting in the SECURITY_WIN32 preprocessor definition. The NTSecPkg.h header included with the Windows SDK I’m using makes the embarrassing error of missing the prototype for LsaApCallPackageUntrusted, while curiously attempting to provide the function pointer type for it.

It is wise to avoid loading the original AP DLLs during DllMain, due to potential loader lock issues. The first functions invoked after an AP DLL has been mapped to a process are those that fill the function dispatch table; SpInitialize for the SSP case, SpLsaModeInitialize for the LSASS-side AP case and SpUserModeInitialize for the secur32.dll logon application (Winlogon) side case. All three functions ought to load the original AP if required and perform any required hooking of the dispatch tables, pointing to your own proxy functions.

The initialization functions should also extract the function pointers for the original AP implementations and preserve them in global data. This can later be used to forward calls to the original implementation. Consider the implementation of LsaApCallPackage in my LsaPrxAp test DLL:

NTSTATUS NTAPI LsaApCallPackage(
	__in PLSA_CLIENT_REQUEST ClientRequest,
	__in PVOID ProtocolSubmitBuffer,
	__in PVOID ClientBufferBase,
	__in ULONG SubmitBufferLength,
	__out PVOID* ProtocolReturnBuffer,
	__out PULONG ReturnBufferLength,
	__out PNTSTATUS ProtocolStatus
)
{
	OutputDebugString(TEXT("lsaaprxap LsaApCallPackage\n"));

	return g_LsaPrxApFunctionTable.pLsaApCallPackage(
		ClientRequest, ProtocolSubmitBuffer, ClientBufferBase,
		SubmitBufferLength, ProtocolReturnBuffer, ReturnBufferLength,
		ProtocolStatus);
}

Obviously a real-world proxy AP would perform required processing before or after forwarding to the original function pointer preserved in global data, as well.

Hopefully this primer on implementing an LSA proxy AP will prove to be a welcome addition to Microsoft’s sparse documentation on LSA authentication packages.

Unexporting a function from a DLL at runtime by name obfuscation

Usually, the set of functions exported by a Windows DLL is considered to be immutable. When load-time binding is used, functions appearing in the loading module’s import table are resolved against the export table of the loaded DLL. If any imported functions are missing, the loader aborts the module load attempt and unmaps the modules.

Library clients which require a more flexible reaction to the presence or absence of specific exports opt for a more dynamic binding approach, either late binding via GetProcAddress directly or through the Visual C++ delay loading facility, which allows for setting callbacks for missing modules or exports.

Recently, as I was experimenting with implementing an LSA proxy authentication package (more on that in a followup post) I considered the issue of properly implementing a proxy DLL for a DLL whose set of exports is only partially known to the proxy. Modern LSA APs have a callback function that provides a function dispatch table and do no rely on the module’s export table, but for pedagogic purposes let us consider an hypothetical situation in which the export table is the only lookup apparatus in use by the LSA.

In the case of an LSA authentication package, the package may provide an implementation of either LsaApLogonUser, LsaApLogonUserEx or LsaApLogonUserEx2. An original LSA AP can export one or more of these functions. If we want to create a generic LSA AP proxy DLL, we may wish to have a specific export from our module, loaded in lieu of the original AP, only if the original module also had it. This presents difficulty since we may not be able to predict the specific set of exports during proxy compile time.

If we recognize that DLL exports are in fact a poor man’s native code reflection mechanism, we can adopt a non-traditional approach of dynamic modification of our module’s set of exported functions. Our proxy DLL shall initially (at compile time) export all functions that could potentially be exported by the original DLL, e.g. by a module definition (.DEF) file like this:

LIBRARY "lsaprxap"
EXPORTS
LsaApInitializePackage
LsaApLogonUser
LsaApLogonUserEx
LsaApLogonUserEx2

LsaApCallPackage
LsaApCallPackagePassthrough
LsaApLogonTerminated
LsaApCallPackageUntrusted
SpInitialize
SpInstanceInit
SpLsaModeInitialize
SpUserModeInitialize

Notice how we export all three functions, even though some may not appear in the original DLL. LSA uses late binding via GetProcAddress to decide which function in the AP to call. If we export LsaApLogonUserEx2 and our original DLL does not have that function, we’ll have nothing to do when our proxy function is called (no original to forward to after our own processing). There is no telling what will happen if we return STATUS_NOT_IMPLEMENTED. Besides, LSA AP’s are only an illustration, and in other cases the export might not even have the option to return a failure exit code. Therefore, the behavior we desire is that GetProcAddress for LsaApLogonUserEx2 will fail and return NULL if the original DLL for which we are acting as a proxy does not export LsaApLogonUserEx2 itself.

The names and addresses of exported functions from a DLL appear in the PE image’s export directory. By accessing the export directory, looking up an exported function of interest and removing it we can alter the behavior of GetProcAddress for the exported function at runtime, after the module has been loaded. Note that this alteration is only useful for GetProcAddress invocations from that time forward, and callers that invoked GetProcAddress earlier and cached the result or callers that used load-time binding against our export table already obtained a function pointer to the exported function. Therefore, this technique is only useful in limited circumstances.

The export directory points to a block of null-terminated string pointers, indexed by export ordinal. In order to outright remove an export from the middle of the export table, we’d have to copy the export table aside, remove the desired functions and point the PE header to the new export table. This is a feasible, but cumbersome approach. Instead I opted for an alternative technique – obfuscating the name of the export to prevent GetProcAddress callers from resolving it by its well-known name. The function is still exported, but its name is unknown to other callers. This is probably sufficient for the vast majority of cases. As for the obfuscation itself, in this illustration we’ll merely increment the character value of the first letter in the export:

// Set by the linker to the base address of the module.
EXTERN_C IMAGE_DOS_HEADER __ImageBase;

void UnexportFunction(LPSTR ExportName)
{
	IMAGE_DOS_HEADER* dosHeader = &__ImageBase;
	assert(dosHeader->e_magic == IMAGE_DOS_SIGNATURE);
	IMAGE_NT_HEADERS* ntHeaders = reinterpret_cast<IMAGE_NT_HEADERS*>(
		reinterpret_cast<BYTE*>(dosHeader) + dosHeader->e_lfanew);
	assert(ntHeaders->Signature == 0x00004550);
	IMAGE_OPTIONAL_HEADER* optionalHeader = &ntHeaders->OptionalHeader;
	IMAGE_DATA_DIRECTORY* exportDataDirectory =
		&optionalHeader->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT];
	IMAGE_EXPORT_DIRECTORY* exportDirectory = reinterpret_cast<IMAGE_EXPORT_DIRECTORY*>(
		reinterpret_cast<BYTE*>(dosHeader) + exportDataDirectory->VirtualAddress);

	ULONG* addressOfNames = reinterpret_cast<ULONG*>(
		reinterpret_cast<BYTE*>(dosHeader) + exportDirectory->AddressOfNames);
	for (DWORD i = 0; i < exportDirectory->NumberOfFunctions; i++)
	{
		LPSTR exportFunctionName = reinterpret_cast<LPSTR>(
			reinterpret_cast<BYTE*>(dosHeader) + addressOfNames[i]);
		if (strcmp(exportFunctionName, ExportName) == 0)
		{
			DWORD oldProtect = 0;
			BOOL rc = VirtualProtect(
				exportFunctionName,
				strlen(exportFunctionName),
				PAGE_READWRITE,
				&oldProtect);
			if (rc == FALSE)
			{
				OutputDebugString(TEXT("VirtualProtect failed.\n"));
			}
			exportFunctionName[0]++;
			break;
		}
	}
}

The sample UnexportFunction function iterates over the current module’s export table until the function of interest is encountered. Since the export table is mapped as read-only memory, VirtualProtect must be used to allow for its modification. The string containing the name of the exported function is modified in place as a trivial obfuscation. This is sufficient to result in the “unexporting” of the symbol:

FARPROC before = GetProcAddress((HMODULE)&__ImageBase, "LsaApLogonUser");
UnexportFunction("LsaApLogonUser");
FARPROC after = GetProcAddress((HMODULE)&__ImageBase, "LsaApLogonUser");
assert(before != after);
assert(after == NULL);

With this tool of export entry removal at our disposal, we can devise an architecture in which the proxy DLL contains all exports that are feasible to be present in the original DLL at runtime. Through “reflection” of the original DLL, the proxy shall determine which redundant exports it wishes to hide, resulting it runtime behavior consistent with that of the original DLL (again, assuming only late binding with GetProcAddress is used).

What if we are unable to construct a superset of all possible exports from the original DLL? Perhaps we wish to be future-proof as new, unknown exports are added to the original DLL. For that a variety of solutions may be considered. In particular, copying the original DLL’s export table to our own and inserting hooks as needed comes to mind. Perhaps a topic for a future post…

Deploying the Visual C++ libraries with an NSIS installer

Beginning with Visual C++ 2005 and continuing into Visual C++ 2008 and the foreseeable future, Microsoft’s various runtime libraries (CRT, ATL, MFC, etc.) are no longer installed into the system32 directory on Windows XP and later, but are rather “side-by-side assemblies” that need to be installed into the side-by-side store, “WinSxS”, in order to be available to all applications.

I’ve discussed the SxS store and the API Microsoft has documented for managing it in a previous post. Nevertheless, at the request of the NSIS maintainer, kichik, I’ll provide some guidance on the issue of runtime deployment and concrete examples to authors of NSIS-based installations. Do keep in mind that I am not adept at authoring NSIS installers and questions beyond the realm of the matter at hand are best targeted at the NSIS forum.

Unlike in the Linux world, where the C runtime library is considered an operating system component and versions of it are never installed by applications (at worst, some proprietary application is linked against an antique version of libc and requires the system administrator to install a compatibility package provided by the distribution), the CRT situation on Windows is more complicated. In the days of yore, Windows NT provided the now long defunct CRTDLL.DLL. Later, the newer variant MSVCRT.DLL shipped with Visual C++, going into the 6.0 release. However, in addition to serving as the runtime of a specific Visual C++ version, MSVCRT.DLL doubles as the “OS CRT”, the version of the C++ runtime deployed with the OS as far back as NT 4.0 and going into Windows Vista. Components included with the operating system itself, such as Notepad and Calculator, are linked against this CRT dynamically. Do not let the identical moniker fool you, the CRTs included with the various NT releases diverge significantly, sporting, for example, a brand new exception handling runtime in Windows Vista, aligned with newer Visual C++ compilers.

The existence of several MSVCRT.DLL variants and the associated versioning issues are probably what led Microsoft to adopt a policy of strongly versioned CRTs beginning with the Visual C++ .NET (2002) release. MSVCR70.DLL was the runtime required by the output of that product, and later versions would require deploying MSVCR71.DLL, MSVCR80.DLL and most recently MSVCR90.DLL. In addition to the CRT itself, there are also the various peripheral libraries that some applications may depend on, such as ATL and MFC.

I’ve discussed in the past an approach utilizing the Windows Driver Kit build environment that allows combining a modern C++ compiler with targeting the Visual C++ 6.0 / OS CRT, MSVCRT.DLL. It is for the brave who don’t mind getting their hands dirty and whose desire to target the broadly deployed runtime exceeds the fear of the plethora of potential version compatibility issues such an application configuration can cause.

For the more conservative lot, the question remains, how do I get the new C++ runtimes to my end-user’s machine? The first approach is that of utilizing static linking. It should be avoided at nearly all cost as it results in both obese executables that are unable to share the runtime’s memory pages with other running processes and is completely unservicable by Microsoft when a security update or another bug fix to all users of the runtime libraries needs to be broadly deployed.

We therefore turn our attention to approaches based on dynamic linking. First of all, the reader should review the official guidance provided by the Visual C++ team on the matter, although he or she may not like what they read. To summarize, Microsoft officially supports the following deployment methods:

  • Use an MSI-based (Windows Installer) installation procedure and utilize their MSM merge modules to include whatever runtime components you require with your application. The MSMs are black box magic that will get those runtime libraries into the “winsxs” store without asking too many questions. If you don’t like those massively complicated MSI installers and the WiX XML schemas make your head spin, that’s too bad.
  • Use the obese VCRedist.exe for the target architecture, without the benefit of picking and choosing only those runtime components that are of interest for your specific application.
  • Deploy the runtime libraries as files in your application’s directory, or “private assemblies” in SxS nomenclature, wasting the end-user’s hard disk space with multiple copies. This is not as bad as it seems, since at least SxS redirection policies can make an updated, security patched version from the “winsxs” store be loaded in place of out of date version deployed privately with the application, unlike with classic non-SxS local DLLs or with static linking.

As the popularity of NSIS as an installation apparatus shows, not everyone are willing to be strong-armed into an MSI-based installation just yet. So how do CRT deploying installers address this acute issue? I was disappointed, but not surprised, to see that VLC, DivX and various other applications with NSIS-based installers, opt for the “private assembly” approach, simplifying life for the installation author but needlessly wasting end-user disk real estate.

The now documented SxS API provides an alternative approach, presumably supported by Microsoft for deploying SxS assemblies in general (such as your own) but not specifically by the Visual C++ folks for theirs. The motivation for this lack of support is unclear, since the end result is as servicable by them as is using Windows Installer merge modules. Nevertheless, it is something that those who follow this path should be aware of.

OK, so let’s get on with it. Unlike with system32, we can’t just waltz into winsxs and drop our assembly’s files there. The directory structure is complicated, differs between XP and Vista, and in fact the ACL on the directory in Vista won’t allow anyone but TrustedInstaller (i.e., MSI) to touch it. Therefore we are required to perform the installation through the SxS API, which provides a COM-based interface for manipulating the store.

For illustration purposes, I shall use the Visual C++ 2005 (8.0) Debug CRT. Note that this is not the CRT you want to deploy to your end users, and in any case is explicitly NOT redistributable by Microsoft’s license terms. I use it for illustrative convenience since my XP virtual machine doesn’t have this assembly. We’ll use an NSIS installer script to drive the wonderful though peculiar System plug-in and get it to invoke the SxS API. Note that elaborate error handling is omitted for brevity. So here we go:


Name "NSIS SxS Test"
OutFile "nsissxs.exe"
SetPluginUnload alwaysoff
ShowInstDetails show
XPStyle on
SetCompressor /SOLID lzma
InstallDir $PROGRAMFILES\NSISSxS


!define FUSION_REFCOUNT_UNINSTALL_SUBKEY_GUID {8cedc215-ac4b-488b-93c0-a50a49cb2fb8}

Section "Uninstall"
DeleteRegKey "HKLM" "Software\Microsoft\Windows\CurrentVersion\Uninstall\nsissxs"
Delete $INSTDIR\uninst.exe
Delete $INSTDIR\dummy.txt
RMDir $INSTDIR
DetailPrint "Removing DebugCRT assembly..."
System::Call "sxs::CreateAssemblyCache(*i .r0, i 0) i.r1"
StrCmp $1 0 0 fail
System::Call "*(i 32, i 0, i 2364391957, i 1217113163, i 178634899, i 3090139977, w 'nsissxs', w '') i.s"
Pop $2
System::Call "$0->3(i 0, w 'Microsoft.VC80.DebugCRT,version=$\"8.0.50727.762$\",type=$\"win32$\",processorArchitecture=$\"x86$\",publicKeyToken=$\"1fc8b3b9a1e18e3b$\"', i r2, *i . r3) i.r1"
StrCmp $1 0 0 fail2
DetailPrint "Disposition returned is $3"
System::Call "$0->2()"
Goto end
fail:
DetailPrint "CreateAssemblyCache failed."
DetailPrint $1
Goto end
fail2:
DetailPrint "UninstallAssembly failed."
DetailPrint $1
Goto end
end:
SectionEnd

Section
SetOutPath $INSTDIR
File "dummy.txt"
WriteUninstaller $INSTDIR\uninst.exe
WriteRegStr "HKLM" "Software\Microsoft\Windows\CurrentVersion\Uninstall\nsissxs" "DisplayName" "NSIS SxS Test"
WriteRegStr "HKLM" "Software\Microsoft\Windows\CurrentVersion\Uninstall\nsissxs" "UninstallString" "$INSTDIR\uninst.exe"
InitPluginsDir
SetOutPath $PLUGINSDIR
File "msvcm80d.dll"
File "msvcp80d.dll"
File "msvcr80d.dll"
File "x86_Microsoft.VC80.DebugCRT_1fc8b3b9a1e18e3b_8.0.50727.762_x-ww_5490cd9f.cat"
File "x86_Microsoft.VC80.DebugCRT_1fc8b3b9a1e18e3b_8.0.50727.762_x-ww_5490cd9f.manifest"

DetailPrint "Installing DebugCRT assembly..."
System::Call "sxs::CreateAssemblyCache(*i .r0, i 0) i.r1"
StrCmp $1 0 0 fail
# Fill a FUSION_INSTALL_REFERENCE.
# fir.cbSize = sizeof(FUSION_INSTALL_REFERENCE) == 32
# fir.dwFlags = 0
# fir.guidScheme = FUSION_REFCOUNT_UNINSTALL_SUBKEY_GUID
# fir.szIdentifier = "nsissxs"
# fir.szNonCanonicalData = 0
System::Call "*(i 32, i 0, i 2364391957, i 1217113163, i 178634899, i 3090139977, w 'nsissxs', w '') i.s"
Pop $2
# IAssemblyCache::InstallAssembly(0, manifestPath, fir)
System::Call "$0->7(i 0, w '$PLUGINSDIR\x86_Microsoft.VC80.DebugCRT_1fc8b3b9a1e18e3b_8.0.50727.762_x-ww_5490cd9f.manifest', i r2) i.r1"
System::Free $2
StrCmp $1 0 0 fail2
System::Call "$0->2()"
Goto end
fail:
DetailPrint "CreateAssemblyCache failed."
DetailPrint $1
Goto end
fail2:
DetailPrint "InstallAssembly failed."
DetailPrint $1
Goto end
end:
SectionEnd

If you are not familiar with NSIS script syntax, now would be a good time to get acquainted. Let us review the contents of the 2nd section, which is the install section. The dummy file is a placeholder for the actual files your installer wants to deploy. Next, we set up an Uninstall entry in the registry as one usually would. Now on to the interesting part.

In order to deploy a SxS assembly, we must place its DLLs together in a temporary directory created by the installer. Note that if an assembly contains several DLLs, we cannot pick and choose only those that our application links with. The assembly is deployed, versioned and bound as a whole. We can figure out which files are part of the assembly in question by reviewing the assembly manifest, which we’ll find installed into the Manifests subdirectory of the WinSxS store on a Windows XP system. If we review the Debug CRT’s manifest, we can see <file> nodes under the <assembly> node, each referencing one of the files that must be deployed with the assembly. You can find the actual assembly files under the subdirectory with the assembly’s strong name in the WinSxS store.

In addition to the DLL files themselves, the assembly manifest and the assembly signing catalog are an integral part of the assembly. The catalog ensures the integrity of the assembly and is a welcome feature over traditional DLL deployment.

With the DLLs, assembly manifest and catalog in place, we are ready to invoke the SxS API for assembly installation. First, we call CreateAssemblyCache to retrieve the IAssemblyCache interface for managing the SxS store. Note that in the context of an NSIS installer, COM has already been initialized (for STA use) at this point, but if you are making a custom installer in another environment you may have to take care of that before reaching this point.

Assuming all goes well the next phase is setting up the FUSION_INSTALL_REFERENCE structure that will describe our assembly installation. Typically, you’ll want the reference to be associated with the registry Uninstall key for your application. Besides, other reference types do not seem to work too well and the documentation doesn’t err on the side of verbosity.

The not-so-seasoned NSIS scripter that I am, I couldn’t figure out a more legible way to specify the GUID argument to the InstallAssembly invocation so I broke down its components by hand. Counting vtable indices including the IUnknown and IAssemblyCache interfaces, InstallAssembly is at vtable slot 7. After the install reference structure is set up and the method invoked, we hope for the best.

Assuming a successful install transaction, we proceed to call the IUnknown method Release (vtable slot 2) to free the SxS cache manager and deem our install sequence completed.

We now turn our attention to the reverse sequence in the Uninstall section of the illustration installer. Being good citizens of the Windows ecosystem, we remove our reference to the shared assembly when the end-user removes our application from their system. WinSxS manages assembly reference counting and will figure out whether the assembly files should actually be removed from the disk.

We create an IAssemblyCache interface instance as before but this time call UninstallAssembly to remove our reference. This is the first method of the interface but is preceded by the IUnknown members and is thus at vtable slot 3. Following a successful invocation we can examine the returned Disposition value if it is of interest and proceed to free the instance.

Note that we remove an assembly by its full, strong name and not by path. You can figure out the assembly’s strong name from its manifest.

OK, installing the VC8 Debug CRT was easy enough. Note that other libraries (ATL, etc.) you’ll want to install may have dependencies on other assemblies, so make sure you get your install sequence in order.

Installing the Visual C++ 2005 runtime is nice and all, but somehow it just feels wrong installing obsolete software. I turned my attention to the Visual C++ 2008 libraries, encountering disappointing results.

I gave it a few shots but installing the Visual C++ 2008 Debug CRT always fails, InstallAssembly promptly returning an HRESULT containing ERROR_SXS_PROTECTION_CATALOG_NOT_VALID. A malformed catalog? In one of Microsoft’s very own assemblies? Say it ain’t so!

If you have the hots for deploying the newer runtime, you’ll have to figure out that one on your own, folks. I made sure the catalog for the Visual C++ 9.0 Debug CRT I picked up from the WinSxS store matches the same catalog file found in the MSI merge module (MSM) at C:\Program Files\Common Files\Merge Modules\Microsoft_VC90_DebugCRT_x86.msm by extracting the MSM’s files with the useful MSIX extractor. The catalog files matched and regardless, the SHA-1 hashes for the assembly files matched the catalog rejected by InstallAssembly. Mysterious.

Reviewing the Windows event log following this error didn’t help too much. The System log was now decorated with SideBySide Event ID 20, stating: “The manifest C:\WINDOWS\WinSxS\InstallTemp\160585\Manifests\x86_Microsoft.VC90.DebugCRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_597c3456.Manifest does not match its source catalog or the catalog is missing.” … No newsflash there.

I figured I’ll stick to VC8 support and leave the VC9 troubleshooting for later. If push comes to shove, one can always figure out how to write WiX installers. :)

Instead I opted to review whether this NSIS-based installation approach is compatible with Windows Vista. I was worried that with the restrictive ACLs on the WinSxS store in that OS, without being an MSI and running from the context of the mighty TrustedInstaller.exe process, the installation will surely fail.

I was therefore positively surprised when the test installer worked on Vista. This surprised me since I knew the elevated installer executable ran as Administrator, but there was no denying I was not supposed to be able to copy files to WinSxS:

C:\Users\User\Desktop>cacls C:\Windows\WinSxS
C:\Windows\winsxs NT SERVICE\TrustedInstaller:(OI)(CI)F
BUILTIN\Administrators:(OI)(CI)R
NT AUTHORITY\SYSTEM:(OI)(CI)R
BUILTIN\Users:(OI)(CI)R

There was no denying it. Users and Administrators alike have read-only access to the store, and only the TrustedInstaller service can actually modify it. I opted to run the installer once again, this time in Windbg, tracing the operation of the SxS API to figure out what was happening behind the scenes.


0:000> sxe ld:sxs
0:000> g
ModLoad: 75500000 7555f000 C:\Windows\system32\sxs.dll
eax=1000162a ebx=00000000 ecx=15bf8bb6 edx=00000007 esi=7ffdd000 edi=20000000
eip=76f99a94 esp=0278f620 ebp=0278f664 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCallRet:
76f99a94 c3 ret
0:002> bp sxs!CreateAssemblyCache
0:002> g
Breakpoint 0 hit
eax=00000000 ebx=002ceda8 ecx=002cedc8 edx=002ce990 esi=002ceda8 edi=00000000
eip=7554a3aa esp=0278fd54 ebp=0278fd6c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
sxs!CreateAssemblyCache:
7554a3aa 8bff mov edi,edi
0:002> kb 2
*** WARNING: Unable to verify checksum for C:\Users\User\AppData\Local\Temp\nspBB71.tmp\System.dll
ChildEBP RetAddr Args to Child
0278fd50 100024b5 00291a98 00000000 1000162a sxs!CreateAssemblyCache
0278fd6c 1000168d 002ceda8 00000000 75bbc780 System+0x24b5

We know from MSDN that CreateAssemblyCache returns the IAssemblyCache pointer through the first, out parameter. We expect a well-behaved caller to pass in storage initialized to zero, and the storage to contain the newly instantiated interface after the function returns:

0:002> dps 00291a98 L1
00291a98 00000000
0:002> gu
eax=00000000 ebx=002ceda8 ecx=00000000 edx=00000008 esi=002ceda8 edi=00000000
eip=100024b5 esp=0278fd60 ebp=0278fd6c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
System+0x24b5:
100024b5 a340400010 mov dword ptr [System+0x4040 (10004040)],eax ds:0023:10004040={sxs!CreateAssemblyCache (7554a3aa)}
0:002> dps 00291a98 L1
00291a98 00291c88

The first pointer value in the storage reached when following a dereferenced interface pointer is a member function vtable. We verify this and follow the vtable to examine where the implementations of the various members reside:

0:002> dps 00291c88 L1
00291c88 755036f0 sxs!CAssemblyCache::`vftable'
0:002> dps 755036f0 L10
755036f0 75549ac4 sxs!CAssemblyCache::QueryInterface
755036f4 7550de6b sxs!CAssemblyCache::AddRef
755036f8 7554a355 sxs!CAssemblyCache::Release
755036fc 75549d35 sxs!CAssemblyCache::UninstallAssembly
75503700 75549b15 sxs!CAssemblyCache::QueryAssemblyInfo
75503704 7554a219 sxs!CAssemblyCache::CreateAssemblyCacheItem
75503708 755542f1 sxs!XMLParser::SetFlags
7550370c 75549e91 sxs!CAssemblyCache::InstallAssembly
75503710 7554a4d4 sxs!CAssemblyName::QueryInterface
75503714 7550de6b sxs!CAssemblyCache::AddRef
75503718 7554ad69 sxs!CAssemblyName::Release
7550371c 7554a525 sxs!CAssemblyName::SetProperty
75503720 7554a64d sxs!CAssemblyName::GetProperty
75503724 7554a4ba sxs!CAssemblyName::Finalize
75503728 7554aaac sxs!CAssemblyName::GetDisplayName
7550372c 7554a4ad sxs!CAssemblyName::Reserved

It is clear the implementation of the InstallAssembly method is sxs!CAssemblyCache::InstallAssembly. We set up a breakpoint, proceed there and perform a high-level trace:

0:002> bp sxs!CAssemblyCache::InstallAssembly
0:002> g
Breakpoint 1 hit
eax=00000000 ebx=002d2668 ecx=002d2688 edx=002cdc10 esi=002d2668 edi=00000000
eip=75549e91 esp=0278fd4c ebp=0278fd6c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
sxs!CAssemblyCache::InstallAssembly:
75549e91 8bff mov edi,edi
0:002> wt -l 2 -m sxs
Tracing sxs!CAssemblyCache::InstallAssembly to return address 100024b5
16 0 [ 0] sxs!CAssemblyCache::InstallAssembly
13 0 [ 1] sxs!CFrame::CFrame
20 13 [ 0] sxs!CAssemblyCache::InstallAssembly
3 0 [ 1] sxs!CFrame::BaseEnter
11 0 [ 2] sxs!FusionpRtlPushFrame
6 11 [ 1] sxs!CFrame::BaseEnter
54 30 [ 0] sxs!CAssemblyCache::InstallAssembly
6 0 [ 1] sxs!CFrame::ClearLastError
58 36 [ 0] sxs!CAssemblyCache::InstallAssembly
10 0 [ 1] sxs!SxspTranslateReferenceFrom
13 0 [ 2] sxs!CFrame::CFrame
14 13 [ 1] sxs!SxspTranslateReferenceFrom
6 0 [ 2] sxs!CFrame::BaseEnter
70 19 [ 1] sxs!SxspTranslateReferenceFrom
9 0 [ 2] sxs!CFnTracerWin32::~CFnTracerWin32
74 28 [ 1] sxs!SxspTranslateReferenceFrom
63 138 [ 0] sxs!CAssemblyCache::InstallAssembly
4 0 [ 1] sxs!CFrame::ClearLastError
66 142 [ 0] sxs!CAssemblyCache::InstallAssembly
16 0 [ 1] sxs!SxsInstallW
13 0 [ 2] sxs!CFrame::CFrame
20 13 [ 1] sxs!SxsInstallW
6 0 [ 2] sxs!CFrame::BaseEnter
25 19 [ 1] sxs!SxsInstallW
13 0 [ 2] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
28 32 [ 1] sxs!SxsInstallW
13 0 [ 2] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
30 45 [ 1] sxs!SxsInstallW
13 0 [ 2] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
68 58 [ 1] sxs!SxsInstallW
4 0 [ 2] sxs!CFrame::ClearLastError
713 62 [ 1] sxs!SxsInstallW
72 0 [ 2] sxs!SxspExpandRelativePathToFull
797 134 [ 1] sxs!SxsInstallW
4 0 [ 2] sxs!CFrame::ClearLastError
800 138 [ 1] sxs!SxsInstallW
ModLoad: 741e0000 741ea000 C:\Windows\system32\sxsstore.dll
eax=ffffffff ebx=00000000 ecx=002e6a60 edx=00000001 esi=7ffdd000 edi=20000000
eip=76f99a94 esp=0278ed48 ebp=0278ed8c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCallRet:
76f99a94 c3 ret

OK, so it looks like this implementation lets sxs!SxsInstallW do the actual work. We rerun the installer and this time perform a trace from that point:

0:000> sxe ld:sxs
0:000> g
ModLoad: 75500000 7555f000 C:\Windows\system32\sxs.dll
eax=1000162a ebx=00000000 ecx=0f78c21d edx=00000007 esi=7ffdc000 edi=20000000
eip=76f99a94 esp=026ef620 ebp=026ef664 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCallRet:
76f99a94 c3 ret
0:002> bp sxs!SxsInstallW
0:002> g
Breakpoint 0 hit
eax=026efce8 ebx=0032db90 ecx=026efd08 edx=0032dbac esi=00332a08 edi=026efd44
eip=755475ad esp=026efcd4 ebp=026efd48 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
sxs!SxsInstallW:
755475ad 8bff mov edi,edi
0:002> wt -l 2 -m sxs
Tracing sxs!SxsInstallW to return address 75549f41
16 0 [ 0] sxs!SxsInstallW
13 0 [ 1] sxs!CFrame::CFrame
20 13 [ 0] sxs!SxsInstallW
3 0 [ 1] sxs!CFrame::BaseEnter
11 0 [ 2] sxs!FusionpRtlPushFrame
6 11 [ 1] sxs!CFrame::BaseEnter
25 30 [ 0] sxs!SxsInstallW
10 0 [ 1] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
12 0 [ 2] sxs!CGenericBaseStringBuffer::InitializeInlineBuffer
13 12 [ 1] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
28 55 [ 0] sxs!SxsInstallW
10 0 [ 1] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
12 0 [ 2] sxs!CGenericBaseStringBuffer::InitializeInlineBuffer
13 12 [ 1] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
30 80 [ 0] sxs!SxsInstallW
10 0 [ 1] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
12 0 [ 2] sxs!CGenericBaseStringBuffer::InitializeInlineBuffer
13 12 [ 1] sxs!CGenericStringBuffer<64,CUnicodeCharTraits>::CGenericStringBuffer<64,CUnicodeCharTraits>
68 105 [ 0] sxs!SxsInstallW
4 0 [ 1] sxs!CFrame::ClearLastError
713 109 [ 0] sxs!SxsInstallW
15 0 [ 1] sxs!SxspExpandRelativePathToFull
13 0 [ 2] sxs!CFrame::CFrame
19 13 [ 1] sxs!SxspExpandRelativePathToFull
6 0 [ 2] sxs!CFrame::BaseEnter
22 19 [ 1] sxs!SxspExpandRelativePathToFull
29 0 [ 2] sxs!CGenericStringBufferAccessor::Attach
23 48 [ 1] sxs!SxspExpandRelativePathToFull
4 0 [ 2] sxs!CFrame::ClearLastError
29 52 [ 1] sxs!SxspExpandRelativePathToFull
13 0 [ 2] kernel32!GetFullPathNameW
36 65 [ 1] sxs!SxspExpandRelativePathToFull
31 0 [ 2] sxs!CGenericStringBufferAccessor::Detach
37 96 [ 1] sxs!SxspExpandRelativePathToFull
4 0 [ 2] sxs!CFrame::ClearLastError
41 100 [ 1] sxs!SxspExpandRelativePathToFull
68 0 [ 2] sxs!CGenericBaseStringBuffer::Win32ResizeBuffer
46 168 [ 1] sxs!SxspExpandRelativePathToFull
29 0 [ 2] sxs!CGenericStringBufferAccessor::Attach
47 197 [ 1] sxs!SxspExpandRelativePathToFull
4 0 [ 2] sxs!CFrame::ClearLastError
52 201 [ 1] sxs!SxspExpandRelativePathToFull
13 0 [ 2] kernel32!GetFullPathNameW
64 214 [ 1] sxs!SxspExpandRelativePathToFull
9 0 [ 2] sxs!CFnTracerWin32::~CFnTracerWin32
66 223 [ 1] sxs!SxspExpandRelativePathToFull
20 0 [ 2] sxs!CGenericStringBufferAccessor::~CGenericStringBufferAccessor
72 243 [ 1] sxs!SxspExpandRelativePathToFull
797 424 [ 0] sxs!SxsInstallW
4 0 [ 1] sxs!CFrame::ClearLastError
800 428 [ 0] sxs!SxsInstallW
28 0 [ 1] sxs!SxspGetRemoteStore
5 0 [ 2] sxs!SxspEnsureUserIsAdmin
34 5 [ 1] sxs!SxspGetRemoteStore
9 0 [ 2] kernel32!LoadLibraryW
40 14 [ 1] sxs!SxspGetRemoteStore
18 0 [ 2] ShimEng!StubGetProcAddress
50 32 [ 1] sxs!SxspGetRemoteStore
24 0 [ 2] ole32!CoCreateInstance
>> No match on ret
24 0 [ 2] ole32!CoCreateInstance
5 0 [ 2] RPCRT4!NdrpGetRpcHelper
>> No match on ret
5 0 [ 2] RPCRT4!NdrpGetRpcHelper
17 0 [ 2] RPCRT4!NdrpGetIIDFromBuffer
>> No match on ret
17 0 [ 2] RPCRT4!NdrpGetIIDFromBuffer
71 0 [ 2] RPCRT4!NdrpInterfacePointerUnmarshall
>> No match on ret
71 0 [ 2] RPCRT4!NdrpInterfacePointerUnmarshall
11 0 [ 2] RPCRT4!NdrpPointerUnmarshall
>> No match on ret
11 0 [ 2] RPCRT4!NdrpPointerUnmarshall
4 0 [ 2] RPCRT4!NdrPointerUnmarshall
>> No match on ret
4 0 [ 2] RPCRT4!NdrPointerUnmarshall
19 0 [ 2] RPCRT4!NdrpPointerUnmarshall
>> No match on ret
19 0 [ 2] RPCRT4!NdrpPointerUnmarshall
4 0 [ 2] RPCRT4!NdrPointerUnmarshall
>> No match on ret
4 0 [ 2] RPCRT4!NdrPointerUnmarshall
65 0 [ 2] RPCRT4!NdrpClientUnMarshal
>> No match on ret
65 0 [ 2] RPCRT4!NdrpClientUnMarshal
16 0 [ 2] RPCRT4!NdrClientCall2
>> No match on ret
16 0 [ 2] RPCRT4!NdrClientCall2
8 0 [ 2] RPCRT4!ObjectStublessClient
>> No match on ret
8 0 [ 2] RPCRT4!ObjectStublessClient
4 0 [ 2] RPCRT4!ObjectStubless
65 0 [ 2] ole32!CRpcResolver::CreateInstance
>> No match on ret
65 0 [ 2] ole32!CRpcResolver::CreateInstance
10 0 [ 2] ole32!CClientContextActivator::CreateInstance
>> No match on ret
10 0 [ 2] ole32!CClientContextActivator::CreateInstance
8 0 [ 2] ole32!ActivationPropertiesIn::DelegateCreateInstance
>> No match on ret
8 0 [ 2] ole32!ActivationPropertiesIn::DelegateCreateInstance
53 0 [ 2] ole32!ICoCreateInstanceEx
>> No match on ret
53 0 [ 2] ole32!ICoCreateInstanceEx
21 0 [ 2] ole32!CComActivator::DoCreateInstance
>> No match on ret
21 0 [ 2] ole32!CComActivator::DoCreateInstance
2 0 [ 2] ole32!CoCreateInstanceEx
>> No match on ret
2 0 [ 2] ole32!CoCreateInstanceEx
5 0 [ 2] ole32!CoCreateInstance
62 444 [ 1] sxs!SxspGetRemoteStore
34 0 [ 2] ole32!CStdIdentity::CInternalUnk::QueryInterface
>> No match on ret
34 0 [ 2] ole32!CStdIdentity::CInternalUnk::QueryInterface
14 0 [ 2] ole32!CreateIdentityHandler
>> No match on ret
14 0 [ 2] ole32!CreateIdentityHandler
21 0 [ 2] ole32!UnmarshalInternalObjRef
>> No match on ret
21 0 [ 2] ole32!UnmarshalInternalObjRef
27 0 [ 2] ole32!OXIDEntry::UnmarshalRemUnk
>> No match on ret
27 0 [ 2] ole32!OXIDEntry::UnmarshalRemUnk
18 0 [ 2] ole32!OXIDEntry::MakeRemUnk
>> No match on ret
18 0 [ 2] ole32!OXIDEntry::MakeRemUnk
7 0 [ 2] ole32!OXIDEntry::GetRemUnk
>> No match on ret
7 0 [ 2] ole32!OXIDEntry::GetRemUnk
2 0 [ 2] ole32!CStdMarshal::GetSecureRemUnk
>> No match on ret
2 0 [ 2] ole32!CStdMarshal::GetSecureRemUnk
23 0 [ 2] ole32!CStdMarshal::Begin_RemQIAndUnmarshal1
>> No match on ret
23 0 [ 2] ole32!CStdMarshal::Begin_RemQIAndUnmarshal1
5 0 [ 2] ole32!CStdMarshal::Begin_QueryRemoteInterfaces
>> No match on ret
5 0 [ 2] ole32!CStdMarshal::Begin_QueryRemoteInterfaces
ModLoad: 741e0000 741ea000 C:\Windows\system32\sxsstore.dll

Woah. That’s verbose. There’s a lot of noise in this trace, but the SxspGetRemoteStore function draws attention and it is obvious from all the OLE32 invocations later on that COM is at work here. Examining the sxs!SxspGetRemoteStore function reveals it instantiates the COM object identified by CLSID_SxsStore (left as an exercise for the reader).

Let’s have a look at the object’s registration information. First, extract the CLSID:

0:002> x sxs!CLSID_SxsStore
7554c454 sxs!CLSID_SxsStore =
0:002> dt nt!_GUID 7554c454
ntdll!_GUID
{3c6859ce-230b-48a4-be6c-932c0c202048}
+0x000 Data1 : 0x3c6859ce
+0x004 Data2 : 0x230b
+0x006 Data3 : 0x48a4
+0x008 Data4 : [8] "???"

Now, we’ll use the command-line to see what’s special about this object’s registration:

C:\Users\User\Desktop>reg query HKCR\CLSID\{3c6859ce-230b-48a4-be6c-932c0c202048
} /s


HKEY_CLASSES_ROOT\CLSID\{3c6859ce-230b-48a4-be6c-932c0c202048}
(Default) REG_SZ Sxs Store Class
AppID REG_SZ {752073A2-23F2-4396-85F0-8FDB879ED0ED}

HKEY_CLASSES_ROOT\CLSID\{3c6859ce-230b-48a4-be6c-932c0c202048}\LocalServer32
(Default) REG_EXPAND_SZ %systemroot%\servicing\TrustedInstaller.exe
ThreadingModel REG_SZ Both

OK, that explains it. The SxS API asks an out-of-process COM server running in the context of the TrustedInstaller service to do its bidding, explaining how things work despite the restrictive ACL on the store.

Hope you enjoyed that digression, but now back to the original business at hand. The installation process works just fine on Vista, but the plot thickens when we examine the uninstallation process.

Strangely and contrary to documentation, UninstallAssembly always returns success but with a disposition value of 0 on Vista, and the assembly files remain in place in the WinSxS store no matter what. The bottom line – if you use this approach to deploy the libraries to a Vista system, you may leave behind unused assembly files after your application is uninstalled, cluttering the user’s system. Take this to heart when considering whether this approach and the avoidance of an MSI installer is appropriate for your scenario.

Both the issue of Visual C++ 9.0 assembly deployment using the SxS API and the weird referencing behavior encounter during assembly uninstallation on Vista remain, as of yet, unresolved issues. If anyone is game for figuring those out, I’d be glad to hear about it.

A lightweight approach for exposing C++ objects to a hosted Active Scripting engine

Microsoft’s Active Scripting architecture allows application developers to host the same implementations of the JScript and VBScript scripting languages used by Internet Explorer for scripts in HTML pages, Active Server Pages (the old, pre-.NET implementation) for server-side dynamic content or the Windows Scripting Host for independent scripts. Additionally, third party scripting engines can and have been developed, for Python, Perl and other interpreted languages.

Hosting a scripting engine involves implementing the IActiveScriptSite interface, providing a method to pass script code to the IActiveScript and IActiveScriptParse interfaces and is extensively documented in the literature. Therefore, I shall not discuss the mechanics of hosting itself and will elaborate only on the topic of exposing objects from the host to the engine.

Enabling scripting in your application only adds value over the external Windows Script Host if you expose unique, internal application functionality to the hosted scripts. If your application already exposes its functionality as COM automation objects to automation controllers that can be used out-of-process, there isn’t much point in hosting. However, if your application is document-oriented, for example, providing scripts with access to the document context can be very useful to your users.

A scripting host can make its object model available to hosted scripts by providing the engine with an IDispatch interface for each object it wants to make available. This interface is the foundation of OLE automation and is used by the scripting languages for late binding.

Since the IDispatch interface is basically a rather raw reflection mechanism, implementing it from scratch for a moderately complex object is tedious and error-prone.

If your application already implements COM objects regardless of scripting, it probably already makes use of a framework for doing so, be it ATL, MFC or the CLR. In that case, you have already paid the framework tax and implementing another interface is no challenge. Specifically, ATL offers the convenient IDispatchImpl class for implementing dual interfaces while the CLR makes it ridiculously simple to implement dispatch interfaces (by default, a .NET class is also a COM dispatch object).

However, a dependency on the CLR might not be a welcome requirement. Similarly, complicating a substantial existing code base with the tedium of COM class registration is an adventure that may not be suitable for the faint of heart. If you do not wish to expose automation objects to external clients like WSH, you have no need or desire to modify the registry and maintain that information across installations, uninstallations, upgrades and the like.

However, both ATL and MFC do not go to any reasonable lengths to facilitate the implementation of internal, unregistered COM objects. The IDispatchImpl class requires that you provide it with type information for your dual interface, but ATL’s only ITypeInfo wrapper, CComTypeInfoHolder, is oriented towards retrieving that from a type library residing in a file, either an independent .TLB or an embedded resource in your .EXE or .DLL file. This means that for exposing an object, you need to describe it in IDL, have your build process generate a .TLB for it with MIDL and possibly embed it as a resource using RC. At run time, you need to take care of the logistics of interface and type library registration. All of this for what you only want as internal functionality.

Apart from being tedious, that approach is also characterized by being rigid and static. Manipulating your exposed objects by making runtime decisions that could change the type information does not go well with them being static embedded resource entities.

I considered what would it to take to come up with binary type information from a source that isn’t a file or a resource. At first glance, the LoadTypeLib API is definitely file-oriented. However, a light bulb turned on in my head when I noticed that if the file name given does not exist, the string is treated as a moniker. I was hoping I could generate binary type information in .TLB format from IDL, store it in a flexible manner and provide LoadTypeLib with a moniker to the type information. I then paused as I realized there was an unanswered question – “a moniker to what?”. As is not uncommon in Microsoft’s documentation, elaboration on this point was scarce. I later found this newsgroup post on the matter. The original poster had the same question as mine and the reply pointed me in the right direction.

Although the responder was incorrect in assuming the pointer moniker implementation actually implemented IMoniker::GetDisplayName, a deficiency for which I can find no excuse, the OBJREF moniker provides a suitable alternative. The OBJREF moniker is a superset of the pointer moniker that supports out-of-process references, although no such functionality is required by me for this purpose, just getting a display name to feed LoadTypeLib.

I promptly implemented a skeleton IUnknown that would simply print what interface was requested on every call to QueryInterface and then return E_NOINTERFACE. I created an OBJREF moniker for this IUnknown implementation and supplied LoadTypeLib with the moniker’s display name. I figured this way, I would figure out what LoadTypeLib is expecting the supplied object to implement as an alternative to being given a file name.

I was disappointed when I saw what happened next – LoadTypeLib was asking my object for an ITypeLib implementation, and nothing else. This basically means that LoadTypeLib’s moniker support is completely useless – it returns an ITypeLib for an ITypeLib you already have.

My next attempt to tap into the existing binary type information parser involved writing a test program that called LoadTypeLib on a .TLB file for the purpose of finding if it loaded the information to memory and then promptly used intermediate functionality on the in-memory data that was also accessible to me. I examined the type library loader’s high level flow using Windbg:
0:000> bp oleaut32!LoadTypeLib
0:000> g
Breakpoint 2 hit
eax=0012ff00 ebx=7ffda000 ecx=81818d85 edx=10313d00 esi=0012fdc8 edi=0012ff5c
eip=771279e5 esp=0012fdbc ebp=0012ff68 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
OLEAUT32!LoadTypeLib:
771279e5 8bff mov edi,edi
0:000> wt -m oleaut32 -l 2 -ns
Tracing OLEAUT32!LoadTypeLib to return address 004117d0
7 0 [ 0] OLEAUT32!LoadTypeLib
43 0 [ 1] OLEAUT32!LoadTypeLibEx
16 0 [ 2] OLEAUT32!InitLoadInfo
44 16 [ 1] OLEAUT32!LoadTypeLibEx
65 0 [ 2] OLEAUT32!InitAppData
51 81 [ 1] OLEAUT32!LoadTypeLibEx
9 0 [ 2] OLEAUT32!LHashValOfNameSys
58 90 [ 1] OLEAUT32!LoadTypeLibEx
25 0 [ 2] OLEAUT32!OLE_TYPEMGR::LookupTypeLib
66 115 [ 1] OLEAUT32!LoadTypeLibEx
46 0 [ 2] OLEAUT32!FindTypeLib
72 161 [ 1] OLEAUT32!LoadTypeLibEx
25 0 [ 2] OLEAUT32!OLE_TYPEMGR::LookupTypeLib
89 186 [ 1] OLEAUT32!LoadTypeLibEx
185 0 [ 2] OLEAUT32!GetOffsetOfResource
101 371 [ 1] OLEAUT32!LoadTypeLibEx
79 0 [ 2] OLEAUT32!CreateFileLockBytesOnHFILE
117 450 [ 1] OLEAUT32!LoadTypeLibEx
23 0 [ 2] OLEAUT32!LoadTypeLib2LockBytes
126 473 [ 1] OLEAUT32!LoadTypeLibEx
17 0 [ 2] OLEAUT32!FileLockBytesMemory::Release
133 490 [ 1] OLEAUT32!LoadTypeLibEx
156 0 [ 2] OLEAUT32!OLE_TYPEMGR::TypeLibLoaded
145 646 [ 1] OLEAUT32!LoadTypeLibEx
15 0 [ 2] OLEAUT32!UninitLoadInfo
155 661 [ 1] OLEAUT32!LoadTypeLibEx
5 0 [ 2] OLEAUT32!__security_check_cookie
157 666 [ 1] OLEAUT32!LoadTypeLibEx
9 823 [ 0] OLEAUT32!LoadTypeLib

It was clear from the trace that LoadTypeLib created an ILockBytes over the .TLB file and promptly provided it to LoadTypeLib2LockBytes. Unfortunately, neither this internal function nor any other leading to its functionality is exported from the OLE automation library. The binary type information parser is not accessible externally for in-memory data. What was missing is that LoadTypeLib did not attempt to QueryInterface for ILockBytes when given a moniker, if ITypeLib is not implemented by the object directly. This approach, therefore, had to be scrapped.

I was hoping I could use MIDL to generate binary type information for me and the notion of implementing ITypeLib completely on my own for in-memory representation seemed like a daunting task. If this is the trade-off, surely reverting to ATL and dealing with the evils of registration would be the better approach?

Not so fast. It turns out there is another approach for getting the type information you need for exposing your C++ object, without generating a full-fledged type library or implementing your own type information provider. The marvelous CreateDispTypeInfo API. You provide it with a INTERFACEDATA structure describing your object and get the type information you need. Combined with CreateStdDispatch, it becomes easy to expose simple objects to automation.

Reviewing the sample included in the MSDN documentation of CreateDispTypeInfo is indicative of the sorry state of affairs in Microsoft’s documentation group, seeing as it is quite incomplete and makes use of macros like METHOD0, METHOD1 and PROPERTY, which are nowhere to be found and must have existed in whatever project the sample code has been copy-pasted from. Detailed discussion of the function’s usage is scarce, but existent, on the Web, primarily in newsgroups. Allow me to illustrate with an example. Consider the following hypothetical C++ class one wishes to expose to scripting:

class MyObject
{
public:
virtual void __stdcall f(int i);
virtual BOOL __stdcall g(float f);
};

As is evident, this class is pretty plain and certainly has nothing to do with COM. Just the sort of class your existing application with no use of COM might have. To expose it, we need to fill some descriptor structures so type information can be generated for it. We add a few static members:
class MyObject
{
public:
virtual void __stdcall f(int i);
virtual BOOL __stdcall g(float f);
static PARAMDATA f_paramData;
static PARAMDATA g_paramData;
static METHODDATA methodData[];
static INTERFACEDATA interfaceData;
};

Let’s fill those babies up:

PARAMDATA MyObject::f_paramData = {
OLESTR("i"), VT_I4
};
PARAMDATA MyObject::g_paramData = {
OLESTR("f"), VT_R4
};
METHODDATA MyObject::methodData[] = {
{ OLESTR("f"), &MyObject::f_paramData, 1, 0, CC_STDCALL, 1, DISPATCH_METHOD, VT_EMPTY },
{ OLESTR("g"), &MyObject::g_paramData, 2, 1, CC_STDCALL, 1, DISPATCH_METHOD, VT_BOOL }
};
INTERFACEDATA MyObject::interfaceData = {
MyObject::methodData,
sizeof(MyObject::methodData) / sizeof(METHODDATA)
};

For each method of our object, we describe the method’s parameters, giving them name and type in a PARAMDATA structure. We then fill a method table for the object with complete information, including the parameter data, return value type, calling convention and such. The INTERFACEDATA wraps the whole thing in a nice little package to feed CreateDispTypeInfo with.

We now proceed to create an automation wrapper for our pure object:

CComPtr<ITypeInfo> pMyobjTypeInfo;
hr = CreateDispTypeInfo(
&MyObject::interfaceData,
LOCALE_SYSTEM_DEFAULT,
&pMyobjTypeInfo);
CComPtr<IUnknown> pMyobj;
hr = CreateStdDispatch(NULL, &myobj, pMyobjTypeInfo, &pMyobj);

At this point, pMyobj is a full fledged COM object implementing IDispatch and wrapping the MyObject class instance myobj, which had no knowledge of COM originally and now bundles tables describing its methods.

The scripting site’s implementation of IActiveScriptSite::GetItemInfo should now return pMyObj, the object’s IUnknown and potential IDispatch, and pMyobjTypeInfo, its ITypeInfo, when requested to do so by the hosted scripting engine. We register the object we wish to expose with the engine:

hr = pActiveScriptEngine->AddNamedItem(
L"myobject",
SCRIPTITEM_ISSOURCE | SCRIPTITEM_ISVISIBLE | SCRIPTITEM_ISPERSISTENT);

If our GetItemInfo does its job when asked for “myobject”, assuming we host the JScript engine, we can now do things like
myobject.f();
var b = myobject.g(0.4);

in script code running in our host.

I find this approach to automation object exposition attractive because it is non-intrusive. If desired, the tables describing the exposed class need not be members of the actual class, but can be stored separately. Notice that you do not even have to generate a CLSID for the exposed class. It is also possible to expose only a certain subset of class methods to the scripting environment.

However, maintaining the type information tables can become a clear scalability issue with more complicated classes. For these cases, rolling an automatic code generation solution may be desired, since MIDL’s functionality in this department cannot be reused. The class and its methods could be described in an XML file, and a tool iterating over its DOM or even an XSLT transformation could generate a C++ header file from the description, complete with the INTERFACEDATA information. This would ensure the method tables and the actual method signatures remain synchronized over the extended life-time of the class.

Finally, a pointer to some tips and a few words of caution to those interested in this solution, this newsgroup post. Let me add to it that CreateDispTypeInfo only seems to work correctly with the __stdcall calling convention, even if you specify CC_CDECL in your type information. Using CC_STDCALL and making sure your classes use __stdcall made everything work. Before that, symptoms included method arguments receiving seemingly random values when called by the scripting engine, due to stack imbalance.

Hey, I said the approach is lightweight, not the post ;-)

CRT deployment made slightly easier by Visual C++ 2008 Beta 2

My infamous post on using the OS CRT (msvcrt.dll) instead of the Visual C++ CRT (msvcrXX.dll) enumerated several compelling reasons for avoiding the headache of deploying the latter with your project. Those of you who are less concerned with the size of your distribution package and more with the added complexity of an MSI-based installation will be glad to know that deploying the new CRT as a private assembly has been made easier by the new Orcas Beta. As explained in this post on Nikola’s blog, private deployment is much easier and doesn’t require weird manifest hanky-panky, including the hefty vcredist.exe or using the MSI merge modules.

And the rest of us that want to create ActiveX controls smaller than 900K keep hoping and waiting for an OS debug CRT release in the next WDK…