Dynamically linking with MSVCRT.DLL using Visual C++ 2005

Despite rapidly approaching its tenth anniversary, Visual C++ 6.0 is still in widespread use for new software development. That much is evident from reviewing the build environments for a variety of open source projects. Some claim its continued popularity stems from old-timers preferring the older, lighter IDE over the newer ones, which also appeared to be more oriented towards the needs of the managed code developer. However, the issue of the C++ runtime and its deployment should not be overlooked.

Visual C++ 6.0 produces C++ executables and DLLs that statically link to the CRT by default in a new project. However, it has become commonplace to dynamically link against its CRT and create a dependency on MSVCRT.DLL. In 2007, this is no big deal, seeing as this runtime DLL has been deployed with every Windows release since at least Windows 98 Second Edition, and even Windows 95 installations with Internet Explorer 4 and later would have it. Software developers happily turned to dynamic linking and over time found themselves not being concerned about CRT deployment – it was already there. This provides for compact and lean binaries with no deployment woes in heterogeneous environments.

With the release of Visual C++ .NET in 2002, a new CRT DLL was born. MSVCR70.DLL was the new name in town, but this time it was nowhere to be seen – developers had to deploy this hefty DLL with their project or go a step backwards to static linking with all its problems. Microsoft cited versioning issues and compatibility (“DLL hell”, etc.) for this change in name, supposedly justifying why the bug fixes and new runtime routines weren’t incorporated into a newer revision of MSVCRT.DLL with the 7.0 compiler release.

The new CRT DLL become just the first one in a series. Visual C++ 2003’s release introduced MSVCR71.DLL and Visual C++ 2005’s release introduced MSVCR80.DLL. 2003’s runtime can only be expected to be present in Windows XP installations while the latter runtime only recently made its way into a shipping OS with the introduction of Windows Vista.

With the availability of Visual C++ .NET, Microsoft recommended and condoned deploying its CRT as a private DLL, simply a file in the directory of installed program. For developers, this only meant an unfortunate ballooning in installer size, but no additional complication. Indeed, various installations and upgrades to the system should leave the dependency intact and the application would always run with the tested DLL version, making the whole thing seemingly more ideal than being dependent on the system’s MSVCRT.DLL which could conceivably change behind your back at any moment.

However, this stance of private deployment is no longer suitable or advisable in our modern world. As we have learned all too well since 2002 and especially from Microsoft’s embarrassing tool to patch a buggy gdiplus.dll across the system, deploying high profile DLLs privately does not allow for central servicing. In such scenarios, the enterprise administrator cannot ensure patched code has been deployed to all dependent applications, even when such code is known to be binary compatible with the original DLL.

It wouldn’t be a shock if a security patch for the CRT were to be posted some day, so deploying the CRT privately is out of the question for the responsible developer.

This leaves us with performing a global installation of the CRT with one of Microsoft’s supported methods. Previously, this was just deploying the DLLs to system32 if they weren’t there, checking if there’s a newer version, etc. – not a big deal, to be sure – although this alone was enough for size and simplicity minded developers to steer clear of the new compiler and the CRT deployment mess that came with it.

Visual C++ 2005’s new CRT further complicates the deployment issues developers are faced with because it makes use of Windows XP’s new side-by-side assembly infrastructure, sometimes referred to as SxS or WinSxS. Readers unfamiliar with it can catch up here.

The most famous SxS assembly is probably version 6 of the Common Controls library, which can only be used by utilizing the SxS infrastructure and is required for enabling “Visual Styles” on common dialog buttons and other controls in Windows XP and later.

In SxS, the dependent application specifies a dependency on a specific version of a DLL using an XML manifest, which can be embedded as a resource in the binary inline, or be a separate file deployed with the binary. Assemblies that wish to become available to such isolation-aware applications can either be deployed privately to the application’s directory (I mentioned why this is a bad idea above, but apparently SxS can mitigate this scenario with its policy and deploy a security update to dependents even in this scenario) or be installed to the side-by-side assembly store, WinSxS.

WinSxS is a subdirectory of the Windows directory. Its function for native side-by-side assemblies is comparable to that of the Global Assembly Cache (GAC) of the .NET world.

So what’s the big deal? New installers would just xcopy the new CRT DLL, MSVCR80.DLL, to WinSxS and get it over with, right? Wrong. It’s more complicated than that.

Microsoft’s official position on this matter has been that there is one supported way to deploy assemblies to the WinSxS store: An .MSI file installed by the Windows Installer. However, as the popularity of this alternative installer shows, not everyone can tolerate its complexity, especially developers of the smaller variety of software.

Microsoft provides another alternative in the form of the “VCRedist” EXE which deploys the CRT and other redistributables to the SxS store that can be invoked by a custom installer. But it can only be redistributed “as-is.”

It’s unreasonable that every small project depending on the new CRT deploy a megabyte or so of DLLs and be sucked into a MSI-based deployment mode. There has to be a better way.

A question arises, what does Microsoft do? They deploy their applications to a variety of Windows environments. A look at Windbg’s dependencies showed it’s using MSVCRT.DLL rather than one of the newer CRTs. Microsoft’s new Network Monitor 3.1 also uses MSVCRT.DLL. Windows Desktop Search is also using the old, trusty CRT.

How can all these new applications be using the vintage CRT? They’re not still using the antique, unsupported Visual C++ 6.0, are they? Well, no. The answer is more complicated and can be found in the Windows Driver Kit (WDK).

What does the Windows Driver Kit, previously known as the Driver Development Kit (DDK), has to do with anything? Isn’t that what all those crazy kernel-mode developers use?

Be that as it may, the Windows Driver Kit also includes a build environment based on the Visual C++ 2005 compiler. The version there seems feature-aligned with 2005’s SP1 release. As driver developers are acutely aware, the WDK’s vintage build system is based upon SOURCES files describing projects and big common Makefile doing all the heavy lifting.

The samples included with Windbg’s SDK (debugger extension DLLs, etc.) also use this build system. Although primarily used for drivers, it can build user space projects as well. Examining the results of the build reveals an interesting fact: generated binaries are compiled with a Visual C++ 2005 SP1 style compiler, yet depend on the MSVCRT.DLL of yore.

We’ll get back to how the WDK build environment (and presumably, other, internal Microsoft build environments for products like Windows Desktop Search) pulls this stunt later. But let us first consider how one would go about doing this independently.

The CRT is just a bunch of runtime functions like strlen and printf and some boilerplate startup code that is statically linked even when using the CRT dynamically due to its very nature (i.e., DllMainCRTStartup which functions as the real DLL entry point, calls constructors for global objects and then calls the user-supplied DllMain). If we get the compiler to use a MSVCRT.LIB import library for the older Visual C++ 6.0 CRT instead of the newer import libraries that bring in the new CRT DLLs and their deployment woes, we’d be fine, right? All we have to do is pass /NODEFAULTLIB and link with an older library and we’re home free, right? Wrong. The naive attempting this simple procedure may get away with simple “Hello, World”-class applications linking and running, but when features such as Structured Exception Handling in C or C++ exceptions are introduced, the whole thing becomes real messy real fast with a bunch of unresolved external symbol errors from the linker.

The reason for this mess is that the CRT serves two distinct functions. One is containing various standard library routines like strlen and printf. The prototypes for these standard functions haven’t changed for years and binary compatibility between different CRTs is a given when it comes to those. The other function of the runtime is to support the language environment, facilitating features such as C++ exception handling with functions such as _CxxFrameHandler3. This part of the CRT is both undocumented and unavailable – it is missing from the CRT sources that have been bundled with Visual C++ releases. It is also the part that appears to have changed significantly between Visual C++ 6.0 and Visual C++ 2005, resulting in the aforementioned incompatibility.

Alan Klietz was the first, to my knowledge, to make a solution to this problem public. It is described in this posting and this one to one of the Microsoft newsgroups. Basically, the solution is the “missing link” (no pun intended) in the linker argument list for getting the old Visual C++ 6.0 runtime import library to satisfy the Visual C++ 2005 compiler. A thunk is added from the new exception handling frame handler to the older one found in the older CRT and other miscellaneous issues are addressed. (new stack allocation functions are required, the run-time checks debugging feature is usually dependent on the CRT, etc.)

This is a nice solution and was available before the WDK and its solution shipped, but unfortunately it requires a post-build step. The exception structures in the compiled binary’s “.rdata” section need to be patched from the new exception magic generated by the new compiler (0x19930522) to the old magic suitable for the 6.0 runtime (0x19930520). The resulting patched binary seems to work, but who knows what’s the meaning of this change and how subtle exception handling semantics are affected by it.

So how does the WDK pull it off? One would presume a solution in use by Microsoft wouldn’t involve a dirty trick like patching the exception handling to use the older infrastructure. Indeed, the WDK’s solution is based on another approach.

Examining the WDK’s makefile.new file reveals the trick – along with the Visual C++ 2005-based compiler, it bundles several object files – msvcrt_win2000.obj, msvcrt_winxp.obj and msvcrt_win2003.obj. There is also mention of msvcrt_winnt4.obj, but this one is nowhere to be found – not a surprise considering the WDK has no build environment for the antique NT 4.0 anymore. One of these objects files, depending on the exact build settings, is passed to the linker along with an import library for MSVCRT.DLL – and viola – the result is a binary produced with a new compiler using the ubiquitous runtime.

So what exactly is going on here? Well, the import library for MSVCRT.DLL included with the WDK is actually not one for the original MSVCRT.DLL included with Visual C++ 6.0, but rather for the latest and greatest version bundled with none other than Windows Vista. That’s right folks – while Microsoft has been preaching to the developer community to move on to the new CRTs, it has been updating the original CRT and compiling OS components and compiling their own products against it. Indeed, Vista’s MSVCRT.DLL includes the new exception handling infrastructure of Visual C++ 2005, previously only found in MSVCR80.DLL.

Unfortunately, naive use of the WDK compiler with the Vista MSVCRT import library results in binaries that attempt to import symbols only found in the Vista version of the DLL and fail miserably on older versions of Windows. This is exactly the kind of versioning mess Microsoft sought to avoid when introducing the new CRT DLL names in the first place.

The various object files included with the WDK are the solution to this issue. They included the difference between the MSVCRT.DLL shipped with a specific OS release and Vista’s. Thus, msvcrt_win2000.obj is the largest, followed by msvcrt_winxp.obj and lastly msvcrt_win2003.obj. Presumably msvcrt_winnt4.obj, available only internally in Microsoft, is even larger. Examining the object files reveals their contents: basically the exception handling code and things like implementations of new secure CRT functions (those with the _s suffix introduced in Visual C++ 2005) required by the modern CRT startup code.

So this seems pretty nice, right? We link with the import library from the WDK and the object file we require for our OS compatibility needs, potentially resulting in a Visual C++ 2005 compiled project that can work with no CRT deployment, at least from Windows 2000 onwards. We use the new exception handling code so we avoid potentially dangerous exception patching we do not understand. Not too shabby.

Unfortunately as with any not-so-documented solution there are quirks. Examination of the LIB directory of the WDK reveals the presence of an import library for the Debug CRT, the infamous MSVCRTD.DLL that provides us with the Debug CRT Heap and other productivity boosting features that have saved innumerable man hours and dollars. So we’d expect to be perfectly able to adapt the WDK build environment for the debug builds of our projects, right?

Wrong. Windows Vista does not include a debug version of its CRT. The latest public release of MSVCRTD.DLL from Microsoft is the one in Visual Studio 6’s Service Pack 6, apparently. Try using that one with a binary linked against the debug CRT import library from the WDK and watch the interesting results – a scary debug assertion failure in initialization time due to a mismatch in the type of a CRT heap block. The new CRT startup code we pull from the WDK’s import library is mismatched with the heap internals of the Visual C++ 6.0 debug CRT DLL. The new debug CRT DLL is nowhere to be found and presumably only exists internally at Microsoft. The public bundling of the debug CRT import library with the WDK is surely an oversight.

Thus developers wishing to produce compact binaries with elegant deployment are expected to give up the beloved debugging features of the debug CRT or develop and test their applications against the different Visual C++ 2005 CRT and only switch to the OS CRT when approaching deployment, risking surprises and unexpected bugs. Every developer should evaluate his priorities and determine whether it’s all worth it.

Unfortunately, since it’s the “small fish” of the developer community that consider a feature such as being to use the OS CRT to simplify heterogeneous deployment important, they have little leverage over Microsoft to convince them to make this solution more readily available or complete it by releasing a debug version of the OS CRT. The Visual C++ team will not add support for the OS CRT directly in the product (my feedback item requesting a feature to that effect was closed and tagged “won’t fix”, “by design”, etc.) while Microsoft’s in-house development teams get access to this functionality in the form of WDK-style build environments, presumably.

Those who for some reason prefer the Visual C++ 2003 compiler should be aware that the Windows Server 2003 DDK includes the Visual C++ 2003 compiler and a build environment for it that also generates binaries depending on the OS CRT, so this is all applicable to that compiler as well. However, there are less implementation differences in areas such as exception handling between Visual C++ 6.0 and Visual C++ 2003, so a hand-rolled solution is easier to come up with. Headers and import libraries from a Platform SDK release (not the newer Windows SDK) should be suitable to get such a build environment in place.

A note on x64. The Platform SDK has been featuring x64 build environments since well before the Visual C++ product finally integrated support for that platform. Presumably intending to make migration of older projects based on Visual C++ 6.0 seemless and the fact the Platform SDK release is aligned with the OS, those x64 build environments also used the OS CRT. Indeed, in x64, MSVCRT.DLL was the name of the game until Visual C++ 2005’s release resulted in the introduction of its CRT to that platform, as well. So, ironically, in the modern x64 platform the profile of the supposedly deprecated CRT is higher than on x86.

I’m hoping this solution becomes more widespread and we can see open source and other projects finally retiring the Visual C++ 6.0 of yesteryear and putting it to the rest. With the issue of runtime deployment out of the way, the advantages of the new compiler far outweigh its disadvantages.

Although I’ve been concentrating on the issue of the core CRT, all of this is applicable to other core assets in the platform such as ATL and MFC. The supposedly driver-oriented WDK has versions of ATL 3.0 and MFC 4.2 suitable for use with the modern Visual C++ 2005 compiler. Using the OS versions of these libraries in addition to the CRT has significant deployment advantages – all those extra bytes sure add up fast.

Just say no to bloatware.