The cons and cons of testing installations with WiX Lux Extension

On my previous post I presented the challenges of testing installations. I've briefly discussed my reluctance with Lux - the only existing framework for unit-testing installations.

In this post I'll show a simple replacement for Lux that only uses standard WiX markup.
First lets summarize the operational concept of Lux tests:

• Immediate custom actions set properties
• Deferred custom actions parse their CustomActionData properties and modify the system accordingly
• Lux tests properties alone, leaving deferred custom actions for other testing methods.

Which reveal the main problem with Lux - its limited scope: It only tests property values. It doesn't check deferred actions' results, nor does it check table values. It doesn't even check that an installation passes successfully - same goes for removal, upgrade, or any other flow.

An example to a basic Lux test - taken from the extension's manual - looks like this:

<Fragment>
  <lux:UnitTest CustomAction="TestCustomActionSimple" 
           Property="SIMPLE" 
           Value="[INSTALLFOLDER]" 
           Operator="equal" />
</Fragment>

This markup tests that a property named SIMPLE equals INSTALLFOLDER property.

One can replace it with this markup, which makes no use of Lux:

<Fragment>

<?ifdef UNIT_TEST ?> 

  <UI>
    <Error Id="10000">UnitTest failed</Error>
  </UI>

  <CustomAction Id="TestCustomActionSimple" 
           Error="10000" 
           Execute="immediate"  
           Return="check" />

  <CustomActionRef Id="WixExitEarlyWithSuccess" /> 

  <InstallExecuteSequence>
      
      <!-- Error if conditions show failure. -->
      <Custom Action="TestCustomActionSimple" Before="InstallFinalize">
        Not (SIMPLE=INSTALLFOLDER)
      </Custom>
      
      <!-- Terminate Successfully if the test passes -->
      <Custom Action="WixExitEarlyWithSuccess" After="TestCustomActionSimple" />
    </InstallExecuteSequence>

<?endif?> 

</Fragment>

This markup defines an error custom action that will be executed if SIMPLE doesn't equal INSTALLFOLDER. If this condition fails (meaning SIMPLE equals INSTALLFOLDER) then the installation will terminate successfully by executing WixExitEarlyWithSuccess.

Lux defines few more test operators, these however are just shims for Windows Installer conditional operators.

Another major drawback of Lux is in its replacement of Product element: Lux replaces the Product element, ignoring anything within it. That means you have to re-organize your markup to be able to test it.
With the alternative markup I presented you don't need to change or re-organize any markup for unit tests to run.

To summarize Lux pitfalls:

• Limited scope - Lux only tests property values
• Lux doesn't check any installation flows
• Product element is ignored altogether.

On my next post I'll present what I think a Windows Installer unit-testing framework should be able to do.

Challenges of Unit-Testing Installations

Every package author has reached the step where a package draft is built and tests are required to find and fix errors.
In many cases testing resolves to eye-check that files reside in the expected folders, and delegated on to QA team for product functional tests.

While most technologies and programming languages are packed with unit-testing frameworks, Windows Installer has none worthy that I encountered. WiX provide a very slim framework called Lux on which I will comment on my next post; however it offers little - if any - capabilities.

Indeed testing an installation package is difficult - files and folders being the easy part. Actually, all standard actions are the easy part as they're already tried and true.
Custom actions are where the problems begin: The basic idea of unit testing is that you can separate small units and test them out of context and independently to the rest of the code.
With installations this isn't true: unless executed by Windows Installer you can't get  valuable information and stimulation, such as built-in properties and execution privileges.

You could design your code in a way that separates input parsing from execution, and test the executing code alone. But even that can be inaccurate.
Examine for example a custom action that kills a process: You may code an immediate custom action to set CustomActionData to the name or ID of the process to kill, and a deferred custom action to kill it.
To test it you will could run the part of that kills the process. Test it with elevated privileges and the test passes; test it as part of an installation and the test fails - that's because Windows Installer - though running with LocalSystem account - runs without SeDebugPrivilege rights, thus can't kill a process.

It seems that installing the product is the only way to test it - same goes for upgrade, uninstall and rollback for each scenario.

In following posts I will expand my reluctance with Lux extension, and suggest a unit-testing framework for installations.

Authoring Very Large Installation Packages Using WiX

A while ago I was authoring an installation package for a customer.
The installation had abundance of files which I harvested using Heat.exe
When I got to running it an unfamiliar error message popped:

Or as it appeared in the log file:
2709: The specified Component name ('cmp8CD8B73AC8349972161D2AEAC3DDEE78') not found in Component table

Checking the MSI with Orca showed that the component was definitely present in the package.
Further investigation revealed an undocumented 65536 limit on component count - while  my customer's application had nearly 200,000 bitmap files.

After having thought of the issue for a while, I reached 3 alternatives to solving it:

1. Splitting the installation to several products, each with no more than 65536 components
2. Zipping the bitmaps into few archives and extracting them during setup
3. Grouping multiple files into fewer components.

While all three alternatives are not perfect, the zipping option is quite outrageous. It generates modifications that Windows Installer isn't aware of, thus can't be rolled-back or un-installed properly.

The first option - splitting the installation to several products - was also cleared off the table due to inferior user experience.

That left me with grouping files into components as the best of imperfect options.
It does have pitfalls:

1. It breaks component best-practices
2. Files may be in-repairable since they are not key paths

Still, considering the product requirements and the alternatives, I decided on going with it.

Now... How was I to group 200,000 files into fewer components?
Having google'd that I concluded XSLT grouping using the Muenchian Method fitted just right.
With that I can group all the files within a directory to a single component. Since there were less than 65536 folders in my client's product, that was good enough for me.
I'll skip explaining the Muenchian Method itself since there's plenty of information about it on the web.

After implementing the grouping the installation passed perfectly.

I've assembled a project to demonstrate both issue and solution here.
To use the demo follow these steps:

1. Download the code for wix-demo project
2. Execute Grouping.bat to generate a folder hierarchy with 65550 files. 
3. Build both projects - NoGrouping will produce the mentioned error, Grouping solves it.

InstallShield - Building multiple releases with MSBuild

A while ago I encountered this problem: I had an InstallShield project in Visual Studio IDE. The project had 3 releases and I wanted Visual Studio to build all 3 releases at every build.

InstallShield's Visual Studio project doesn't inherently support this. It allows you to select one of the releases in the solution's Configuration Manager dialog, and this release only gets built when you build the solution. So in order to build 3 releases I needed to build the solution 3 times - each with a different release selection in Configuration Manager.

To overcome this I added an empty project to the solution where I called InstallShield's command-line builder IsCmdBld 3 times - one for each release.

This involves some manual changes to the MSBuild file:

First we create a project configuration for each release. It will allow us to select 'Release', 'Release_1', 'Release_2' or 'All_Releases' in VS Configuration Manager dialog.

<PropertyGroup Condition=" '$(Configuration)' == 'Release' " />
<PropertyGroup Condition=" '$(Configuration)' == 'Release_1' " />
<PropertyGroup Condition=" '$(Configuration)' == 'Release_2' " />
<PropertyGroup Condition=" '$(Configuration)' == 'All_Releases' " />

And create an ItemGroup for target batching. Target batching directs MSBuild to execute the target once for each release that we want to build - resembling a 'for-each' loop in programming. Notice the condition attribute which creates each item only if its own configuration or All_Releases was selected.

<ItemGroup>
 <IS_Release Include="Release" Condition="'$(Configuration)' == 'Release' OR '$(Configuration)' == 'All_Releases'">
  <Name>Release</Name>
 </IS_Release>
 <IS_Release Include="Release_1" Condition="'$(Configuration)' == 'Release_1' OR '$(Configuration)' == 'All_Releases'">
  <Name>Release_1</Name>
 </IS_Release>
 <IS_Release Include="Release_2" Condition="'$(Configuration)' == 'Release_2' OR '$(Configuration)' == 'All_Releases'">
  <Name>Release_2</Name>
 </IS_Release>
</ItemGroup>


Now we'll create the 'Build' target. To tell MSBuild that this target should be called once for each release we set the 'Outputs' attribute to an item metadata.

<Target Name="Build" Outputs="%(IS_Release.Name).NeverToExist">
 <Exec Command="IsCmdBld.exe -p &quot;$(IS_ProjectFile)&quot; -a &quot;Default Configuration&quot; -r &quot;%(IS_Release.Name)&quot; -b &quot;$(IS_BuildDir)&quot;" />
</Target>

I've passed these command line arguments to the command-line  builder:
-p "project file path"
-a "configuration name"
-r "release name" (This is the batching ItemGroup)
-b "build folder"
There are other parameters which you can explore in InstallShield documentation.
I had to wrap each parameter with " which denotes the " character in XML

The above code contains some properties the I've omitted for readability - Configuration, IS_ProjectFile, and  IS_BuildDir.

Now I can build one of the releases, or all of them at once - which is what I want to happen in the nightly build.

Custom actions come in triples

Custom actions allow us to extend the standard Windows Installer functionality.
We can call JScript, VBScript, DLLs, or an executable. Each of these may be installed during the deployment, be embedded within the package, or already present on the target platform.

Commonly a custom action makes changes to the target platform - such as writing to files, executing DB scripts, writing to registry or any other modification.

This is all well if the deployment finishes successfully. However what would happen were the user  to click 'Cancel'? Or if some error happened during the deployment?

In these cases Windows Installer executes a rollback script starting at the action that failed/cancelled and going backwards.

The aim of rollback actions is to undo any modification to the target platform and revert the system back to the state it was at prior to the stopped installation.

Standard actions have their own rollback information written to the rollback script.
For custom actions that modify the target platform we need to provide a rollback actions.

So far we got 2 custom actions:

1. Deferred custom actions that performs:
1. Create backup prior to any modification
2. Modify the target system
2. Rollback custom action:
1. Undo the modification (restore the backup)
2. Remove the backup

Now we're safe on the rollback issue. However another issue reveals: If the deployment succeeds then an unused backup is left behind.
After the installation has finished successfully we can rid of it. Here comes the commit custom actions - these actions are executed after the installation has finished successfully.
A commit custom action can delete the unused backup.

Now we ended up with the trio of custom actions for any modification to the target system.

To have it all executing orderly we need to sequence the rollback action first, the deferred custom action second, and the commit action last.

EDIT:
Per Dai Corry Comment - I agree that an uninstall custom action should be added to the triple, leading us to a "Custom actions come in quadruples" conclusion.

InstallShield: Deploying 3rd party software prerequisites

Deploying a 3rd party software is a most useful feature in software packaging. Almost any software system relies on another software to function. Probably the most common examples would be .NET framework and SQL server.

InstallShield has a lengthy list of pre-configured prerequisite that you can choose from. To select a prerequisite you simply check it in "Redistributables" view.

Sometimes however you need to deploy a prerequisite that InstallShield hasn't prepared for you. To accomplish that you create a prerequisite definition in "Prerequisite Editor" (Go to Tools->Prerequisite Editor) and it will add to the list.

Creating prerequisite can be divided into two phases:

1. Determining whether or not the prerequisite should be deployed. For instance by checking if the prerequisite is already installed on the target machine
2. Deploying the prerequisite (assuming the answer to the 1st phase was yes)

To determine whether or not a prerequisite should be deployed you perform tests on the target system.
InstallShield lets you test it by:

• Looking for the existence of a registry key
• Comparing the data of a registry value
• Comparing the data of a registry value assuming the data is a version string
• Testing for the existence of a file
• Searching for a given file with a specified date
• Searching for a given file file with a specified version
• Checking the OS version on the target machine

The "Prerequisite Editor" dialog has more options that you can explore (deployment command line, exit code handling, etc...)

Sharing my application packaging experience

Hi,
For some years I've been consulting, tutoring, and packaging applications to Windows platforms.
Over these years I've encountered many confused developers who struggled with application packaging  without having proper knowledge of this issue
Many S/W companies simply purchase a license for a packaging tool (InstallShield, Advanced Installer etc...) and let one of the developers find their way. Usually the assumption is that the UI will lead them the way to a proper package.

Well... In most cases the poor developer manages to build a package. Alas the package is far below professional standards and somewhere down the road it will float (when the installation rolls back, an update is released, or whatever other obstacle is encountered)

In this blog I'll share my experience to help packagers understand and improve their packaging skills. While not a replacement for thorough learning it'll give some references and rules-of-thumb

Nir