DocProject Conversion to 2008 and Backlog

In case anyone's interested, this post contains my notes for converting the code base of DocProject 1.7.0 to support Visual Studio 2008 Beta 2. After that I've included DocProject's backlog of tasks, which may contain some ideas that you'd be interested in seeing me implement in a subsequent release.

Notes on the Conversion from 1.7.0 to 2008

It was no easy task, though I don't think my experience is any indication of what must be done for the conversion of general 2.0 Framework applications since DocProject relies heavily on Visual Studio itself at runtime, which is uncommon (at least until future apps start using the new Visual Studio Shell).

While logging this information I was also running into various issues building the source code and running the program. In order to ease maintenance I made whatever changes I could to both the 1.7.0 and 2008 code bases to fix the bugs while keeping them in sync. As a result the code base of DocProject 2008 is 99.99% the same as DocProject 1.7.0, with a few special adjustments such as a different hard-coded installation path under the root installation directory, for example. Many of the updates that I logged were simply changes to references of version numbers but there were also many modifications to the InstallPrep and Installer projects. I don't think any bug fixes are logged here, however.

This is not an exhaustive list of what was required for the conversion. If the time comes again that I must convert DocProject's source I'll certainly follow these notes as a guideline, but I'm sure the process will require my attention.

1. After installing DocProject on the build machine, copy the Source folder to "My Documents\Visual Studio 2008\Projects\DaveSexton.DocProject08\" and uninstall DocProject.
2. Update the Installer project to search for "VSORCAS OR VCSORCASEXPRESS OR VBORCASEXPRESS", but rename the search and conditions to use 2008 instead of ORCAS; e.g., VS2008 OR ...
   NOTE: In 1.7.0 RC for VS 2005 the VSORCAS searches were removed to support side-by-side installation. The VS2005 searches should actually be replaced with VS2008 searches instead.
3. Update the DaveSexton.DocProject.InstallPrep.CustomActions.Contents property to return ONLY the Orcas content.
4. Update all of the projects, including the templates, to use the 3.5 framework. Note that the DocSite templates require a temporary modification to the web.config file before hand unless the modification is done manually. To automate it, use the properties dialog - Application tab for C# projects and Compile tab > Advanced Compile Options button for VB projects.
5. Replace the 2.0 framework prereq with the 3.5 framework in the Installer project.
6. Update the DaveSexton.DocProject VCProjectEngine reference to 9.0.0.0
7. Update DaveSexton.DocProject.InstallPrep\PostBuildEvents\DaveSexton.DocProject.bat, and the Sandcastle, Sandcastle/Deployment and HtmlEditor projects' post build events:
       "%programfiles%\Microsoft SDKs\Windows\v6.0A\bin\gacutil.exe" ...
8. Update DaveSexton.DocProject.InstallPrep:
   1. BootStrapper\vs_piaredist should be changed to vs90_piaredist.
   2. Copy in the vs90_piaredist files and update the Readme.txt file.
   3. Update the Readme.txt file in the project's root to reference vs90_piaredist instead of vs_piaredist.
   4. Contents\Installer\DocProject.AddIn file should be updated to version 9.0 in the HostApplication element.
   5. Replace all occurances of <ContentVersion>1.0</ContentVersion> with <ContentVersion>2.0</ContentVersion> in Contents\Installer\DocProject.vscontent
9. Delete the DaveSexton.DocProject.DocProjectAddIn.GetMSBuildBinPath methods since they are no longer used (see next step)
10. Delete all the code in the DaveSexton.DocProject.MSBuild.MSBuildAnyProject(string) class constructor above and including the local variable named, "vsVersion" and use the parameterless Engine() constructor instead. (There is a warning about this in the Error List when building the DocProject Add-In assembly).
11. Delete Microsoft.Build... references from DaveSexton.DocProject and DaveSexton.DocProject.ExternalUI and replace them with references to the new 3.5 versions.
12. Remove the DaveSexton.DocProject.MSBuild.BuildDocProject.MSBuildBinPath property and all remaining related code, including code that constructs and MSBuild Engine class using a custom bin path. (There will be warnings in the Error List after building the DocProject Add-In assembly).
13. Update the DaveSexton.DocProject.InstallPrep\DaveSexton.DocProject.targets file by removing the MSBuildBinPath attribute from the BuildDocProject and CleanDocProject tasks.
14. Update DocProject and DocSite project templates (found in the InstallPrep's Contents\[template name] folders):
    1. Change version to <ProductVersion>9.0.20706</ProductVersion>
    2. Append <TargetFrameworkVersion>v3.5</TargetFrameworkVersion> to the first PropertyGroup
    3. Change targets reference to <Import Project="$(DocProjectPath)\VS2008\bin\DaveSexton.DocProject.targets" />
       (NOTE: DocProjectPath environment variable must be defined - this is indicated below)
15. Update DocSite templates:
    1. Change type guids to (actually, I don't think this step is necessary):

       C#: <ProjectTypeGuids>{349c5851-65df-11da-9384-00065b846f21};{fae04ec0-301f-11d3-bf4b-00c04f79efbc}</ProjectTypeGuids>
       VB: <ProjectTypeGuids>{349c5851-65df-11da-9384-00065b846f21};{F184B08F-C81C-45F6-A57F-5ABD9991F28F}</ProjectTypeGuids>

    2. Change web app target to <Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v9.0\WebApplications\Microsoft.WebApplication.targets" />
16. Global search for "8.0" just in case any more updates are required for "9.0" (NOTE: the binding redirect in the External UI project's app.config file should NOT be updated to 9.0.0.0)
17. In the InstallPrep project delete the WiRunSQL.vbs file and the UpdateMSI.bat file, and then remove the Installer project's post build event.
18. In the Installer project:
    1. In the File System Editor, change the Application Folder's DefaultLocation to "[ProgramFilesFolder][Manufacturer]\DocProject"
    2. Create a folder under the Application Folder and name it "VS2008"
    3. Drag each root folder into the VS2008 folder and then drag the files located in the root into VS2008 as well.
    4. Changeg the Version to 1.0.0
    5. Change the ProductCode and the UpgradeCode.
    6. Change the Subject to "DocProject 2008 (Full)"
    7. Change the ProductName to "DocProject 2008"
    8. Open the Registry Editor. Hard-code "DocProject" instead of using [ProductName] used in the key for the InstallPath value (for both the HKCU and HKLM).
    9. In the Registry Editor, change the value of each SafeImports key (DocProject value) to [TARGETDIR]VS2008\bin\DaveSexton.DocProject.targets
    10. In the Registry Editor, change the "8.0" key to "9.0" in each of the paths to the SafeImports values.
    11. For each SafeImports key (DocProject value) also udpate the condition to use 2008 instead of ORCAS; e.g., VS2008, VCS2008EXPRESS and VB2008EXPRESS
    12. Change the VS2008\bin\DaveSexton.DocProject.dll.config file's TargetName property to DaveSexton.DocProject08.dll.config
19. Change the assembly and file version for each project to "0.0.1.0"
20. Append "08" to each project's assembly name (Applications tab in the project properties dialog).
21. Do a global search in the entire solution for "DaveSexton.DocProject (with a single quote prefix). Where appropriate, append "08" to the assembly name of references and upate the version to "0.0.1.0" as well, for all projects - not just DaveSexton.DocProject.dll.
22. Rename the solution to "DaveSexton.DocProject08". Then locate the .sln file in the Installer project's File System Editor under the Source folder. Delete the existing file and add the new 08 .sln file.
23. Rename the DocProject.AddIn and DocProject.vscontent files in the InstallPrep project to DocProject08.AddIn and DocProject08.vscontent. They're located in the Contents\Installer folder. Also update DocProject08.vscontent so that it references the renamed .AddIn file.
24. Append "08" to the name of each template folder and update the Installer\DocProject08.vscontent file to point to the renamed .zip files.
25. Update the InstallPrep project's Install.CustomActions class:
    1. Repeat the previous step on the static fields that reference the template zip files and the .AddIn file.
    2. Update the InstallEnvironmentVariables method to use "targetEnvironmentVariable, Target"
    3. Update the UninstallEnvironmentVariables to use "targetEnvironmentVariable, null"
    4. Delete the TargetBin property
    5. Change the buildPathEnvironmentVariable field to: private const string targetEnvironmentVariable = "DocProjectPath";
    6. Add a new property (after the Target property) named TargetFlavor that returns "Path.Combine(Target, @"VS2008\");"
    7. Update the CreateInstaller method to use TargetFlavor instead of Target.
26. Update the DocProjectAddIn class's InstallPath property to return Path.Combine(installPath, @"VS2008\");
27. Update the BuildOutput.bat file in the root of InstallPrep:
    1. Update the template names (only the second argument must be updated). For example: CALL :ZIP_TEMPLATE "%TemplateDirRoot%\VBDocSiteTemplate" VBDocSiteTemplate08
    2. SET InstallerName=DocProjectInstaller08
28. Build the InstallPrep project once. Then open the Installer project's File System Editor and navigate to Application Folder\VS2008 in the tree:
    1. Delete the DocProjectInstaller.vsi file and replace it with the new DocProjectInstaller08.vsi file found in the InstallPrep project directory.
    2. Open the Setup\AddIn folder and delete DocProject.AddIn. Add DocProject08.AddIn from the Contents\Installer folder.
    3. Open the Setup\Templates folder and delete all of them. Add them back from the Contents\Templates folder (hidden in Solution Explorer), but only choose the 08 versions (previous versions can be deleted since this is a temporary folder and they are no longer in use).
29. Run Code Analysis on each project, including InstallPrep but not the templates or the Installer. In some projects there may be certain rules that should be disabled. For example:
    1. Naming: CA1703 (resource spell-checker)
    2. Naming: CA1704 (identifier spell-checker)
    3. Naming: CA1716 (indentifiers that are keywords, such as "step")
30. Make sure to exclude all detected dependencies in the Installer project before building it.

Make sure that the solution builds without any errors. Then build the InstallPrep project and make sure that it completes without any errors.
(Check the build output window for DSZip errors). Build the Installer project and install DocProject on the build machine.

• Install DSZip on the build machine.
• Make sure to copy the "vs90_piaredist" bootstrapper folder into the "C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bootstrapper\Packages\" folder on the build machine.

Backlog

Here's some stuff that's been on the backburner for quite some time now.

• Auto-merge changes to partial-build items. Concrete engines are responsible, but the BuildEngine class could copy all partial items to a temporary folder before a build. Then after the build completes, call the derived-implementation of protected abstract void MergePartialBuildItem(DocProjectItem oldItem, DocProjectItem newItem) for each file in the temp folder. Finally, each temp file would be deleted. This method should be added to the IBuildEngine interface as well. If an implementation can't or shouldn't merge the changes then it doesn't have to do anything with the temp file, or even open it.
• Create an item template for Sandcastle's build component (the New Project Wizard may have to be responsible for referencing the BuildAssemblerLibrary assembly unless the item template supports some sort of wizard that can do the job – but this needs to be researched)
• Components dialog that served as an interface for the .config files. It would be a list of components found in the Sandcastle assemblies, the project's assembly, and any registered assemblies. The order and parameters could be specified, somehow.
  • Create a wizard step that asks the user to choose from various build components to be included in the project's {root}\Components\ folder. For example, XAMLSyntaxGenerator.cs, which would automatically be registered in the .config file.
  • Allow users to add cusotm build components to a DocProject or DocSite via an item template.
  • List of "registered" assemblies on left (registered assemblies cannot be removed from the list if there are stack components that reference them)
  • Build component stack displayed on right ("New" drop-down button, with "External Component", like a browse button, and "Internal Component", that searches the project's assembly (also with "Add New" as an option, somewhere)
  • Up down arrows that control the order that components are executed in the stack
  • Reset button that will import the configuration file for the project's presentation, replacing all changes (Are you sure? dialog)
• Have a start debug button ">" in the Sandcastle toolbar that will put Visual Studio into debug mode and attach to an external "msbuild.exe /project" process for debugging custom build componenents and the build process component. For DocProjects, this may be possible without a custom button by setting the startup program on the Debug tag, if it's possible to do that from the templates.
• ApiFilter Issues:
  • reference method signatures?
  • discover generic type name? (currently, the id is parsed by searching for .name` where name is the value of the name attribute of the corresponding apidata element)
• API Topic Management Dialog:
  • Add/Save/Load feature: Allow users to add a filter (regex or category) to a list that can be ordered using up/down arrows. Saving the list would create an XML doc. Loading the XML doc would recreate the list in the dialog, which could be applied by clicking the list's Apply button (each individual filter would be processed one at a time, top-down).
• Attribute Filter Dialog for Syntax Generator ("Syntax Attribute Filter")
• Automatic token replacement in code comments:
  • Have a dialog that lists token values that can expand to full HTML, which can be edited by clicking an ellipses button that opens an Html Editor dialog, for each individual token.
  • Users can import and export the list to and from an xml file.
  • Make the token replacement start and end strings (both "$" in the examples below) configurable in the tools options page in case the default values conflict with some other device. The values are required and cannot contain ANY white-space. Token replacement will only occur if the following regular expression is matched: \$[^\s\$]+\$
  • When customized, for example, if the start token is % and the end token is ^ then the expression would be: \%[^\s\%]+\^

• Examples:

  Custom Tokens
  
  $isnull$      "is a <strong>null</strong> reference (<strong>Nothing</strong> in Visual Basic)"
                (this could be built-in, but editable)
  $my-token$    "Hello <strong>World</strong>!"
  more custom...
  

  
  Uneditable Tokens (cannot be modified, imported or exported):

  $time$        DateTime.Now.ToLongTimeString(), in the current culture
  $date$        DateTime.Now.ToLongDateString(), in the current culture
  $datetime$    DateTime.Now.ToString(), in the current culture
  $user$        Expands to the current user's name.  e.g., Administrator
  $name$        Expands to the name of the member or type to which the comment applies.  
                e.g., ToString, IsEnabled, BuildEngineProvider
  $fullname$    Expands to the full $name$.  e.g., MyNamesapce.MyObject.ToString, 
                MyNamesapce.MyObject.IsEnabled, MyNamesapce.BuildEngineProvider
  $type$        Expands to the containing type of the member, or the type itself if the 
                comment applies directory to a type.
  $-type$       Exapnds $type$ to a link.  Same as: <see cref="$type$"/>
  $-x|y$        Expands to a hyperlink: <a href="y" mce_href="y">x</a>
                or <a href="x" mce_href="x">x</a> if the vertical bar is not specified.
  $project$     Expands to the name of the project: {project name}
  $-project$    Expands to the project link:  <a href="R_Project.htm" >{project name}</a>
                (or the GUID form)
  $home$        Expands to a hyperlink with its url designated by the user.  By default, this token 
                will be ignored unless the user enters a valid Uri and name.  (custom editor, I guess)
  $art$         Expands to the virtual artwork path; currently, /Art
  

Several of these ideas are already work items that you can vote on. Please let me know if any of them seem interesting to you. I have several other ideas as well that I haven't written down yet but I will when the time comes.

The only reason why I haven't started on the token replacement feature is because I think Microsoft already has an internal implementation that they use, but I'm just itching to create it myself. I'm waiting now to see if they'll make theirs public first before I attempt to reinvent the wheel, if in fact their implementation supports the feature set that I'm after.