In my first DocProject Roadmap post I described my preliminary plans for DocProject 1.x and later versions. In this post I'm going to discuss the project's current status and some updates to my plans for future versions.
1.11.0 RC
DocProject 1.11.0 RC will be deployed within a week after the upcoming Sandcastle release. It contains several new features and bug fixes, and also a few breaking changes that I'll be sure to mention on the release page.
I expect a few changes to Sandcastle that may prompt the addition of a few new features to DocProject before the next release (e.g., support for Help 1.x and Help 2.x project management and, possibly, the ability to include raw HTML topics using Topic Explorer), but since I can't start working on them until Sandcastle is released, there's obviously going to be a delay.
A week should be more than enough time though since I've taken some time off from paid employment to work on my open source projects - a sacrifice that I can barely afford, but working on my own projects is more fun than writing line of business apps and I just couldn't resist after losing my only client anyway (they lost their biggest client first and had to suspend my services, maybe indefinitely).
Universal Installer
The next version of DocProject, 1.11.0 RC, can install DocProject for Visual Studio 2005 and 2008 simultaneously, including all supported Express editions. The code base for DocProject 1.x and DocProject 2008 are now identical and as a result DocProject 2008 now uses the 1.x assemblies.
Ending Phase I
I've done a lot of refactoring in 1.11.0 RC, but I still plan to deploy it as the final release candidate (RC) for DocProject 1.x, with compiler optimizations enabled and without any debug symbol databases (.pdb).
Feedback is a must in this release so that I can iron out showstoppers and maybe even add some mission-critical features that people have been waiting for. If you're a fan of DocProject and you use VS 2005 then now would be the time to get in your suggestions for DocProject 1.x since most of my focus in the next phase of development is going to be on Visual Studio 2008.
Unfortunately I didn't get to add some of the features to 1.11.0 RC as I had planned, but they may be added to 1.12.0 or, more likely, DocProject 2008 Beta 3.
First Production Release
I still expect 1.12.0 to be the first production release of DocProject, but now it will target both Visual Studio 2005 and 2008. To support this I've reorganized source control, once again, and included the source for other projects as well, such as DocToMaml. I've also decided not to package the source code with the installer anymore, starting with 1.11.0 RC.
My plans for DocProject 1.x after the first production release are to provide subsequent releases for bug fixes and small feature updates only, targeting both VS 2005 and 2008. The major features that I have been planning will make their first appearance in DocProject 2008 Beta 3.
DocProject 2008
After I deploy 1.11.0 RC, development will continue on both 1.12.0 RTW and DocProject 2008 Beta 3 simultaneously, although the most notable of new features will only be included in DocProject 2008 Beta 3. Some of these features are described below.
Please let me know which features you're most interested in so that I can prioritize.
Visual Studio Package
DocProject currently provides an Add-In that only uses VS automation, but I'd like to provide more integration to create a better Topic Explorer, Topic Editor and Topic Designer, and even a preliminary MAML editor. I've done some initial research and have already come up with a plan to start development.
This is an exciting task to work on but I don't plan on the first release being extremely stable as this will be my first VS package; although, I do plan on reusing a fair amount of the automation code since it has already been tested. I'll also incorporate much of DocProject's current architecture, including its high level of extensibility and its "openness".
Business logic, infrastructure and utilities will probably end up being mostly the same, but I do want to take this as an opportunity to update some of the designs that DocProject currently uses, like change buffering.
Standalone UI
The current DocProject External UI program relies on DocProjects and DocSites that have been created in Visual Studio. For help authors that just-so-happen to be developers, this isn't a problem. But for anyone that isn't a developer, the free Visual Studio Express edition works well but doesn't make much sense since it's obviously not designed just for building documentation, and certainly not for authoring it.
DocProject 2008 Beta 3 will use the new Visual Studio 2008 Shell to provide a tool that has all of DocProject's features, but without requiring Visual Studio to be installed. I haven't decided yet whether I'm going to use integrated or isolated mode, but I'm leaning towards integrated.
This should be quite easy to create since DocProject is becoming a VS package anyway. A blocking issue though is whether MSBuild is provided by the Shell redistributable. If not, then I may just have to continue to deploy the External UI instead. I'm also not sure whether the Shell provides C# and VB.NET source code editors, but if not it's no big deal since they aren't available in the current External UI either. (The code editors would be used to edit the project's build process component. I think this would be a great feature to have in the standalone app, if possible, but it's not a requirement.)
My ultimate goal for the standalone UI is to create a program that works with or without Visual Studio. It must provide the ability to create new DocProjects and DocSites from existing MSBuild project templates, so they work in Visual Studio as well. It will also provide all of DocProject's existing and upcoming features, including support for UI extensibility; e.g., providers can register new tool bars and tool windows.
Continuous Integration
A project option will be available to enable support for continuous integration. When enabled, DocProject will copy all dependencies into the project, including DocProject and Sandcastle assemblies, configuration files, and the HTML Help Workshop program, allowing them to be easily committed to source control. After a DocProject or DocSite is checked-in, another team member can get the latest version to their computer or to a build server and all the dependencies will be updated as well, without requiring any installation.
The only blocking issue of which I'm aware seems to be that Help 2.x requires installation. DocProject does not use any of the command-line tools that are installed by the VS SDK, however, it does use the MS Help COM interfaces. Unfortunately, to get the libraries and COM objects installed you must install the VS SDK, but to install the SDK you must first install Visual Studio. I'll continue to research this when the time comes to see if I can find a workaround, but so far it doesn't look good.
Continuous Builds
I mentioned this in my previous roadmap post. The idea is to detect only the changes made to APIs and XML documentation and have a web server that continuously rebuilds a live DocSite so that the updates are visible immediately to all team members.
This feature will require dependency detection and differencing algorithms for both API references and XML documentation. One question is whether these algorithms will perform better than building entire documentation sets from scratch, but I believe that they certainly will in the scenarios where it really matters.
The goal is that for projects with many large assemblies there should be little delay to see the updates in documentation, at least compared to the very long delay that ensues when rebuilding entire documentation sets from scratch, which is currently the only option with Sandcastle.
To use this feature you would start be creating a new DocSite project and probably selecting some option in the New Project Wizard to indicate that you want continuous build support. You would also select a VS solution file or an MSBuild project that the DocSite will use to automatically build sources. In the last wizard step you would add all of the sources to the project as project references, like normal. When finished configuring the project as usual, you would deploy it to an intranet web server that all team members can access over HTTP.
The live DocSite would trigger a rebuild using MSBuild at a configurable time interval. This way MSBuild will detect updates automatically and the DocSite will always have the latest versions of all sources. The DocSite will monitor changes to the sources and continuously rebuild only those topics that were affected. It will also update the DocSiteContents.xml and DocSiteIndex.xml files, as well as the full-text search index.
One problem might be getting the latest source code onto the server so that the DocSite can rebuild it. In a team scenario it's probably safe to assume that some sort of source control system is in use. I'm not sure if it's possible, but if source control systems provide an event model that could be used to automatically get the latest version to the web server after each check-in, that would solve the problem. Otherwise, manually getting the latest version to the server may be required. It may be possible to access the source code over a network share (to a build server perhaps) so that it doesn't have to be stored on the web server, although I'm not sure if MSBuild works with UNC.
Using external sources instead of project references may be another option, but replacing the assemblies on a network share automatically might be just as difficult. It could be done manually or, as with the solution for project references, using source control events to run a build after each (labeled?) check-in.
Localization
I've already put a few hours into researching localization support in DocProject. What I discovered, and I'm not surprised, is that if I want an implementation that isn't a hack I'll have to add some new features to support dynamic changes to various configuration files during builds. Anticipating this problem, I had decided to defer localization support into DocProject 2008.
I suspect that these new support features will be useful on their own as well, much in the same way that extension methods in C# 3.0 were probably added to support LINQ yet they are even useful by themselves. The new features will add to DocProject's already high level of extensibility and will be customizable via the DocProject configuration file.
My goal for localization support is for a single DocProject or DocSite to produce multiple compiled help files for different locales in a single build. This must be accomplished without affecting the current feature-set of DocProject, including the Sandcastle Build Component drop-down properties and the fact that you can currently change almost anything that you want in the BuildAssembler configuration files. So in other words, the localization feature must be completely transparent when it's not being used.
All Sandcastle content items and user-written XML documentation comments will be localizable. Once implemented, I plan to ask the community to create language packs for content items in various locales (Unfortunately, I only know English, and not so well ;). A language pack will be a .zip file that contains one or more Sandcastle content documents and an XML manifest file, which describes the contents and rules for locale inheritance. DocProject 2008 will be able to locate the packs in its installation directory, by default, or a user-configured location.
With language packs, DocProject can offer out-of-the-box localization for multiple languages so that help authors can simply check a box and build their project to produce multiple compiled help files, one per language. Localized XML documentation comments can be added to the project's Help\Comments\[locale] folder (e.g., Help\Comments\en-GB) and it will be merged automatically with XML documentation comments from the source code (how it's merged will depend upon the Merge XML documentation project option, in the same exact way that it works now in the current version of DocProject). Likewise, localized conceptual topics (MAML) may be added to the project's Help\Topics\[locale] folder.
Instead of trying to write down the many concepts related to implementing localization support that I have echoing around between my ears, I'll just post my notes in their raw form. If you want more details, please let me know.
1. add content item mappings to DocProject config file:
<item name="..." presentationNames="Visual Studio 2005, Sandcastle Hana" source="options-property-name"/>
- Items that are added here are automatically removed from the content item documents when a project is
first created.
- Before each build a dynamic content item document is generated in the working directory for all of the
items that are configured here.
- Note that a configChange must be added to register the dynamic content item document.
2. Add a wizard step, before the Create Shared Content step, that allows a user to specify the values for
each of the pre-defined_enum_names:
- Documentation set name
- Running header text
- Copyright
- Company name
- Default locale (combox box)
3. Add a Localization Management dialog.
- Choose the different locales in which distinct documentation sets will be generated by selecting from a
drop down list, which will add the selection to another list.
- one locale may be selected as the "working" locale for the dialog; UI elements will apply only to the
"working" locale.
- a tree view, like solution explorer, shows only the folders that may be localized within the project;
e.g., Help\Topics, Help\Comments, etc.
- one node may be selected at a time.
- for the selected locale/node, a button can be clicked that will prompt the user to choose a locale from
which to import files, with the option of specifying whether the import should be performed recursively.
- Choose a default locale (corresponds to the wizard step option indicated above).
- A single build will produce all of the documentation sets in each locale.
- The BuildContext should have a property that returns the current LCID that's being built and another that
returns the list of all LCIDs that are being built.
- Output will be stored in sub folders named with the locale (e.g., bin\Debug\sp-SP\ or Help\bin\sp-SP).
- Local input will be stored in sub folders named with the locale (e.g., Help\Topics\sp-SP\,
Help\Comments\sp-SP\, Help\Presentation\Style\Content\sp-SP, Help\Presentation\Shared\Content\sp-SP).
- The default locale's content will be stored at the root; e.g., Help\Topics\, Help\Comments\, etc. so the
build process for the default locale will work the same way that it does now.
- Source input can be managed in the same way that version management works.
4. Support dynamic updates to config files at build time.
- Take all of the <add> elements in the <configChanges> element and move them under a sub element named,
import.
- Add a new sub element named build that will contain <add> elements for changes to be applied during
builds.
- replacements for build changes will work the same as for import changes, although they will also allow
string replacement tokens: {0} {1} {2} ...
- The values of replacement tokens will be defined by attributes on the <add> element: value0="..."
value1="..." value2="..."
- The values of the replacement attributes are strings that identify the names of public properties on
the current derived implementation of the BuildSettings class during builds; e.g.,
value0="DocumentationSetName"
- Single-valued properties are converted to strings using Convert.ToString().
- Properties of System.Collections.IEnumerable implementations are iterated and each element is handled
recursively. Single-valued elements are converted to strings and IEnumerables are iterated.
The replacement is executed once for each iteration, recursively.
- When a build starts the configuration files that will be used are copied to the working directory and
then the build changes are applied.
- Build assembler will use the config files in the working directory.
5. Support dynamic localization of config files at build time.
- Add a new sub element to <configChanges> named, localization.
- replacements for localization changes will work the same as for build changes, including replacement
tokens, expect that replacements are specified as properties of the current BuildContext object.
- BuildEngine implementations should be able to override a method to create a BuildContext
implementation, such as SandcastleBuildContext, so that properties such as Lcid and Locale are dynamic.
- When a localized build starts (not the BuildStarting event but the first step of a localized build, which
is executed repeatedly within a single build - once for each locale), the configured localization
changes are applied, for the current local, to the working copy of the build assembler config files.
Build assembler must then be executed against the localized copy. (The original copy must not be
replaced.)
** The explanations above were created before this example. This example illustrates the _better_ approach
when there are conflicts in the notes above since it's more thought out.
- DaveSexton.DocProject.dll.config -
Sandcastle presentation configuration section
<presentations>
<presentation name="..."
... all of the attributes that currently appear in a presentation's add element ... >
<options>
<add name="RunningHeaderText" type="System.String" displayName="Running header text"
description="The text that will appear at the top of each topic." category="Configuration"
sort="80" /> <!-- the specified type must be convertible to and from System.String
(serialization is not supported) -->
</options>
<content>
<item name="runningHeaderText" value="RunningHeaderText"/>
<!-- value is the name of a dynamic project option here, but it can be any property that
exists on a DocProjectOptions implementation -->
</content>
<newProjectWizard>
<step type="type name" assembly="assembly name"/>
<!-- steps are inserted before the Create Shared Content step in the order that
they are added here -->
</newProjectWizard>
</presentation>
... more presentations
</presentations>
<configChanges>
<import>
... all of the stuff that is currently in the configChanges section ...
Note that the default comments folder should be changed from Comments\*.xml to Comments\default\*.xml
since comments should be segregated by locale; 'default' is used since the default locale can be changed.
</import>
<build>
<add name="assign default reference links locale" ....
xpath="find ResolveReferenceLinksComponent2 locale attribute"
setAttribute="{0}"
value0="Locale" /> <!-- Locale refers to a property (that must be added) on the
SandcastleSettings class -->
</build>
<localization>
<add name="assign reference links locale" ....
xpath="find ResolveReferenceLinksComponent2 locale attribute"
replaceWith="{0}"
value0="Locale" required="true" />
<!-- Locale refers to a property on the SandcastleBuildContext class (this class does not currently
exist; it will derive from BuildContext) -->
<!-- The required attribute indicates that append should be used if no node is matched by xpath;
although, I'm not sure if I'll be able to determine programmatically whether xpath is selecting an
attribute or an element -->
<add name="assign localized comments" ....
xpath="find CopyFromIndexComponent with index name='comments' and select
files='Comments\default\*.xml'"
replaceWith="Comments\{0}\*.xml"
value0="Locale" />
<add name="assign shared content items" ....
xpath="find all of the shared content component's content items that start
with ..\..\Help\Presentation\"
replaceWith="..\..\Help\Presentation\content\{0}\{1}" value0="Locale" value1="Regex:[^\\]+.xml"
collection="true" /> <!-- the collection attribute will indicate whether xpath is expected to
return a node collection, in which case this change is applied to each node -->
</localization>
</configChanges>