DocToMaml 1.0 Beta is a tool that converts HTML and XHTML help files to Microsoft Assistance Markup Language (MAML) in a batch process.

MAML help files are used by DocProject and Sandcastle to build user documentation in various presentation styles.

DocToMaml provides a console mode and a project-based graphical user interface (GUI) for adding inputs and defining conversion rules to quickly and easily convert HTML files into MAML. Future releases will provide more flexibility for defining rules, but the beta can still be used to greatly decrease the amount of effort that would be required to convert large numbers of HTML files into MAML, compared to doing it manually.

Features

• GUI and console mode.
• Project-based for quickly saving and loading different configurations.
• File and folder inputs are supported.
• Global and input-specific rule sets may be defined.
• Edit the source HTML of a file input using a WYSIWYG editor for full control over the conversion.
• Convert a file input in memory so that you can see the result quickly.
• MAML results for file inputs can be modified in a text box.
• Batch conversion saves results to disc for all file and folder inputs and generates a conceptual artwork file that can be used by Sandcastle's ResolveArtLinksComponent build component.
• Hyperlink references to local topics are updated automatically.
• Preliminary user documentation.

Current Limitations

• Only the Conceptual MAML document type is supported.
• Subsections and sub containers are not supported. For example, nested tables and lists are added to the current in-line element, if one exists; otherwise, they are added as new top-level containers.
• Images appear to be broken in source view, although this has no effect on the output. You don't have to update the paths to broken images since DocToMaml doesn't do anything with image files in the current release anyway.

Jan Schreuder wrote an article on Sandcastle for Microsoft's Dutch .NET Magazine and it's now available on the web.  In it he provides an overview of Sandcastle and some instructions on how to use it.  The article also features a few popular community tools that are available for use with Sandcastle; namely, Sandcastle Help File Builder, AjaxDoc (ScriptDoc) and DocProject.  Jan mentions that DocProject is his preferred tool for automating Sandcastle and cites some of its advantages.  Thanks!

The article is written in Dutch, but I was able to successfully translate it using Google Translator.  First, download the PDF (visit Jan's blog for the link) and then save it as text (check the File menu).  Copy and paste the text into the translator, select Dutch to English and press Translate.  It works well enough to get the gist of the article.

Since the article was written there have been some important changes to Sandcastle and DocProject.  Most notably, Sandcastle is now RTW on CodePlex and DocProject now provides first-class support for conceptual documentation using MAML, which of course wasn't mentioned in the article.

More Information about Sandcastle

For an up-to-date reference check out my (peer reviewed) Sandcastle Help article on CodePlex.  It doesn't go into detail about how to use Sandcastle but it does provide a bullet-point overview and catalogs the different tools and documentation that exist elsewhere on the web.  It's a good starting point.

More Information about DocProject

See my About DocProject article on CodePlex for an overview, screen shots and how to get started.

I also plan to write a few more DocProject tutorials in the future, as well as Sandcastle tutorials for things like how to create a custom presentation style and how to add custom tags, so keep an eye on my blog and DocProject's RSS feed on CodePlex.

The DocProject 1.10.1 Release Candidate is now available for download on CodePlex.

This release contains some important bug fixes for 1.10.0 RC.  One of which, it was reported, prevented the DocProject Add-In from working at all on some installations of Visual Studio (the Project context menu could not be located).

It also provides built-in support for ResolveExternalLinksComponent, which has been updated to include support for URL mapping provider editors and a built-in database URL mapping provider, among other features and bug fixes. The stand-alone component will be released later this week.

ResolveExternalLinksComponent is a Sandcastle build component that I wrote to solve a general usability problem that was reported here, by Karl, and to dog food my Hosted Sandcastle Build Component Template.  The component's current behavior was designed with Karl's help.  Thanks Karl :)

Preliminary documentation for the component is here.

Note that the latest release of the component is built-in to DocProject 1.10.1 - a separate download is not required.  When you create a new DocProject or DocSite the component will be included automatically in the Help1.x and Help 2.x reference stacks so that you can begin to use its features right away in your XML documentation comments.

Overview

The original problem, to paraphrase, was that Karl had to keep writing repetitive URLs for <see href="url"/> links throughout his XML documentation comments.  The common URLs he was using pointed to third-party online documentation and each URL shared the same base, so only the relative portion changed between links.  To simplify the process he wanted to write only the relative path each time a link was required, specify the base URL only once, somewhere else, and then have some magic combine them into an absolute path, post-compilation.

As it turns out, a Sandcastle build component was right for the job.

I liked the idea of using IDs instead of hard-coding common URLs throughout the documentation and felt that this would be a feature that I might use, so I developed the component to resolve <see xref="id"/> into hyperlinks using pre-configured ID-to-URL mappings.  For Karl's particular scenario I added support for the vref attribute, which adds additional semantics to the ID so that it will act as the URL itself if a mapping is not defined.

During development I noticed a few opportunities to add more features, which I did, like being able to associate a group of ID-to-URL mappings with any combination of optional attributes.   In particular, a base URL, hyperlink target, auto-generated reference topic ID or import file.  The component allows for any number of mapping groups with any combination of these attributes, allowing for some pretty interesting scenarios.

For example, in one scenario you might define common URLs as IDs for all of your documentation but then override the URLs in one particular topic to point elsewhere, without even having to modify the XML comments in the API.

Another possibility would be to group mappings by hyperlink target.  So, for example, you could set the default target for all mapped links to open in the same browser frame when clicked but then configure a group of links that reference a third-party website to open in a new browser window when clicked.

The component has a graphical editor that works in DocProject and the Sandcastle Help File Builder, which makes configuring it really simple.  It also provides a few DocProject-specific features, such as providing a list of topic IDs that you can choose from, defaulting to the project's settings directory when using an external mappings file, using a relative path to an external mappings file, automatically including an external mappings file as a project item and expansion options for build component stacks in the DocProject Properties window.

What's New In Beta 2

The next release has a few bug fixes and some neat features that I'd like to mention here.  You can read about additional updates on the release page.

First, I've added support for resolving IDs to URLs that are stored in a database.  (See the section on URL Mapping Providers below for more information.)

You can now use xref and vref attributes on seealso tags as well; although, due to a current limitation in Sandcastle, the target attribute will be ignored.  However, if Sandcastle eventually supports the target attribute on seealso tags then my component should work without having to be rebuilt.

The editor now provides the ability to edit the base and target attributes on the component element.

In DocProject, if you configure an external mappings file then it will be included as a project item automatically.  Also, you can use the open file dialog to specify the name of a file that doesn't actually exist and it will be created when the editor is closed.  The path of the file will be made relative to DocProject's working directory.

URL Mapping Providers

The common way to define a group of mappings is to add them to the component's XML configuration using the mappings and url elements.  The class that parses url elements is, XmlUrlMappingProvider, and as you may have already guessed, it's a specialization of the abstract base class, UrlMappingProvider.  Here's what a common configuration might look like:

<component type="DaveSexton.Sandcastle.ResolveExternalLinksComponent"
           assembly="C:\...\DaveSexton.Sandcastle.dll">
  <mappings>
    <url id="home" href="http://davesexton.com" />
    <url id="contact" href="http://davesexton.com/Contact" />
  </mappings>
</component>

You can create your own custom providers as well.  To use them simply specify the full type name and path to the assembly using the type and assembly attributes, respectively, on the mappings element.  The outer XML of the mappings element will be supplied to your provider as raw text.

URL mapping providers can have a UI that will be displayed in the component's editor to easily modify the provider's configuration.  The built-in XmlUrlMappingProvider uses a simple DataGridView for editing the url elements.

DatabaseUrlMappingProvider

DatabaseUrlMappingProvider is a new provider as of DocProject 1.10.1 and the ResolveExternalLinksComponent Beta 2 release.  It allows you to define ID-to-URL mappings in a database table instead of the inner XML of a mappings element.

This provider could be useful in scenarios where you already have URLs stored in a database or when you have a large number of URLs that must be shared between multiple tools, whether related to documentation or not.

A custom editor that allows you to configure the provider is embedded into the component's editor.  With it you can easily specify the data provider, connection string, query, parameters and other settings all in one place:

Here's an example configuration that might be used with this provider:

<component type="DaveSexton.Sandcastle.ResolveExternalLinksComponent"
           assembly="%DocProjectBuildPath%\DaveSexton.Sandcastle.dll">
  <mappings type="DaveSexton.Sandcastle.DatabaseUrlMappingProvider"
            topic="M:SomeNamespace.SomeType.DocumentedMethod(System.String)">
    <connection provider="System.Data.SqlClient">
      Data Source=DEV1\SQLEXPRESS;Initial Catalog=Docs;Integrated Security=True
    </connection>
    <query>
      SELECT ID, HREF FROM UrlMappings WHERE TopicID = @TopicID;
    </query>
    <parameters>
      <parameter name="@TopicID" source="TopicID" />
    </parameters>
  </mappings>
</component>

The component can use any of the built-in Framework data providers (e.g, Odbc, OleDb, SqlClient) or even plug-ins such as Oracle.

Parameterized textual queries and stored procedures are supported, but only the first result set is used.  By default the provider will look for ID and HREF columns, but you can specify different names.

Parameters can have hard-coded values or they can be mapped to a specific source.  In the example above the value of the topic attribute will be used as the value for the @TopicID parameter.  The other acceptable values for the source attribute are: BaseUrl, File and Target, which also correspond to attributes on the mappings element.  To define a specific value for a parameter use the value attribute instead of source.

The Sandcastle Build Component Templates 1.1 release provides two C# Item Templates that you can use to easily create custom Sandcastle Build Components in Visual Studio 2005 and Visual Studio 2008.

In this blog post I intend to provide some details about the templates, with more focus on hosted components and DocProject's API.  Instructions for template installation and usage can be found here.

Basic Sandcastle Build Component Item Template

The Basic Sandcastle Build Component item template is a multi-file Visual Studio Item Template, written in C#, that provides a working example of a custom build component that can be used for command-line builds or with editor support in automation tools such as DocProject and the Sandcastle Help File Builder.  It comes with a built-in graphical editor and a dynamic sub property for the DocProject Properties window, which can be shown by expanding the component in a build component stack (for more information about the stack properties, see How To Use Third-Party Sandcastle Components in DocProject).

Hosted Sandcastle Build Component Item Template

The Hosted Sandcastle Build Component item template is the same as the basic template, but it also provides direct access and a working example of how to gather information from DocProject's API, thus DocProject is required in order to compile the build components that are created based on this template.  Although, DocProject isn't actually required to use the components if you provide a fall-back initialization path.

Initialization Paths

The template's Configuration class provides two initialization paths, one of which is a fall-back that's automatically invoked if the DocProject host API is not detected.

Note: The host API is never available during builds - it's only available when the component's editor is displayed.

Fall-back initialization works since the template depends upon just-in-time (JIT) compilation.  If you properly encapsulate the use of host interfaces, as the template does out-of-the-box, then you can control the initialization path of the component's Configuration class.

The template's Host class is used to encapsulate host interfaces and services, and it provides a way to safely check, at runtime, whether a particular host is reachable.  Support for DocProject's host API is built-in to the template.  All other hosts will use the fall-back initialization path automatically unless you add support for them to the Host and Configuration classes.  The only requirement is that the host invokes your component's editor by calling its EditValue method and passes in an IServiceProvider implementation that can be used to obtain a reference to one or more of the host's services.  (On a side note, the value argument should be set to a System.Xml.XPath.IXPathNavigable implementation that's positioned on the outer XML of the component's configuration, which includes the <component> element itself.)

Modifying Project Options

The template gathers read-only data as an example and does not actually modify any build settings or project options at runtime.  If you want your component implementation to write to the API, then make sure there's a clear relationship between your component and the changes that it's making.

For example, a proprietary AuthorBuildComponent changing the value of the Generate root API topic setting when edited will probably not be obvious to end-users.  But it might be appropriate for the same component editor to add external sources if, for example, it were to provide a way for an end-user to create a list of associations between sources and author names, much like the Version management dialog does with version information.   In this case, the component's editor could save the input-to-author mappings as its inner XML configuration data and add the external sources as a convenience for the user.

Hosted Components as Plug-Ins

Hosted build components are not meant to be a plug-in mechanism for DocProject, per se, but instead are a plug-in mechanism for Sandcastle.  DocProject's API is exposed merely to provide a more user-friendly and integrated experience when editing Sandcastle build components in DocProject.

If you want to create a true DocProject plug-in, then create a custom build engine provider or a build process component, which can provide control over all aspects of DocProject, including the entire build process, engine settings, project options, tool bars, tool windows, menus, custom wizard steps in the New Project Wizard, etc.

DocProject's API for Hosted Components

With the recent release of DocProject 1.10.0 RC, a new interface was added that would provide build components access to DocProject's API, for their graphical editors.  The API can be used to gather information about the environment in which the component's editor is being hosted.  The component can then provide automatic configuration and apply appropriate editor defaults, simplifying configuration for end-users while also adhering to the project paradigm that DocProjects and DocSites use to store their settings.

The purpose of the gateway interface, IDocProjectHost is to provide components with access to DocProject-specific interfaces and types for the context in which the component's editor is being hosted.  Here's the interface's definition:

public interface IDocProjectHost
{
    IDocProject Project { get; }
    BuildSettings Settings { get; }
    DocProjectOptions Options { get; }
    IBuildEngine Engine { get; }
    IEnvironmentHost EnvironmentHost { get; }
    bool RunningInVisualStudio { get; }
}

The Hosted template gets a reference to an IDocProjectHost implementation through the service provider that's passed to the editor's EditValue method.  If the host is available then the Configuration class's InitializeForDocProject() method is invoked instead of the fall-back Initialize() method (see above for information about initialization paths).

The InitializeForDocProject method uses the types and interfaces exposed through IDocProjectHost to gather information, which it then stores in private fields.  The fields back properties that are used by the template's EditorControl class.  They are typed as nullable primitives, such as System.String and bool?, so if the values are never set then they'll remain null, in which case the EditorControl will substitute them with "Unavailable" for its read-only display.

Developers are free to remove this code (actually, it's recommended) and replace it with appropriate code for gathering information that's required by the component being developed.  However, the same pattern can be used to safely encapsulate data that can be initialized with or without a host API.

Note: The configuration fields must not be typed as host-specific types or interfaces if you want to be able to use your components in other environments, without the host API installed.

Conclusion

With the release of DocProject 1.10.0 RC and the corresponding Hosted Sandcastle Build Component Item Template 1.1, developers can now have seamless integration in DocProject for their custom build components' editors, providing a more robust platform on which to configure and build documentation while potentially simplifying configuration for end-users through automation and inferred default settings.

The DocProject 1.10.0 Release Candidate is now available for download on CodePlex.

It contains lots of bug fixes and some new features that I hope you'll like.  You can read my preview post for information on a few of the more important features that have been added, and a comprehensive list of all of DocProject's features is available here.

Be aware that I'm still in the process of updating the wikis on CodePlex, which can take a while.

If you feel like you want to leave a comment here, or tell me about your experiences with the new version of DocProject, good or bad, please go right ahead :)

Oh, and one more thing, Conceptual Content!!!

Now you can use DocProject to build a .CHM and .HxS for conceptual documentation, without the need for auto-generated reference topics.

In other words, DocProject automates the Microsoft HTML Help Workshop and Help 2.x compiler to build conceptual documentation right inside of Visual Studio using Sandcastle's conceptual build process and MAML, so the output looks identical to a presentation style such as Sandcastle's Visual Studio 2005.  (Note that I haven't even attempted to build conceptual documentation with the other presentation styles since they were not tested by Microsoft.)

Getting Started With Conceptual Content

Create a new DocProject or DocSite and you can immediately start writing your conceptual documentation in Visual Studio's XML editor, with full IntelliSense for MAML schemas.  (Though it seems that you may need to open your project's Help\Schemas\developer.xsd file in the designer first before you'll get IntelliSense.)

To get started, press the Topic Explorer button on the Sandcastle tool bar and then add a conceptual topic by clickingon the Topic Explorer's tool bar.  DocProject will ask you which template you want to use.  For general documentation use the template named simply, Conceptual.  If you don't need auto-generated reference documentation, then you can build the project now and it will produce conceptual-only documentation.

Mixed Documentation

But even better, if you do have APIs that you want documented, DocProject will build all of your topics together as a single conceptual/reference project without any additional requirements.  Simply add references to your source projects and build.

If a DocProject or DocSite has one or more sources for documentation (external or project references) then an auto-generated root topic for reference documentation will appear in Topic Explorer.  You can add new conceptual topics to the TOC relative to the root reference topic and even drag & drop conceptual topics and the root reference topic to produce the TOC that you want.  Add as many conceptual topics and sources as you'd like, then build.  That's it!

Enjoy :D

As you may have already heard, Sandcastle [1] was officially released to the web (RTW) through CodePlex on January 15, 2008.  The license that was chosen is the Microsoft Public License (Ms-PL) [2], but keep an eye on it because it may change [3].  There are still a few unanswered questions that I have [3] about how Microsoft's work on Sandcastle will continue and how it will affect DocProject, but things are looking very positive.

In this blog post I'd like to express a few of my goals and ideas for DocProject and maybe even try to establish some preliminary timeline.  And of course, if you have any thoughts your feedback will be welcomed :)

The End of Phase 1

With the help of community feedback and my propensity for being anti-social in favor of sitting in a dark room, programming, DocProject's long initial development phase is nearing an end.  The next release, which you can read about in my blog (DocProject 1.10.0 RC Preview [4]), brings together a few long-awaited features that will hopefully start to make DocProject into a useful help authoring tool (HAT) [5].  I'm going to try for Monday, January 21st, to release 1.10.0 RC, and I'm hoping that the worst-case scenario will be the following Friday.

From there I expect to deploy only one more Release Candidate that contains various outstanding bug fixes and feature requests and then I'll freeze DocProject's feature set for Visual Studio 2005.  The following deployment, 1.12.0, should therefore be the first DocProject Production release.  Expect that in March, 2008.

Visual Studio 2005 vs. Visual Studio 2008

I expect to have my own licensed copy of VS 2008 Standard edition sometime before the end of February, but until then I've decided that I will not be concentrating on any new deployments for DocProject 2008 Beta, although I'm very excited about using VS 2008 and I'll certainly jump on that bandwagon as soon as I can.

So for now, you can expect development to continue like normal for VS 2005.  But once I start concentrating on VS 2008, I'm probably going to freeze the feature set for VS 2005, distribute the first Production release of DocProject 1.x, and then switch gears for .NET 3.5 and, possibly, VS Package development.  I intended to eventually release DocProject as a true package, hopefully with its own editors and actual project types [6] (i.e., DocProject and DocSite Templates [7] that are recognized by Visual Studio as distinct project types, with their own property pages, etc.).

After DocProject 1.x goes into stabilization (the end of phase 1) I'll certainly provide support for it and help people whenever I can if they have questions about how to modify the source code or how to add additional features themselves.  Although, the only official deployments that you'll see will probably be major releases that fix a large number of bugs.

Well that's my plan anyway because in the last couple of DocProject 2008 releases I've realized that maintaining two versions of the project is almost like double the work; however, since Visual Studio 2008 will allow me to target the .NET 2.0 Framework I may be able to work on both versions of DocProject simultaneously, without much additional effort.  If that's true then hopefully I won't feel the need to freeze the DocProject 1.x feature set.  Although, technically I can't really freeze DocProject's feature set anyway since it's open source ;)

DocProject Documentation

I plan to distribute compiled help for DocProject 1.x that consists of both API reference and conceptual documentation after the first Production release.  I'd say that a week or two should be enough time to author useful, preliminary documentation.  After that I'll probably just add content here and there and deploy whenever I reach certain milestones that I define on the fly.

A .chm will be downloadable from CodePlex and the installer will present you with the option to have DocProject's help content (.HxS) merged with Visual Studio's help.  I'm even considering hosting a DocSite as an online reference and as a great working example of DocProject's and Sandcastle's capabilities.

Phase 2

The next phase will pickup with the development of DocProject 2008, targeting Visual Studio 2008 and the .NET 3.5 Framework.  The timeline will probably start sometime in March, 2008.  From there I hope to deploy on a monthly basis as I've done with DocProject for the past year+ (since December, 2006 as a matter of fact).

Planning

Although DocProject, in the next release [4], will provide first-class support for building mixed reference and conceptual documentation, unbounded filtering capabilities, much improved performance and a mostly finalized API, I still see lots of room for improvement.  Namely, in the area of content-based features such as for authoring topics and localization.  Also, here's a few of the outstanding tasks [8] that I'm going to look into for DocProject 2008: 

• Running DocProject in areas of heightened security (e.g., Vista with UAC enabled).
• SQL Server full-text search provider for the DocSite templates.
• Microsoft Word and Adobe PDF output.
• The ability to annotate comments with developer notes. 
• Automatic token replacement.
• Some new DocSite skins, maybe including one that makes judicious use of SilverLight.  (Documentation might start looking better than the software it documents, if I achieve my goal, that is ;)
• Change control and better support for team scenarios; e.g., the ability to have a server that continuously builds live documentation to keep it up-to-date.
• The development of a simple community wiki for the DocSite templates, which I guess we can start calling the "DocSite Community Wiki" feature (sure, some might say it's a rip-off of the MSDN Community Content Wiki idea, and I guess they'd be correct ;).  This should be very useful in team scenarios for large, volatile, hosted documentation sets.
• The ability to register multiple Build Process Components [9] per project, with a management UI as well.  I may even turn the Sandcastle/Deployment [10] engine into a BPC that can be added to any DocProject or DocSite.
• Other various user-defined tasks (bug fixes and feature requests).

I have even more ideas that I've been tossing around for a while, so I hope to aggregate them into a blog post and eventually add them as work items in CodePlex.  And if you have any ideas for DocProject features please let me know!

Development

Development will target Visual Studio 2008 and the .NET 3.5 Framework, and as I mentioned previously, I'm definitely considering rewriting areas of DocProject as a true VS Package with its own custom editors and project types.  I'm also considering using WPF for the interfaces.  There will be a learning curve but it's one that I'm going to have to accept eventually (and I really do want to learn).

Collaboration

I expect to continue collaborating with community members to find bugs and DocProject's potential for new features.  It seems to me that the experience of DocProject users has been positive, so I'd like to continue working on DocProject in the future in the same way as I've been for the last year.  But if you feel differently please let me know.  Even though it's open source I'd still like to provide an acceptable level of service and support in my free time since the time that I put into helping end-users is also time that I'm putting into DocProject (even if it's just thought), which helps to make a better product even for me to use.

Extensibility

It might also be worth noting that I'm going to maintain the "openness" of DocProject by continuing to introduce new public APIs and areas for extensibility whenever possible.  With the limited feedback that I've received, I'd say that DocProject's high level of extensibility was a success.  A few people have mentioned to me that their businesses have created custom build engine providers [11] and build process components, and I've even used BPCs to help people debug issues in the past.  Actually, my favorite feature of DocProject might actually be the build process component, which, incidentally, doesn't really have anything in particular to do with documentation.

Deployment

It's hard to predict when the first Production release for DocProject 2008 will be deployed, but if I can use the past year as a metric then I'd say another year from now is certainly reasonable, and possibly even longer than might be required.  To be honest, I think the summer of 2008 is a reasonable goal as long as I keep my eyes on the prize.

Phase 3

At the start of phase 3 I hope to have already achieved my major goals for DocProject, which includes much better integration into Visual Studio and additional features that will make authoring documentation extremely simple and, in many cases, automated.  Thanks to Sandcastle I believe that this goal is attainable.

For the future I think I'm aiming for much better external support so that non-Visual Studio users will be able to benefit from DocProject's features as well.  The team-based features that I mentioned previously might get pushed into this phase, but we'll see how it goes.  And finally, I'd like to hear your ideas for where you'd like to see DocProject go.  After I'm able to meet most of my own goals I'd like to brainstorm with the community to come up with some new and innovative ideas to be a part of DocProject in the future.

References

[1] Sandcastle on CodePlex, http://www.codeplex.com/Sandcastle

[2] Microsoft Public License (Ms-PL), http://opensource.org/licenses/ms-pl.html

[3] CTP, License and Source Code, http://www.codeplex.com/Sandcastle/Thread/View.aspx?ThreadId=20557

[4] Dave Sexton's Blog: DocProject 1.10.0 RC Preview, http://davesexton.com/blog/blogs/blog/archive/2008/01/15/docproject-1-10-0-rc-preview.aspx

[5] Help authoring tool. (2008, January 9). In Wikipedia, The Free Encyclopedia. Retrieved 11:22, January 17, 2008, from http://en.wikipedia.org/w/index.php?title=Help_authoring_tool&oldid=183180723

[6] DocProject Work Items: VS 2008 Shell and Extensibility, http://www.codeplex.com/DocProject/WorkItem/View.aspx?WorkItemId=14029

[7] DocProject Components, http://www.codeplex.com/DocProject/Wiki/View.aspx?title=DocProject+Components

[8] DocProject Work Items, Advanced List, http://www.codeplex.com/DocProject/WorkItem/AdvancedList.aspx

[9] DocProject's Build Process, Build Process Components, http://www.codeplex.com/DocProject/Wiki/View.aspx?title=Build+Process#BuildProcessComponent

[10] DocProject Sandcastle/Deployment Plug-In, http://www.codeplex.com/DocProject/Wiki/View.aspx?title=Sandcastle+Deployment+Plugin

[11] DocProject tutorial: Creating a Build Engine Provider, http://www.codeplex.com/DocProject/Wiki/View.aspx?title=Creating+a+Build+Engine+Provider

In this post I'm going to provide details and screen shots of some of the new features in the DocProject 1.10.0 Release Candidate, which includes several bug fixes and new features such as first-class support for conceptual topics (additional content written in MAML markup), some new tool windows for Visual Studio Standard+, and increased performance of the Build Assembler step.

I plan to deploy after the next Sandcastle release, which will give me some more time for additional testing, to ensure compatibility and to finish a few incomplete features.  And BTW, I'm going to try to get back on my monthly deployment schedule again :)

Improved Performance

First, I'd like to start out by saying that performance is something I should have paid closer attention to throughout development, but at some point I just lost focus on it (probably when I started testing exclusively against smaller documentation sets).  So as Sandcastle's performance improved, DocProject's didn't.  I think I may have finally redeemed DocProject in this release though.

A new project option named, Build assembler options allows you to choose from various settings that have a direct affect on the performance of the Build Assembler step:

The options that you have chosen will be used whenever Build Assembler is executed, for all types of builds; e.g., reference, conceptual, help 1.x and 2.x.

The default for new projects is Trace errors only, which should provide the same performance as None for a build that does not fail, while providing diagnostic information if it does.

In testing, a substantial speed increase of approximately 8 times is normal when compared to building in Visual Studio and the External UI in the 1.9.0 RC.  The speed is now similar to that of building a DocProject or DocSite on the command-line with MSBuild.  Although, with informational and warning messages disabled there is no log or trace of the tasks that Build Assembler performs and the UI is not responsive while the Build Assembler step is executing.

Enabling Trace all and Cancelable will provide the same behavior as in previous versions of DocProject; i.e., all information and warnings are traced and the build may be canceled at any time.  And although there's an obvious performance penalty with these options enabled, there will also be a noticeable improvement of approximately 5 times compared to 1.9.0 RC.  The speed is now similar to the speed increase garnered by opening a modal dialog during the Build Assembler step in 1.9.0 RC, as per the discovery by Cis in this discussion.  So if you want the Build Assembler step in 1.10.0 to function the same as in 1.9.0, enable these two options and you should still see a speed increase of up to 5 times.

Build assembler options is located under the Build category in the DocProject Properties window.  It's implemented as an enum type with FlagsAttribute, EnumFlagsConverter and related classes (link is to the 1.9.0 RC codebase).

New Tool Windows

In preparation for first-class support of conceptual content I had previously written an XmlTreeView control that was included in DocProject 1.9.0 RC.  It extended the FCL TreeView control by adding support for internal and external drag & drop operations, an XML data source and a specialized binding manager.  In 1.10.0 RC it has been refactored and new features have been added so that it's now being used to its full extent in the Topic Management dialog (formerly known as the API Topic Management dialog) and the new Topic Explorer tool window (shown at right).

The new Topic Filters and Topic Editor tool windows were also extracted from the old API Topic Management dialog's Topic Filters and XML Comments tabs, respectively, so that they can be used without having to open a modal dialog (in VS Standard or higher).  Both windows are automatically synchronized to the selected topic in Topic Explorer.  They function as documents although since they're actually tool windows there can only be one instance of them opened at any given time, which means that they can only be used to manage one topic at a time.  Their appearance and functionality hasn't changed since DocProject 1.9.0 RC (preview post) so I'm not going to go into any more detail about them here.

Topic Explorer

The new Topic Explorer tool window is only available in Visual Studio Standard edition or higher.  It can be shown by pressing the Topic Management button on the Sandcastle toolbar and supports docking with other tool windows or floating on its own.  It functions the same as the tree view from the old API Topic Management dialog, with additional features.

Table of Contents

You can drag & drop topics to create the table of contents (TOC) that will be used in compiled help and the DocSite templates.  The TOC will be generated exactly as it appears in Topic Explorer.  Its layout is stored in a file named topics.xml, found in your project's Help\Settings folder.

When you save your changes by clicking Visual Studio's Save All command button or menu item, Topic Explorer will rewrite the topics.xml file based on the current organization of its tree.

Auto-Generated Reference Topics

The Namespaces topic, which is the root API reference topic generated by Sandcastle, starts out as a placeholder topic and loads in the background so that you can manage the TOC without having to wait for Sandcastle.  Once Sandcastle has generated the reflection XML document and the reference TOC, the placeholder is automatically replaced with the Namespaces node (for large documentation sets this may freeze the UI for a few seconds while the tree is being updated).

This node does not appear, however, if your project has no valid project references or external sources.

You can drag & drop the Namespaces topic or its placeholder topic at any time to place it within the TOC; however, drag & drop is not allowed for the individual reference topics.  Neither is dropping a conceptual topic anywhere within the hierarchy of the Namespaces topic.

When saved, the placement of the reference TOC is stored as a single <stoc /> element, which may appear anywhere within the Help\Settings\topics.xml file.

Editing Topics

To open the new Topic Editor window for any particular API reference topic, simply double-click the topic in Topic Explorer.  To edit a conceptual topic simply double-click the topic in Topic Explorer and it will be opened in Visual Studio's XML editor.

I really wanted to provide the same editing support for conceptual topics as is available currently for reference topics, however, I discovered after some experimentation that the MAML schemas are a bit too complex for my simple XmlContentEditor control, and not to mention that HTML is not currently passed through into the final HTML topic as of the Sandcastle Oct. CTP, which would make an HTML-based editor kind of pointless.

In the future I hope to create editors for some of the specific MAML schemas and make them as dynamic as possible.

Toolbar

Topic Explorer's toolbar provides the following functions (from left to right):

Button	Description
	Toggles expansion of all nodes.
	Opens or activates the Topic Filters tool window for the selected topic, which provides a document-like UI for the regular expression and categorical filters that are also present in the Topic Management dialog.  Note that you can also filter topics simply by unchecking them in Topic Explorer.
	Synchronizes the tree with the active project.  Topic Explorer is automatically synchronized with the selected DocProject or DocSite in Solution Explorer, although if the topics.xml file is manually updated then the refresh button must be clicked to update the display immediately.  (This should be automatic though in a subsequent release.)
	Creates a new container topic that does not have an associated file, although as of the Sandcastle Oct. 2007 CTP this doesn't appear to be supported so I'll probably just hide the button for now.
	Creates a new conceptual topic node as a child to the current node and asks the user which conceptual template should be used to create the file on disc.  The file is then opened in Visual Studio's XML editor and the node is placed into edit mode so that the label can be set.  The label is used as the topic's title.
	Imports existing XML topic files into the project.  Note that existing topics must use MAML to be supported by Sandcastle; however, if a topic does not use MAML then you're free to update it using Visual Studio's XML editor, manually, after it's imported.
	Deletes the selected topic after confirmation.  Reference topics cannot be deleted however.  If a container topic (with the folder icon) is currently selected then its entire hierarchy of topics, including itself, will be deleted from the TOC and disc.  If an individual topic is currently selected then only it will be deleted from the TOC and disc, and if it has child topics they will be promoted to children of the deleted topic's parent.

Sandcastle Build Component Editors

I've only actually attempted to create one editor at this time, but expect to see a few more by the time I deploy 1.10.0 RC.  The editor that I've created already is for ResolveReferenceLinksComponent2:

To open this editor simply click the ellipses button (...) of a Reference Link Types component in the DocProject Properties window:

MSDN hyperlink options

These options apply to all hyperlinks that are generated for the MSDN link type.

The Target window option indicates the target for all hyperlinks created by the component for MSDN topics online.  It's implemented as a combo box that accepts the name of a target window (i.e., any string) or one of the values from the drop down list: _blank, _parent, _self or _top.  The default is _blank, which means that clicking the hyperlink to an MSDN topic will open up a new window.

MSDN Locale indicates the default locale to use when generating links to MSDN topics online.  The combo box accepts a custom locale string or you can select one from the drop down list, which is automatically populated with a long list of known locales, none of which are guaranteed to have content on MSDN.  The default value is en-US.

XML reflection file targets

The grid lists each of the individual <targets /> elements of the component, one per row.  You can delete existing rows or add your own.

Base Path is simply concatenated, with respect to file path conventions, to the specified File Pattern.  If a Base Path isn't specified then the current directory is used (most likely being your project's, buildhelp\assembler directory).

File Pattern indicates the XML reflection files from which hyperlinks may be generated.

Recursive indicates whether the Base Path should be searched recursively for files that match the specified File Pattern.

Link Type must be one of the values from the drop down list: None, Local, Index or MSDN.  You can read about the different link types on the Sandcastle blog in this blog post.

Current Tasks

I'm still working on creating some of the Sandcastle Build Component editors and implementing a few work items.

Currently, I've been refactoring the behavior of state persistence for DocProject options.  In previous releases, options are committed immediately when a change is made in the DocProject Properties window and changes are buffered in the Tools Options page.  However, in 1.10.0 RC, options are committed only when the project is saved.  This is done manually by clicking Visual Studio's Save All command button or menu item (and the standard configuration of VS will automatically save the project before each build).  To support this feature, options have been branched into two categories (internally): project and custom.  Project options are those that read and write their state directly using the IAnyProject.Settings property, whereas custom options, for example, might instead persist state to some file.  If you are writing your own DocProjectOptions implementation then you must override the new Save method in order to save changes to custom options.  For your project options (the ones that use IAnyProject.Settings) you shouldn't have to change anything.  The Commit method was used previously for this purpose, but it also was used to commit buffered changes in other situations, without persisting them to disc.  This type of functionality is now only required by the Tools Options page; although, clicking OK on Visual Studio's Options dialog will no longer save the changes to disc - it will only cause Commit to be invoked.  When the project is actually saved, then Save will be called.  In the DocProject Properties window changes are committed immediately, so Commit is never actually called (although that may change - I haven't determined it yet).  To check whether your custom options should be commited immediately have your code read the DocProjectOptions.CommitChangesImmediately property, which is already implemented in 1.9.0 RC.

The purpose of all this is to make sure that the behavior when updating DocProject options is always the same, regardless of whether they are project options or custom options or whether they are being edited in the DocProject Properties window or the Tools Options page.  The difference from previous versions being that changes are committed immediately and only saved to disc when Visual Studio's Save All command is executed.

Conclusion

The next release of DocProject finally provides that long awaited, first-class support for conceptual content.  Now you'll be able to use Visual Studio to easily create, manage and automate the generation of Help 1.x and Help 2.x documentation, optionally in combination with auto-generated API reference documentation.  This feature substantially increases DocProject's potential as a HAT (help authoring tool) and provides a strong foundation for more content-based features in the future.

DocProject 2008 Beta 2 has been deployed to CodePlex.

• Compatible with Visual Studio 2008 RTM (tested against Team Suite Trial).

• Compatible with Visual C# 2008 Express and Visual Basic 2008 Express.

• Supports side-by-side installation with the DocProject 1.9.0 RC for Visual Studio 2005.

• Built using the same codebase as the 1.9.0 RC, with updates to support Visual Studio 2008, a few subtle changes that will also be included in the 1.10.0 RC, and some bug fixes.

Bug Fixes

1. Topic Management Error: Value cannot be null
2. Deleting a comment section does not mark the content item as dirty, so if it's the only change that's made it's not committed when the dialog is closed.
3. The Cancel and Back buttons cause control validation on the Wizard Form, but they shouldn't.
4. DocProjectEnvironment.InstallVSExtensions adds the same commands and tool windows to the commands and tool window collections even if they're already present.
5. DocProject Properties window may be added to the wrong Project command bar, depending upon the environment.

Known Issue

The Build Component Stack configuration options display Load Error messages. This issue is related to a problem in Visual Studio 2008 that has yet to be resolved; however, if you manually add Sandcastle's assemblies to the GAC then you will not experience this issue.

See my blog post about this issue for more information.

Side-By-Side Installation

DocProject 2008 Beta 2 will work side-by-side with DocProject 1.9.0 RC, although you must update new and existing 1.9.0 DocProjects and DocSites so that they use a specific version of the DaveSexton.DocProject assembly.

To ensure that your 1.9.0 RC projects continue to work, follow these steps after installing DocProject 2008 Beta 2:

1. Open an existing 1.9.0 RC DocProject or DocSite in Visual Studio 2005.
   1. If you need to create a new project instead then continue to the next step after completing DocProject's New Project Wizard.
2. Expand the References folder in Solution Explorer.
3. Right-mouse click the DaveSexton.DocProject assembly and select Properties.
4. Set Specific Version to True. The version number, which is read-only, should automatically change from 2.0.0.0 to 1.9.0.0.
5. Save the project.

As you probably already know, DocProject is kind of weak in the documentation department (that is, in its own API documentation :)  I plan to address this issue for the first RTW, after I include first-class support for generating conceptual content and after Sandcastle's first RTW.  If you are new to DocProject, the articles on CodePlex should provide more than enough information to get you started.

In a series of blog posts, with this post as the leader, I'm going to provide some supplementary information that you may find useful.  Tips and tricks that will add value to DocProject and allow you to make the most of it.  Code examples can be found at the end of posts and will have names like, Example #.

Community Tricksters

As the author, I probably know more about DocProject than anyone.  But as an end-user, I haven't taken advantage of the full range of its capabilities yet - including things like using DocProject with Team Build, building AJAX documentation or customizing Help 2.x output.  Therefore, I'd be happy to accept tips and tricks from the community that I can include in this series.  If you have ideas please submit them as blog comments or email them to me and I'll consider including them in my posts.  Thanks!

General-Purpose Tips

Here are some tips for DocProject that apply to all types of templates and build engines:

1. If you do not want to include build output as project items, and you don't want to be prompted after you build a project for the first time, then set Include Project Output Dialog | Don't ask me this again to True in the DocProject Properties window before you build.  You can leave the default values for the two related options, which should both be False.  (Note that the Apply to all default option is not used when Don't ask me this again is True.)
2. You can insert dynamic build steps using your project's Build Process Component (see Example 1 below).
3. Use an external or third-party Build Process Component:
   1. Open the DocProject Properties window (right-mouse click your project in Solution Explorer and select, DocProject Properties).  Visual Studio Express users must use the DocProject External UI.
   2. Locate the Extensibility | Process component type name option.
   3. Enter an assembly-qualified type name (the assembly must be located in the GAC or your project's output folder).
   4. Alternatively, you can enter a full type name and an absolute or relative assembly path, separated by a vertical bar; e.g., MyNamespace.MyType|..\anotherproj\bin\Debug\anotherproj.dll

DocSites Tips

These tips apply to the DocSite Templates only:

1. To change the DocSite logo simply replace the Images\Logo.jpg file with a file of your own.  To use a different file name, change the value of the ImageUrl attribute on the <asp:Image ... id="logoImage" ... /> element in the Controls\DocSiteHeader.ascx file.
2. To show rank information in DocSite search results, for the purpose of fine-tuning the search index, open the App_Themes\BasicBlue\DocSiteSearch.skin file and set Visible="True" on <asp:Label ... SkinId="searchResultRank" ... />.  When performing a search the rank value will appear before the title of each matching topic.  If you already have search results in your browser just refresh the page.
3. To change the DocSite title, company name and copyright, set the value of the AssemblyTitle, AssemblyCompany and AssemblyCopyright attributes, respectively, in the Properties\AssemblyInfo.(cs|vb) file and then restart the web application.  (The DocSite caches these values in memory upon the first request to the site.)

Sandcastle Tips

This section provides some useful tips for projects that use the Sandcastle or Sandcastle/Deployment build engine providers.

1. Set the Help 1.x Configuration File Name and Help 2.x Configuration File Name options to the same values and DocProject will only run Build Assembler once to build both Help 1.x and Help 2.x output.  The cost of doing this is that reference link types in one of the help output formats might not be valid (this was the main reason for splitting the Sandcastle configuration file into two distinct files in the first place.)
2. Use XML documentation files generated in the API Topic Management dialog for IntelliSense by adding a short code snippet in your DocProject's Build Process Component (see Example 2 below).
3. In your Build Process Component, you can get access to all of the Sandcastle-specific project options and APIs.
   1. Add a reference in your DocProject or DocSite to the DaveSexton.DocProject.Sandcastle.dll assembly, commonly found at C:\Program Files\Dave Sexton\DocProject\bin\.
   2. Check out the code in Example 3 below, which illustrates how to access the custom Sandcastle APIs that are provided by DocProject.
4. Insert images into your documentation using the XML documentation editor in the API Topic Management dialog:
   1. In design mode, place the text cursor where you want to insert an image.
   2. Click the button on the editor's tool strip or right-mouse click and select Insert > Picture.
   3. In the Picture dialog enter the full path and name of an existing image or click the Browse... button to locate one.
   4. You can enter a local file path or even a web address such as, http://example.com/someimage.jpg.  Click OK and the image is inserted.
   5. After you are done authoring your documentation, click OK on the API Topic Management dialog and DocProject will automatically import the image into your project's Help\Art folder.  It will also rebase the reference in the XML documentation to ..\Art\{image name}.
      1. Note: If you enter a web address then DocProject will copy the image from your web browser's cache; otherwise, the file is copied from the location that you specify.
   6. Note: The next time that you view the XML documentation in the API Topic Management dialog, the path will be rebased so that it's rooted, for display purposes only.  The path in your actual XML documentation on disc will remain relative, however.
5. Filter out <Module> topics when the Documentation scope option is set to Complete by adding a dynamic regular expression filter:
   1. Open the API Topic Management dialog.
   2. Select the Topic Filters tab.
   3. Select the Regular Expression tab.
   4. Enter the following into the Pattern text box: \<Module\>
   5. Click the Save button.  The Saved Filters tab will be shown automatically and your filter will appear in the list.
   6. For the new filter, ensure that the Build column is checked and that the Include column is unchecked.
   7. Enter a short description of this filter into the Memo field (optional).
   8. Click OK on the API Topic Management dialog to commit your changes.

C# Code Examples

The following examples illustrate points made in the information above.  They are referenced by number.

Example 1:

The following example shows how to inject a dynamic build step immediately before the Help 1.x compiler step using your project's Build Process Component.  The step will be executed in full and partial builds and will run on a background thread.  It outputs some text in the build trace window:

Step # of ##: Hello

World!

To use this example, replace the BuildStarting method in your project's BuildProcess.(cs|vb) file.  (Keep the existing code though).

   1: protected override void BuildStarting(BuildContext context)

   2: {

   3:   // existing code goes here 

   4:  

   5:   base.InsertBeforeBuildStep("Compile Help 1.x", "Hello", true, false, true,

   6:     delegate(BuildContext stepContext)

   7:     {

   8:       stepContext.TraceLine("World!");

   9:     });

  10: }

Example 2:

The following example will overwrite the source XML documentation file (that contains your code comments) with the merged XML documentation file (that contains both your external documentation and the source code comments) after each help build completes.

To use this example, replace the BuildCompleted method in your project's BuildProcess.(cs|vb) file.  (The existing code should be placed at the end).  You must also add a using System.IO declaration at the top of the file.

   1: protected override void BuildCompleted(BuildContext context)

   2: {

   3:   TraceLine();

   4:   TraceLine("Overwriting XML documentation files in source projects..."); 

   5:  

   6:   IDocProject project = context.Engine.Project;

   7:   BuildSettings settings = context.Engine.Settings; 

   8:  

   9:   foreach (ISourceProject source in project.Sources)

  10:   {

  11:     string target = source.Output.XmlDocumentationFile; 

  12:  

  13:     FileInfo autoGenFile = new FileInfo(Path.Combine(

  14:       settings.WorkingXmlDocumentationPath, 

  15:       Path.GetFileName(target))); 

  16:  

  17:     if (autoGenFile.Exists)

  18:       autoGenFile.CopyTo(target, true);  // overwrite

  19:   } 

  20:  

  21:   TraceLine("Done."); 

  22:  

  23:   // existing code goes here

  24: }

Example 3:

You can access Sandcastle-specific project options and APIs in your Build Process Components by adding a reference to the DaveSexton.DocProject.Sandcastle.dll assembly.  The following example shows how to obtain the SandcastleBuildEngine instance, and from that retrieve the SandcastleSettings and SandcastleProjectOptions:

   1: SandcastleBuildEngine engine = (SandcastleBuildEngine) context.Engine;

   2: SandcastleSettings settings = engine.Settings;

   3: SandcastleProjectOptions options = engine.Options;

The context is an argument that is passed to all of the four main Build Process Component overrides: BuildStarting, BeforeExecuteStep, AfterExecuteStep and BuildCompleted.