A couple more public DocSites have come to my attention and I thought I'd share them with my readership. You can read my original DocSites Online post here.

If you haven't been paying attention and you're not sure what a DocSite is, it's an AJAX-enabled ASP.NET Web Application project in Visual Studio 2005/2008 that provides a web interface for API documentation. The documentation can be built using Microsoft's Sandcastle utilities from managed assemblies and optional XML documentation files that contain your triple-slash code comments. DocSite templates come in two flavors: C# and VB.NET (although either template can build help for projects written in other languages, including managed C++).

DocSite templates are available as part of DocProject.

MbUnit 2.4

Ben Hall provided some great feedback on DocProject and has done an excellent job creating his DocSite at http://docs.mbunit.com/, which includes a custom TOC, topics and appearance.

According to the docs…

MbUnit is an advanced, extensible unit testing framework originally developed by Jonathon 'Peli' de Halleux and Jamie Cansdale.

Ben posted about the documentation's release at http://blog.benhall.me.uk/2007/08/mbunit-new-website-and-documentation.html.

You can read about the MbUnit project at: http://www.mbunit.com/.

Interactive Brokers C# API

Dinosaur Technology and Trading has ported a Java socket interface to C# for the Interactive Brokers API.

Their DocSite can be found at http://ibhelp.dinosaurtech.com/default.aspx and you can read about the utility at http://www.dinosaurtech.com/utilities/.

Nice work guys.

Style

The API topics for each DocSite were built using Sandcastle's Hana presentation style, which adds a number of improvements over MSDN's style; however, it seems that the final VS 2008 (Orcas) documentation will probably still use the VS 2005 style according to a post at Sandcastle's blog.

You can read about the Hana style here.

The latest version of DocProject has been deployed.  Also, there is a new version called DocProject 2008, currently in beta, that provides support for Visual Studio 2008 Beta 2, Visual C# 2008 Express Beta and Visual Basic 2008 Express Beta.

You can find DocProject at http://www.codeplex.com/DocProject.

It's about that time again…

Just FYI, DocProject 1.6.2 RC had more downloads in the last month than any other version to date. Thanks to everyone who tried it and provided valuable feedback!

And BTW, I wrote a new article called How to Diagnose and Resolve Issues that provides guidance for gathering diagnostic information if you're having issues with DocProject and would like to try to solve the problem yourself or if you would like to ask for assistance.  You can find other How To... articles here.

In the next version, 1.7.0, there are a number of really cool features that I think people will like, as well as some bug fixes that should solve the problem with Team Build and provide a better experience using the HTML Editor. In this blog post I'll show a preview of some of the new features and describe my plans for them in the future.

Like usual, please let me know what you think so that with your support I can continue to make DocProject a useful community tool.

New Features

• XML Documentation Editor
• API Topic (Shared Content) Designer Improvements
• Missing Dependencies List
• External UI (DocProject.exe)

I'm still testing and finishing up some other tasks, features and bugs. I'm running a bit late since I was hoping for a mid-July release, but a few of the features that I will describe below somehow snuck into this release (that seems to happen a lot with me ;). I'm also quite busy with work but I think I might be able to manage a deployment next week, without any showstoppers popping up. So far, things are looking good though.

XML Documentation Editor

The API Topic Management Dialog now supports authoring XML documentation (summary, remarks and example) for all API members, not just the project and namespace summaries. See Figure 1 below.

I've created a new control named, ContentEditor and another control named, ContentListEditor, which is used for the new and improved content dialogs: API Topic Management (Figure 1) and the API Topic Designer (Figure 4). As you can see in both figures, there are now two tabs at the bottom for switching between design-mode and source-mode and a list of items that can be edited. Only one item can be edited at a time by selecting it in the list. After editing an item its name becomes bolded in the list until you save your changes.

Note: The content editor control actually provides another tab named, Preview, which raises an event that controls can handle to provide read-only HTML based on the changes made in the editor. Because this feature has not been implemented yet the Preview tab is currently hidden at runtime.

Figure 1: API Topic Management – Xml Documentation Editor

Xml Documentation

Remember that this is an xml documentation editor, which means that tags such as see, c and code may be used in Source mode even though they won't be formatted in Design mode. Sandcastle uses these tags when it builds your documentation to apply special formatting or to link to other topics.

One feature that I plan on adding in the future is the ability to drag nodes from the tree onto the designer and have it serialize <see cref="{id}">name</see> automatically. I'd also like to show see links as hyperlinks in the designer, but maybe as green or using double-underline to distinguish them from normal hyperlinks. Another possibility is to use the stylesheets from your DocProject's or DocSite's selected presentation style and then format all of the xml documentation tags in the designer as they will appear in the compiled documentation.

Another feature that I plan on adding is API type-specific sections in the item list. In other words, if you select a method node to edit then the editor should automatically list each of the method's parameters next to summary, remarks and example. Tags such as permissions and seealso should be listed as well, although I'd like to open a custom editor interface for these tags.

Currently, you can add custom tags to the list using DocProject's configuration file (DaveSexton.DocProject.dll.config, normally found in C:\Program Files\Dave Sexton\DocProject\bin), but they'll apply to every API element. I'd like to implement a feature that allows you to specify which sections apply to which API elements though (probably using DocProject's configuration file, partially shown in Figure 2 below).

Note: Do not confuse the DaveSexton.DocProject.dll.config file with the DocProject.exe.config file (I'll explain more on that below) located in the same directory.

Figure 2: DaveSexton.DocProject.dll.config xmlDocumentationTags attribute

HTML to XML Conversion

I've made a few improvements to my HTML Editor control and the way HTML is converted to XML (a few bug fixes).

The HTML Editor control that you see in Figure 1, which you may already be familiar with if you've used DocProject 1.6.0, is based on the WebBrowser control. The WebBrowser control is basically a managed wrapper around Internet Explorer. For the HTML Editor it's a wrapper around IE's design mode, which only produces HTML markup (in some cases, very poor HTML markup.) Sandcastle requires valid XML, so DocProject converts your HTML to XML automatically. The results of the conversion can be viewed in Source mode. The markup that you enter in Source mode is also converted automatically, so don't be surprised if when you forget to add a closing p tag, DocProject adds it for you when switching views or saving your changes :)

The HTML Editor is a work-in-progress, so I expect issues with HTML to XML conversions to spring up every now and then. Please let me know if you experience an error. If you can provide me with the exact steps to reproduce the problem and the markup that you used, I'll do my best to fix parser bugs and improve the editor's usefulness.

Persistence

Your comments are saved in individual xml files in your DocProject's or DocSite's Comments folder, at the root of the project. An {assembly name}.xml file is created for each assembly and one project.xml file is created to contain the project and namespace summaries. If you prefer you can edit these files using Visual Studio's XML editor. If you haven't edited any API elements for a particular assembly then its XML file will not be present in the Comments folder, but you can create it yourself. The name of an assembly's xml documentation file must be the display name of the assembly with .xml appended to the end or else DocProject will not be able to merge your documentation with imported documentation.

Merging

XML documentation that appears in your code is not shown in this dialog. By default, comments created in the dialog overwrite code comments, on a section by section basis; e.g., summary, remarks or example. But you can change this setting to give code comments precedence instead of being overwritten by selecting KeepSource for the Merge xml documentation setting found in Tools > Options > DocProject > Active Projects > Content.

Figure 3: Merge xml documentation options for documentation written in the API Topic Management dialog

As you can see in the figure above, the API Topic Management dialog can be accessed from the option immediately below the Merge xml documentation option.

Xml Documentation Clean-up

If you create documentation for a type that is later removed from your project then the documentation will still remain in the XML file for the assembly in which the API element is defined, however it will no longer be displayed in the API Topic Management dialog so you must remove it from the file manually. In the future I may add a feature that will allow you to clean up your xml documentation files at the click of a button.

API Topic (Shared Content) Designer Improvements

The API Topic Designer uses the new content list editor control and provides a list of content items that you can edit, such as the header, footer and various titles.   The items are loaded from the Presentation\Style\Content\shared_content.xml file in your DocProject or DocSite.  Changes to the items using the dialog are written directly to this file as well when they are saved.  Deleting an item in the file will remove it from the list and adding an item to the file will add it to the list the next time the dialog is opened.

Figure 4: API Topic Designer in 1.7.0

Just like in 1.6.0, the API Topic Designer appears as a step in the New Project Wizard with all of the new features.  The header section is the default item selected.

As of the June 2007 CTP release of Sandcastle, presentations have a hard-coded header and footer that will probably be inappropriate for your projects.  I've decided not to handle this in DocProject so you must delete the header and footer manually if you do not want them to appear in your documentation.  Although the New Project Wizard is a good place to do that, you can still update the header and footer at a later time using the API Topic Designer.

FYI, I've already requested in the forums that the Sandcastle team set these values to empty in the default presentations to save effort on behalf of end-users.

Text-only Content Items

Not all of the items are XML-editable using the API Topic Designer. Most are text-only since there will probably not be a need to use HTML formatting for them. Text-only items only show the Source tab and will escape any XML markup that you enter. To configure which items use the HTML editor on a per-presentation basis you must edit DocProject's configuration file (DaveSexton.DocProject.dll.config, normally found in C:\Program Files\Dave Sexton\DocProject\bin):

Figure 5: DaveSexton.DocProject.dll.config sharedContentXmlItems attribute

Missing Dependencies List

A while back, around version 1.3.0, I attempted to build documentation for DocProject using DocProject itself but I quickly found that a bug in Sandcastle's MRefBuilder program (actually, CCI) relating to binding redirection prevented me from doing so. In the last version, 1.6.2, I decided to try again but this time I found that DocProject's ReferenceResolver class wasn't able to resolve the Interop.MSHelpCompiler.dll reference (provides the Help 2.x compiler services) for the Sandcastle project (DaveSexton.DocProject.Sandcastle.dll). The problem is that DocProject's Sandcastle assembly is being resolved in the GAC, but the MSHelpCompiler interoperability assembly is not installed in the GAC, so it cannot be found. I do plan to fix this bug in a future release, but since I've expierenced this issue myself I assume that there may be other scenarios that cannot be handled automatically by the reference resolver.  This features was added so that users can manually provide the path to dependencies that cannot be resolved automatically.

FYI, nobody has actually mentioned any problems with DocProject's automatic discovery of source dependencies so I assume that this feature will be used quite rarely, if at all. But then again, it must be used by DocProject itself just to build its own documentation ;)

I suggest that you do not use this feature unless you get an error when building. The error will be produced by Sandcastle's MRefBuilder utility and will state something about one or more unresolved dependencies, mentioning the full assembly name as well. From the information in the build output window you should be able to locate and add the missing dependency yourself using the new dialog:

Figure 6: Missing Dependencies List

This dialog is similar to the External Sources dialog that was added to DocProject 1.6.2 RC. You can find both dialogs in Tools > Options > DocProject > Active Projects > Build.

To add an assembly (.dll or .exe that cannot be resolved automatically by DocProject) to the list you must enter the full path and name into the text box in the new row (the row with the *) or click the button with the ellipses on the right side to select a file. You can also edit existing rows or delete rows by clicking the row header and pressing the Delete key.

The button on the left side of each row will map an absolute path to a path that is relative from the project's root directory, like the example path in Figure 6. This is useful for team and build-server scenarios.

External UI

DocProject 1.6.0 provided preliminary support for Visual C# Express and Visual Basic Express editions of Visual Studio. Unfortunately, because add-ins are only supported by Visual Studio Standard and higher, Express users had to manually edit the project files of DocProjects and DocSites to configure them after using the New Project Wizard since all of DocProject's options and dialogs are hosted by the add-in.

DocProject 1.7.0 RC has an external UI that uses a property grid, just like DocProject's Active Projects tools options page, so Express users can configure their DocProjects and DocSites using the same tools as Visual Studio Standard+ users: 

Figure 7: DocProject.exe

For Standard+ users, though this feature may be handy at times, I still recommend using Visual Studio in most situations to configure your DocProjects (via the tools options pages) and to build them in Visual Studio since it will automatically build dependency projects for you. Also, the Include Project Output Dialog does not work in the external UI and if you add an image to the XML documentation editor or the API Topic Designer while using the external UI, the image will not be included as a project item in Visual Studio (although it will still be imported into the Art folder - you'll have to Show All Files and include it manually).

Automatically Open the Active DocProject

You can add a link to the program in Visual Studio for quick access by going to Tools > External Tools... and clicking Add (this can be done in Visual Studio Standard and higher as well if desired).

By specifying the arguments in Figure 8 below, when you run the External UI from Visual Studio's Tools menu the program will automatically open the active DocProject or DocSite in Solution Explorer (that is, the project that contains the current file that is opened or the current file/folder that is selected in Solution Explorer).

The DocProjectBuildPath environment variable used in Figure 8 is registered on your system when you install DocProject.  It points to DocProject's bin directory, commonly found at C:\Program Files\Dave Sexton\DocProject\bin.

Figure 8: DocProject.exe - Visual Studio Tools Registration

Building DocProjects and DocSites

While developing this program I realized that it would be quite easy to build a DocProject from this interface using DocProject's BuildController class, so I gave it a shot. It actually worked so well that I now build my test DocProjects and DocSites using the GUI for debugging purposes. Breakpoints in code being hosted by Visual Studio's process tend to time-out after a few seconds of inactivity, due to COM issues, but since DocProject.exe is fully-managed it makes it much easier to attach, debug and continue, without having to restart Visual Studio and load up an entire solution.

Figure 9: DocProject.exe build trace

DocProject.exe will not only build your DocProject's or DocSite's help, but will also build the project's assembly output using MSBuild first (remember, DocProjects and DocSites are MSBuild projects). During a build you'll see descriptive status information describing the current step and the progress bar, just like you see in Visual Studio. You can even cancel builds at any time.

The program will also remember some settings for when you restart, such as the last position, size and window state, with multi-monitor support, the position of the toolbars and the height of the build trace window (Figure 9).

Currently, this program does not support DocProjects and DocSites that have managed C++ projects as sources since they can only be built using a solution file, but I plan to look into implementing that feature in a future version of DocProject.

DocProject.exe Extensibility

The code that is used to configure DocProjects and DocSites in Visual Studio Standard+ is the same code that is executed by the external UI. This means that if you have created a custom build engine provider for your organization it will automatically work with the external UI. Not only does this include the options in your DocProjectOptions implementation, which will appear as new properties in the grid, but also any Visual Studio toolbars that are created by your provider will appear as well. For example, in Figure 7 above you can see the Sandcastle toolbar that appears in Visual Studio Standard and higher in the DocProject UI (the right-most toolbar).

Sandcastle and Sandcastle/Deployment are examples of build engine providers that are registered in DocProject's configuration file (DaveSexton.DocProject.dll.config, commonly found in C:\Program Files\Dave Sexton\DocProject\bin).  Several of the options that appear in the figures above, including the API Topic Designer and Management Dialogs are provided by the Sandcastle build engine provider plug-in.

The next major How To... article on my to-do list is guidance for creating custom providers that can be used across projects in an organization, but if you feel like testing the waters yourself check out the DaveSexton.DocProject.Sandcastle and DaveSexton.DocProject.DeploymentSandcastle projects in the source code and try creating your own plug-in.

Conclusion

With the 1.7.0 RC, Visual Studio Express users will be able to do much more with DocProject than they could before with the new VS-like external UI. As a documentation author, you'll be able to write long examples and remarks sections using an HTML Editor and store all of the comments in your DocProject's Comments folder, which is merged with your xml code comments automatically before each build. You'll also be able to use the HTML editor to design topics' shared content, such as the header and footer – like before – but with the addition of every other item that appears in Sandcastle's shared_content.xml file as well. And now you can configure your DocProjects and DocSites to reference dependencies that could not be resolved automatically, however rare that might be.

I hope you like the new features and I look forward to receiving your feedback.

Recently I've been receiving more feedback than usual about the addition of non-API topics when generating compiled help with DocProject (also referred to as custom, additional or conceptual topics). In this blog post I'm going to describe DocProject's current support for conceptual content and my goals for better integration in the future.

Community Feedback

First, I want to say thanks to the users that have emailed me, added feature requests to the issue tracker or started discussions. It's been very helpful.

Special thanks goes to Ben Hall, a computer science student in the UK and also a Microsoft Student Partner (MSP), for his great blog post on how to extend my custom topics tutorial (discussed below in Creating Custom Topics with Partial Builds) to make it work in a more automated fashion.

Ben has also written another blog post on DocProject in which he discusses his experience with DocProject, and for which I have a few responses (a bit-off topic for this blog post though, sorry):

• I know that developers can give up easily sometimes, myself included, when an open source tool doesn't work correctly the first time around. But as you've discovered there can be problems with Visual Studio that I just can't account for in DocProject so patience and a bit of effort is sometimes required to get things going. So thanks for trying DocProject again :)
• You mention that all of the settings are saved in the project.xml file, but actually the settings are saved in the project file; e.g., .csproj or .vbproj. The project.xml file is simply used for the project and namespace summaries (but that will be changing in a subsequent release - I'm going to give each namespace summary its own file.)
• Technically speaking, DocSites don't have a search – they have an index filter. I've already submitted a feature request in the issue tracker for a DocSite search, which I'm currently working on and have been planning for a while now. The indexer and search engine are being written from the ground-up, but are nowhere near completion yet. I also plan to provide an administration screen for managing the index and viewing statistics. The search will be in addition to the filter and will appear in the top-right corner of the DocSite template, just below the link to download the compiled help file.
• The TOC icons, as you mentioned, can be changed easily since it's just an ASP.NET TreeView control. To change the icons open the DocSiteContents.ascx file and change the ImageSet attribute's value from WindowsHelp to Msdn, for example. Maybe I'll change the default in the next release :)

Thanks again Ben! And now back to our scheduled programming…

Custom Topics - But other tools are doing it!

DocProject supports the addition of conceptual content through its partial-build functionality, which I'll explain below. I'm aware that this may not be ideal for most authors.

If you've used other community projects that automate Sandcastle you may have noticed one important difference between them and DocProject: Other tools provide extensions to Sandcastle, such as Eric Woodruff's Sandcastle Help File Builder (SHFB), which apparently has support for additional topics, custom build components such as one that formats code examples, and even patches for some of Sandcastle's presentation bugs. I think it's really great that Eric has done all of that work, but my philosophy for DocProject is different (and no, it's not to slack off ;).

First and foremost, DocProject's purpose is the integration of Sandcastle into Visual Studio 2005 and to expose its core functionality through easy-to-use dialogs and the New Project Wizard. DocProject does not provide any custom build components or bug fixes for Sandcastle, and does not add any additional functionality to Sandcastle's standard build process. DocProject exposes Sandcastle's configuration and transformation files to end-users with minimal changes so that they can customize Sandcastle without being restricted to GUIs, the same as on the command-line without using DocProject or any other automation tools.

A feature that allows users to add additional non-API topics may require a custom build component, custom configuration file or custom transformation files to be shipped with DocProject, possibly breaking DocProject's open-Sandcastle-source policy. In addition, or alternatively, DocProject could make changes to Sandcastle's intermediary files during the help-build process to support the addition of conceptual content, but IMO that would be a hack. Sandcastle already supports conceptual content for Microsoft's internal use according to this blog post – it just hasn't been exposed to the community yet.

As soon as Sandcastle provides a standardized and supported way to include conceptual content in the canonical build process, I'll begin work so that DocProject will support it too.

If Sandcastle goes RTW before providing this functionality, which to be perfectly honest is what I've expected for a while now, then I will attempt to extend DocProject so that it supports the addition of conceptual topics without Sandcastle's support. But let's hope Microsoft realizes how important it is that Sandcastle provides this functionality before it is RTW.

I also plan to offer a Custom Topic Management Dialog that will allow users to manage their conceptual content and even create new topics based on an HTML template, specific for the Sandcastle presentation chosen for DocProjects and DocSites.

Once DocProject fully supports the addition and management of conceptual content it will be a practical solution for generating compiled help in Visual Studio 2005, with designer support, and even without the need to build API documentation.

If you haven't already then hopefully now you see why DocProject does not support the addition of custom topics in the same way as the other community tools.

Creating Custom Topics with Partial Builds

When I first started getting some feedback from the community about conceptual content, not too long ago, I decided to write a tutorial that would help users take advantage of the partial-build functionality offered by DocProject. The tutorial is meant for anyone that uses DocProject, but is especially useful for help content authors that are already familiar with Html Help Workshop's file formats and manually editing them to add additional content. However, this tutorial doesn't describe the end of DocProject's support for custom content, but currently it's the easiest way without having to modify DocProject's code-base.

Partial Builds

To see a partial build in action, load up a DocProject or DocSite and build it as you normally would, but make sure that it will output compiled help (configure your project to produce Help 1.x, for example). After it builds successfully, delete the compiled help file (.chm, for example) and try building the project again. Notice that there is only 1 build step this time since DocProject detects that the file dependencies used to generate the compiled help are still in-tact and that none of the source projects have changed. Only the compiled help file is rebuilt, forgoing the entire Sandcastle build process.

This allows authors to modify the Workshop TOC file manually (.hhc) for the addition of conceptual content and DocProject will just rebuild the compiled help, which will include your changes. If you're using a DocSite template then the DocSiteContents.xml file will be synchronized automatically during a partial build, so your changes to the Workshop TOC will appear in the DocSite's TOC as well.

Conclusion

DocProject does support the addition of conceptual content, but not in the way that most end-users desire or expect since it's not entirely automated. The lack of complete automation is because Sandcastle does not provide a standardized means yet for adding conceptual content, but once it does DocProject will integrate the process into Visual Studio 2005, without hiding the implementation details from end-users. The process should still be flexible and customizable to a certain extent. DocProject, as with several other features of Sandcastle, will provide a management dialog for authors to add, edit, delete and organize their conceptual content. As such, DocProject may eventually be a productive environment for creating and managing help topics and building compiled help inside Visual Studio 2005, even without API topics to be documented.

I'd like to know your thoughts on this so please leave a comment here, start a discussion on CodePlex or contact me using my site's contact form. Thanks!

I was browsing the web the other day looking for community feedback on DocProject and I was pleased to come across two public websites that use DocSite templates to provide public API documentation over the web.

XML Library API

The first DocSite that I found is at http://www.xmllab.net/mvpxml. It was created by Microsoft MVPs in XML technologies for their XML Library API on CodePlex. Nice work guys :)

Full Motion Racing

I stumbled across another DocSite at http://lfs.fullmotionracing.com/ for a project called, LFSLib.NET - A Live For Speed InSim/OutSim/OutGauge API. More information about the project can be found at the author's blog: http://www.claassen.net/geek/blog/.  The latest post even mentions DocProject: LFSLib.NET 0.13b released.  Thanks!

Others?

If anyone else has a public DocSite that they'd like to share with me please let me know :)

DocProject 1.6.2 was released and is now available for download on CodePlex.

I want to thank all of DocProject's users that took the time to report bugs and provide diagnostic information to me. You've been helpful.

Recently, Anand blogged about DocProject and some of the other community projects that can automate Sandcastle to generate help for managed APIs. After his post, DocProject's rate of download increased by three times and then kept increasing. And since then I've certainly received more feedback than usual. Thanks Anand!

Updates

The latest release addresses some issues that users were reporting relating to the refresh version of the June 2007 CTP of Sandcastle, which was released a few days after its original release. I would have updated DocProject sooner if I knew :)

There is also support now for adding external sources (assemblies and corresponding xml documentation). This new feature also provides support for Web Site Projects since you must use a Web Deployment Project to generate a managed assembly, which can be added as an external source for your DocProject or DocSite.

In Progress

Currently, I'm looking at providing the ability to create and manage custom topics. I noticed that in Sandcastle's June 2007 CTP refresh there are some transformations and files related to conceptual topics. That sounds interesting to me and may be the feature that people have been asking me to add. I'll definitely research this more.

In this blog post you'll see and read about some of the new features of DocProject that will be released with the 1.6.0 Release Candidate. But first, I'd just like to say thanks to everyone who downloaded DocProject and provided feedback. I appreciate it. Please keep the comments and feedback coming!

If you're only interested in screen shots – scroll down :)

1.6.0 Changes

This is one of the biggest releases yet, IMO. Much of the code-base has been refactored to support new features and to improve existing ones. Changes were made not only to the source code, but also to the solution and project files, and to almost every aspect of DocProject. There were also several new additions to the code-base in the form of source code and new projects. But don't worry; the end-user experience has not changed other than to include new features and improve existing ones. And the source code, for the most part, is still basically the same. Your existing knowledge of how to use DocProject will still apply, but it will no longer be the whole story.

Please note that existing DocProjects and DocSites may no longer work as expected. You are encouraged (if not required) to create new documentation projects again by removing existing ones and using the New Project Wizard to recreate them. I have not tested older projects with DocProject 1.6.0, but I suspect that they will not be able to complete a help-build.

Solution and Projects

The first change to the solution was to start using solution folders to categorize the projects. The Sandcastle code-base was then pulled out of the Add-In project (which has been renamed) and included in a new project named, DaveSexton.DocProject.Sandcastle, which appears under the Plug-Ins solution folder along with the Sandcastle/Deployment project. There is also a new project named, DaveSexton.Controls.HtmlEditor. I'll leave that one alone for now and let you figure out its purpose for yourself ;) There have also been several changes to the InstallPrep and Installer projects, although existing functionality has not really been removed or altered, for the most part. The preparation process is still the same as it was before, but with some additional files and requirements.

Building the Installer

If you're planning on building the Installer project then the only concern you should have is that the HtmlEditor project requires the Microsoft.mshtml PIA to be installed on the target machine, so the installer requires a bootstrapper to be built (Setup.exe) that includes the redistributable vs_piaredist.exe file, which installs the required dependencies. In order to include MSHtml in the bootstrapper I created a prerequisite bootstrapper package that is included in the InstallPrep project under the Bootstrapper folder, the contents of which should be copied to Visual Studio's bootstrapper directory manually. The Readme.txt file that appears in the same folder explains its use and also provides import legal information that you must agree to before using DocProject's source code. The agreement is also presented in a dialog when DocProject is installed, but only if the required dependencies must be installed on your system. You should read this file before attempting to build and redistribute the installer and the source code.

Although I haven't tested yet whether building the installer will fail without having the MSHtml bootstrapper package present on your system, I assume that it will. I'll test this to make sure and update the appropriate wikis on CodePlex to indicate end-user requirements to build the installer using the source code, but for now just assume that you must follow the instructions in the Readme.txt file.

1.6.0 Features

Here's a preview of some of the new features that you can expect in DocProject 1.6.0 RC. I'm still testing some of them on Vista, XP, and Windows Server 2003, but things are going well. I expect to deploy by Monday June 4, 2007.

Installation

The installer will now detect the following versions of Visual Studio on your system and automatically install the templates and Add-In, where appropriate, for both Just Me and Everyone installations. Having more than one version of Visual Studio on your system at the same time is acceptable since they can all share the same templates or have their own copies if you install for everyone:

• Visual Studio 2005 Standard+
  • The Add-In and all templates are installed.
• Visual C# 2005 Express
  • Only the C# Project Documentation template is installed.
• Visual Basic 2005 Express
  • Only the VB.NET Project Documentation template is installed.

Note: Currently, the DocSite templates will not work in Visual Web Developer 2005 Express.

The DocProject installation directory will always have the same contents regardless of what versions of Visual Studio are on your system when you install DocProject.  The only variance based on the different versions of Visual Studio are updates to the registry and the location where the Add-In and/or templates are copied.

There is only limited support for express editions, which I'll detail in the release notes for the 1.6.0 RC. Basically, express editions get the New Project Wizard and use the MSBuild task, but projects in express editions do not have Add-In-specific features such as the Include Project Output Dialog and the Tools > Options > DocProject pages. Users can still configure DocProject by editing the project file manually. However, Sandcastle is still configurable in the same way as the full version of DocProject; i.e., by editing files in the Presentation folder inside Visual Studio. Projects are also built in the same way, with support for a build process component as well.

Visual C++ Source Projects

DocProject now provides support for Visual C++ projects as sources for DocProjects and DocSites. Reference Visual C++ projects as you would any other project reference and their APIs will be included in the compiled help.

Html Files Names: GUID or Friendly

By default, the Sandcastle Build Assembler utility, which generates the HTML topic files, is now configured to use GUID file names instead of friendly names that corresponded to the names of the API topics, as used previously. This allows projects that have long, nested directory structures to build without throwing a System.IO.PathTooLongException (special thanks to the people who pointed this out to me :). You can change a setting to use the friendly names if you want: Tools > Options > DocProject > Active Projects > Build > Use friendly HTML file names (disabled, by default).

MSBuild Task and Command-Line Support

The template files for DocProjects and DocSites have been refactored to invoke a custom target named, BuildHelp, which references the DaveSexton.DocProject.targets file.  The targets file, located in DocProject's bin folder, starts the assembly-build process and then, afterwards, starts the help-buld process, which is executed by the Add-In in much the same way that it has in previous versions of DocProject.

To support this feature the installer creates an environment variable named, DocProjectBuildPath, which points to DocProject's bin directory. The environment variable is created with user-scope if you select Just Me in the installer and system-scope if you select Everyone.

The MSBuild task simply delegates the help-build process to the DocProject Add-In, which has been refactored so that it can execute outside of the Visual Studio IDE as well as inside. In other words, MSBuild is now used to start the build process and DocProject's Add-In takes over, using the appropriate method depending upon whether it has been initialized as a Visual Studio Add-In or simply invoked out-of-process by the MSBuild engine. This means that the Add-In will build your project files when they are used as Visual Studio projects, as they always have been, but also on build servers that use MSBuild - without running Visual Studio. No changes to the DocProject or DocSite project files are required for them to be buildable in any environment.

You can build DocProjects and DocSites on the command-line by simply running MSBuild and specifying the project file as an argument. For example, open the Visual Studio Command Prompt and enter the following to build a C# DocProject named, "your project":

msbuild.exe "c:\your project.csproj"

Note: Visual C++ projects must be built by the solution file. MSBuild will warn you about this if you attempt to build DocProjects or DocSites that reference a Visual C++ project. In that case, just pass the solution file to msbuild.exe.

Your DocProjects and DocSites can be specified as targets in other MSBuild project files on a build server as long as DocProject has been installed on the server (or at least with the DocProject assemblies and appropriate configuration present, even if DocProject is not actually installed). Make sure that the relative references to other projects in the DocProject or DocSite project file are valid.

New Project Wizard Header

The first change that you'll notice when creating a new DocProject or DocSite is in the New Project Wizard's header, modified slightly just to keep things interesting ;) (See Figure 1 below)

New Project Wizard – Choose a Sandcastle Presentation

The first useful change that you'll notice in the New Project Wizard is that the Choose a Sandcastle Presentation step provides readable names and a short description of the presentations that are available. (See Figure 1 below)

The data is located in DocProject's configuration file (commonly found at, C:\Program Files\Dave Sexton\DocProject\bin\DaveSexton.DocProject.dll.config) in the new presentation configuration section. This section describes the presentations that are installed with Sandcastle, indicating a human-readable name and description, and providing the locations to all of the tools and directories that DocProject expects. There is also a place to add custom Regular Expression and XPath expressions that will be used by DocProject to automatically transform the imported Build Assembler configuration file (sandcastle.config) when creating a new DocProject or DocSite. You can modify, remove or add new transformations to customize the default configuration files as they are imported into new projects.

Note: You can use the new presentation configuration section to add your own custom presentations too!

Figure 1: New Project Wizard - Choose a Sandcastle Presentation

As you can see from the screen-shot, the wizard looks a little bit different now. Other than the slight change to appearance and the new pre-load behavior, it functions identically to earlier versions of DocProject.

New Project Wizard – Create Shared Content

The next feature is the ability to create a header and footer using an Html Editor. The header and footer content are shared by all API topics and can be found in your project's Presentation\Content\shared_content.xml file, which you may edit manually as well. I'm also considering whether to add more sections that would correspond with other items in the shared_content.xml file, for a future version of DocProject.

Figure 2: New Project Wizard - Create Shared Content
Figure 2.1: Html Editor Context Menu

Notice in Figure 2 that the Header and Footer sections may be toggled between visible and hidden states by clicking their respective title bars. The box next to the title will contain a plus sign (+) when a section is collapsed and a minus sign (-) when a section is expanded.

The header and footer may be edited at a later time using the API Topic Designer, which is located at Tools > Options > DocProject > Active Projects > Content > API Topic Designer. Locate this setting for your project and click the ellipses button to open the editor, which is identical to the editor in the Create Shared Content wizard step, but without the wizard's chrome.

The Html Editor that is used here may be the beginning of a topic editor control as well; although I'd much rather use Visual Studio's HTML editor instead of reinventing the wheel, which would not be easy. One of the reasons that I wanted to use Sandcastle inside Visual Studio in the first place is so I could use its HTML and XML editors, but more on this discussion another time :)

API Topic Management Dialog

The API Topic Management dialog provides some new options for filtering API topics based on categories and regular expressions, such as the ability to find the next match only and to specify whether the dialog should automatically show matched topics in the TOC (i.e., if a matched node is hidden then its parent nodes will be expanded until it becomes visible, and then it will be scrolled into view).

Figure 3: API Topic Management Dialog – Topic Filters

The TOC is loaded on-demand now, by initially loading only the root topics and then loading the children of each individual topic as they are manually expanded for the first time. This saves some initialization time, but may increase the time that it takes for the filters to work. For this reason, the filters now execute asynchronously and the find next match only option is selected by default.

The API Topic Management dialog also provides support for editing project and namespace summaries in text mode (the contents of which must be valid xml just like with code comments) or using an Html Editor, as in Figure 4 below. You can freely switch back and forth between text mode and html mode and your changes will be persisted.

The Summary Text and Summary HTML tabs are only activated when the project node is selected, as in Figure 4, or when one of the namespace nodes is selected. For example, the Contoso namespace that appears in Figure 4 can be selected in order to edit its summary. If the AppConfigElement node is selected, however, the summary editor will be disabled.

Figure 4: API Topic Management Dialog – Summary HTML Editor

You can open this dialog by locating Tools > Options > DocProject > Active Projects > Configuration > API Topic Management and clicking the ellipses button. It's in the same place that it was in 1.5.0 :)

And More…

There are other features that I haven't discussed here, but I think the ones that I've mentioned are probably the most important.

I hope you're excited to use these new features, since I worked hard to create them. I'd love to know what you think about them and your ideas for other features, so please use the contact form here or on CodePlex, or leave a comment on this blog post and let me know what's on your mind.

Thanks!

Recently, a few people have mentioned that they'd like support in DocProject for generating custom topics that maintain the look and feel of the selected presentation. While designing DocProject I had thought about this and tried to provide an easy way for developers to create custom topics, which is why the partial build functionality was developed. But until now I haven't mentioned much about this feature or how it works.

I just added a tutorial to DocProject on CodePlex, which explains how to create custom topics using the partial build feature:

Tutorial: Creating Custom Topics
http://www.codeplex.com/DocProject/Wiki/View.aspx?title=Creating+Custom+Topics

The tutorial explains how to create an HTML topic template from one of the files that Sandcastle generates (namely, R_Project.htm). It also describes how you can use Visual Studio's HTML designer to generate your own topic files based on the template and how to include them in the Html Help Workshop table of contents (TOC). Code is also provided that extends the default build process component to automate some of the steps in the tutorial.

Enjoy, and let me know what you think :)

In this blog entry I'm going to show a preview of one of the new features that I've included in DocProject for the next release. I'll also write a little about my open-source development habits, how I test DocProject and my plans for DocProject in the future.

But before I begin I'd like to thank everyone that has downloaded DocProject and provided feedback. It's been helpful. Please keep adding work items and starting discussions so that I can continue to be informed of the community's needs.

Those of us that use Visual Studio 2005 do so for different purposes and in different manners, and there's no way that I could test DocProject in every possible situation on my own, so I'm relying on the community to let me know when I've missed a bug in testing and when there's room for improvement. And I'm not just looking for constructive criticism either, so if you have something nice to say about DocProject then please don't be shy :) It's also useful for me to know that people are actually using DocProject and that it's working to meet their needs.

My Multi-Project Open Source Development Life-cycle

I work weeknights and weekends on the planning and development of my open source projects and my proprietary software. I normally don't work from beginning to end on any particular release, but instead jump back and forth between projects. Usually, a context switch occurs when an idea pops into my head or if I get bored with what I'm doing. That's the beauty of working alone and on personal software. I'm currently working full-time on a desktop application for a client so I don't do much coding on personal projects during daylight hours; however, I do try to answer questions and provide help to the community in a timely manner and, from time-to-time, I'll do some wiki-work mid-day. Working from home has its perks ;)

Recently, I published AIP 1.0.0 RTW to CodePlex and now I'm switching gears to finish work on DocProject for the April 30^th deployment. I've fixed all of the bugs and added all of the features that have associated work items, including a few that don't, and now I'm on to stabilization.

Stabilizing DocProject

Stabilizing DocProject isn't so easy. Because it's an Add-In I have to make sure that every feature works in Visual Studio 2005, in many different contexts. Of course, I must also ensure that regression issues don't spring up.

I didn't write unit tests when DocProject was first conceived, but I'm actually glad now. Unit tests aren't going to work for a very complex Add-In that is hosted by Visual Studio 2005 because I won't be able to properly emulate VS in any mock object, IMO. And even if I could it wouldn't give me peace of mind. Having a script to automate Visual Studio might be useful, but I don't think it's worth the effort to create one.

I test DocProject by installing it on my Vista box, which is also my development box, and then I create new DocProjects and DocSites from scratch, testing each of the features as I go. I also test DocProject on a Windows Server 2003 VM. To be perfectly honest though, I don't test all of the features each time. There are just too many features to test; although, I usually don't make code changes that will affect every bit. I guess that's another benefit of working alone: I know what I did and usually have a good idea of what components it may have affected. The components that aren't touched at all, for which I'm sure, I'll skip testing.

Thankfully, I haven't run into many regression issues. I think that's due, in part, to the amount of refactoring that I did in previous versions before I added several of the major features that are in DocProject now. There's still certainly a lot of room for refactoring and once I publish the first RTW I think I may revisit those classes for which I made a mental note. Actually, I'll probably review the entire code-base from scratch and see if I can re-architect DocProject for some of my future plans (more on them below).

But for now, I'm going to continue stabilizing DocProject until I'm ready to release 1.5.0, by April 30^th. (BTW, I try to schedule deployments on a Monday so that I have an entire weekend prior to really get down and dirty with testing, last-minute tweaks and writing wikis.)

API Topic Management Dialog - Preview

Here's a preview of a new feature that I've added to DocProject. It's a dialog that allows a user to configure the apiFilter configuration element in Sandcastle's MRefBuilder configuration file, which is now included as a project item in the Presentation\Configuration folder. You can also edit the configuration file manually without tripping up the dialog, as long as your changes conform to the xml schema that Sandcastle requires, of course.

An API topic is included by checking its check box node in the tree view control and excluded by unchecking it. All topics are included (checked) by default if the apiFilter configuration section hasn't been used.

The dialog is accessed in the Tools Options Page for Active Projects by clicking the ellipses button of the Configuration > API Topic Management property. Notice in the image below that another new property has been added as well, MRefBuilder Configuration File Name, which specifies the name of the config file in the Presentation\Configuration directory to be used with the Sandcastle MRefBuilder program.

Figure 1: Finding the API Topic Management Dialog

Figure 2: The API Topic Management Dialog

Regular Expression Filter

Use the regular expression filter to include, exclude or locate topics by name using a managed-style regular expression. A link to an MSDN help reference on regular expressions is provided next to a link that pops-up a regular expression quick-help guide, which includes a few examples. You can also choose the regular expression's options.

The Find button will locate all matching topics by highlighting them.

The Apply button will locate all matching topics, highlight each of them, and ensure that the state of each check box matches the state of the Include matching topics check box.

An icon will appear next to the Find button after either button is clicked. The icon's tooltip displays the number of matching topics. The tooltip is automatically displayed for one and a half seconds after the search has completed. Clicking the icon repeatedly will cycle through all of the matching topics, selecting the current node and expanding nodes when necessary. The highlighted color and the icon's appearance differ depending upon whether you have clicked Find or Apply. Find was clicked in Figure 2 and the blue highlights in the tree view are the matching topics.

Clicking the icon to cycle through the matching topics can be useful. Use the spacebar to toggle (check or uncheck) a matching topic and click the icon to cycle to the next match. Using the icon and the Find button instead of the Apply button will allow you to manually choose from all of the matched topics the ones that should be toggled.

Category Filter

Use the category filter to include or exclude topics by API category. For instance, to include all interfaces you would first check the Interfaces checkbox and ensure that no other category is checked. Then make sure that the Include matching topics checkbox is checked to include matching topics (unchecking it will exclude matching topics). Press the Apply button to apply the changes.

Like the regular expression filter, an icon will appear next to the Apply button after it's clicked. The icon's tooltip displays the number of matching topics. The tooltip is automatically displayed for one and a half seconds after the search has completed. Clicking the icon repeatedly will cycle through all of the matching topics, selecting the current node and expanding nodes when necessary.

How It All Works

When you open the dialog it takes some time to fill the tree view, but the dialog remains responsive to user interaction and displays a marquee progress bar in the mean time. In the background, DocProject is actually using the Sandcastle build engine to run a build of the TOC only (the engine was refactored a bit to provide this new functionality).

Basically, all of the steps up to the Build Assembler are executed in the background.  The toc.xml and reflection.xml files that are created by the the Sandcastle build steps are then parsed into the tree view nodes.  The toc.xml file is used for the layout of the tree view and the reflection.xml file provides most of the information used to construct each topic node.  And for each topic node a look-up into the existing apiFilter configuration section of the Presentation\Configuration\MRefBuilder.config file is performed to discover whether the topic should be included or excluded (checked or unchecked, respectively), by default.

A temporary copy of each node is made at the time they are created.  When you modify the tree view, the copy of each node is preserved for the OK and Cancel functionality of the dialog.

Clicking OK on the dialog doesn't apply the changes, however. You must click OK on Visual Studio's Options dialog to commit the changes. This allows you to edit the API Topic Management property multiple times before you finally save your changes. If you click OK on the property's dialog and then reopen it, modify something, and click Cancel, then only the changes that you made since the last time the dialog was opened will be canceled. The original changes for which you clicked OK last will be retained.

When you finally save the changes by clicking OK on the Options dialog, they are written directly to the MRefBuilder.config file, overwriting any existing configuration of the apiFilter element. Other existing elements and content in the configuration file is left unmodified.

One stipulation to use this feature is that you must build all of the source projects once before the dialog will display their topics. If you only build some of the source projects then the dialog will only display those API topics. If you clean the solution or otherwise delete any of the output assemblies from the source projects then the corresponding API topics will not be displayed. In other words, the source assemblies have to be present in order for the dialog to display their topics. Showing the dialog does not automatically build all of the source projects so you must build the projects before using the dialog. If you close Visual Studio after building a source project and then reopen it, the assembly will still be there so you won't need to build the project again to use the API Topic Management dialog for a referencing DocProject or DocSite.

Performance

The MRefBuilder utility is very quick, even for the 13-assembly library that I use to test DocProject's performance. But the dialog, for the same library, does take a considerable amount of time to fill the tree view, and that's what the dialog is doing for the majority of the time that you wait for it to be ready. In testing it took about 5 minutes for the dialog to be ready for 2715 API topics. However, that number only represents API  topics, not other topics such as All Members, so the number presented as the API Topic Management property value is somewhat lower than the total number of topics that end up in the compiled help.

I already see room for improvement in the code base to increase the performance of the dialog, and I was even thinking about possibly adding nodes-on-demand functionality, but I'm going to save those changes for a subsequent release. The idea behind the on-demand behavior, BTW, is that expanding a node for the first time will cause its children to be loaded at that time instead of all nodes being loaded when the dialog opens. However, I have some doubts if that would be useful since the filters will require all nodes to be loaded before they will function properly. Obviously, I still have some things to work out for that feature (which is why I'm not adding it now).

To improve the performance I plan on using an XmlReader to parse the topics instead of an XmlDocument and I plan on using a lot more custom iterators in C# too (i.e., the yield statement).

The performance of the tree view and the filters after all of the topics have been loaded is actually quite good; as is the process that writes the changes to disc when you click OK on the Options dialog.

DocProject's Past, Present and Future

DocProject hasn't been around that long but it has accumulated over 800 downloads between all of its versions, up to the 1.4.0 RC. Currently, DocProject is downloaded about 10 times per day, on average.

I have some plans for the future of DocProject, feature-wise, but nothing is set in stone. Feedback is a must for me. Please let me know what features you want to see in DocProject and I will consider them.

Why doesn't DocProject provide any content-based features?

Currently, DocProject is built for automating Sandcastle from within Visual Studio 2005, with extensibility built-in to provide an open-ended environment for developers to control the help-build process. All of the Sandcastle presentation files (xml transformations and configuration files) are included in DocProjects and DocSites so that users can configure Sandcastle the way they want without being restricted to GUIs. In other words, DocProject is not focused on content-based features. DocProject relies on Sandcastle and you, the user, to develop and configure your own documentation. This has always been the most important guideline that I followed since I started developing the first beta.

Recently I received a request from a user to include some GUI support for things such as adding namespace summaries and customizing the default copyright, header and footer of the HTML topic files generated by Sandcastle (see work item #9717 and vote if you're interested). Although I have considered adding these features in past releases, I always ended up siding with the idea that DocProject is better off without content-related features, for now. What I mean is that DocProject has always been geared toward making it easier to build documentation using Sandcastle and for users to be able to develop their documentation using Sandcastle, directly. I've purposefully avoided creating interfaces for users to manage the HTML topic files that Sandcastle generates in favor of simply including the xml transformations and configuration files as project items so that users could configure Sandcastle themselves. The reasons are simple:

• Sandcastle is not RTW yet and there are changes between CTP releases, naturally.
• I consider adding content-based features a major version change from not having any content-based features and I'd rather wait for the first DocProject RTW to correspond with the first Sandcastle RTW before any major changes are implemented in DocProject.
• I didn't want to be bogged down with presentation-related issues while I was developing and stabilizing DocProject's feature-base for building documentation, which I consider to be more important now since all Sandcastle files are visible to users and are not modified by DocProject in any way. So users are completely free to configure Sandcastle however they want and I don't have to worry about supporting end-user/DocProject configuration mistakes or issues with the appearance of HTML files that Sandcastle generates.
• It seems a bit ridiculous to me for DocProject to offer a textbox so users could edit an HTML header, for example, inside of the Visual Studio 2005 IDE when they can use the XML editor to edit the entire shared content XML file.
• I can't help but think that it should be all or nothing. A topic designer must be included or else DocProject should simply rely on Sandcastle and DocProject users to produce the documentation, as it currently does. Without a topic designer, any UI support for presentation would just be a hack, IMO.

Planned Features

I plan to research some content-based features, including the ability to add namespace summaries and maybe even a basic topic designer for the next major release of DocProject.

In preparation for the next release of Sandcastle, which I read will include a new presentation design (A. Raman, MSDN Forums, Apr 20, 2007; Last Post), I've already begun to find places in the code base that I want to refactor. I'm also going to start testing DocProject in the Orcas CTPs so that it will be ready for Orcas documentation, as long as Sandcastle supports it.

I'd also like to add a command-line build feature (work item #9788) and an MSBuild task (work item #9787). I've already thought about how that will work and started refactoring the code base in 1.5.0 in preparation for the changes required to support these features. Some of the changes were included in 1.5.0 simply because they were easy enough to do.

I've begun to create a command-line executable named, docop (DocProject Operator) that will eventually automate the build process of a DocProject or DocSite outside of the Visual Studio 2005 IDE. The plan is to run the MSBuild task using the MSBuild API, which I started playing around with last weekend.

If you are interested in any of the features that I mentioned then please follow the corresponding link and vote on the work item in CodePlex. Thanks!

I've made some more modifications to the DocProject software. Several classes have been refactored again and some new classes have been added. A few work items have been addressed and now there is support for Visual Basic.NET and ASP.NET Web Application projects (DocSites). You can also cancel the build in the middle of a step.

Download the latest release, version 1.2.0, at DocProject on CodePlex.

Collaboration

Steve Bokser, a programmer and a friend of mine, helped me to test the new release by installing it on his machine before I made it available to the public. We identified a few issues that didn't surface during my own testing since we have different versions of Visual Studio installed. I took care of the issues that I could fix before the CodePlex deployment and I adjusted the release notes and wikis to include information regarding the setup and configuration issues. Thanks Steve!

Custom Templates

An organization may want to create a custom template that their developers can use to create DocProjects or DocSites for their Visual Studio projects. A custom DocProject or DocSite template can provide a standardized means of building compiled help documentation for projects across an entire organization.

A new tutorial has been added to provide guidance for creating new DocProject and DocSite templates:

Creating New Templates
http://www.codeplex.com/DocProject/Wiki/View.aspx?title=Creating%20New%20Templates

Wiki Updates

Many of the How To... wikis have been modified to reflect changes in the new release.

A new diagram that illustrates the different components of DocProject and their relationships appears on the home page wiki now:

Figure 1: DocProject Components

As always, I'd love to hear from anyone that has something to say about this post.  Drop me a comment.