In this blog entry I'm going to show a preview of one of the new features that I've included in DocProject for the next release. I'll also write a little about my open-source development habits, how I test DocProject and my plans for DocProject in the future.
But before I begin I'd like to thank everyone that has downloaded DocProject and provided feedback. It's been helpful. Please keep adding work items and starting discussions so that I can continue to be informed of the community's needs.
Those of us that use Visual Studio 2005 do so for different purposes and in different manners, and there's no way that I could test DocProject in every possible situation on my own, so I'm relying on the community to let me know when I've missed a bug in testing and when there's room for improvement. And I'm not just looking for constructive criticism either, so if you have something nice to say about DocProject then please don't be shy :) It's also useful for me to know that people are actually using DocProject and that it's working to meet their needs.
My Multi-Project Open Source Development Life-cycle
I work weeknights and weekends on the planning and development of my open source projects and my proprietary software. I normally don't work from beginning to end on any particular release, but instead jump back and forth between projects. Usually, a context switch occurs when an idea pops into my head or if I get bored with what I'm doing. That's the beauty of working alone and on personal software. I'm currently working full-time on a desktop application for a client so I don't do much coding on personal projects during daylight hours; however, I do try to answer questions and provide help to the community in a timely manner and, from time-to-time, I'll do some wiki-work mid-day. Working from home has its perks ;)
Recently, I published AIP 1.0.0 RTW to CodePlex and now I'm switching gears to finish work on DocProject for the April 30th deployment. I've fixed all of the bugs and added all of the features that have associated work items, including a few that don't, and now I'm on to stabilization.
Stabilizing DocProject
Stabilizing DocProject isn't so easy. Because it's an Add-In I have to make sure that every feature works in Visual Studio 2005, in many different contexts. Of course, I must also ensure that regression issues don't spring up.
I didn't write unit tests when DocProject was first conceived, but I'm actually glad now. Unit tests aren't going to work for a very complex Add-In that is hosted by Visual Studio 2005 because I won't be able to properly emulate VS in any mock object, IMO. And even if I could it wouldn't give me peace of mind. Having a script to automate Visual Studio might be useful, but I don't think it's worth the effort to create one.
I test DocProject by installing it on my Vista box, which is also my development box, and then I create new DocProjects and DocSites from scratch, testing each of the features as I go. I also test DocProject on a Windows Server 2003 VM. To be perfectly honest though, I don't test all of the features each time. There are just too many features to test; although, I usually don't make code changes that will affect every bit. I guess that's another benefit of working alone: I know what I did and usually have a good idea of what components it may have affected. The components that aren't touched at all, for which I'm sure, I'll skip testing.
Thankfully, I haven't run into many regression issues. I think that's due, in part, to the amount of refactoring that I did in previous versions before I added several of the major features that are in DocProject now. There's still certainly a lot of room for refactoring and once I publish the first RTW I think I may revisit those classes for which I made a mental note. Actually, I'll probably review the entire code-base from scratch and see if I can re-architect DocProject for some of my future plans (more on them below).
But for now, I'm going to continue stabilizing DocProject until I'm ready to release 1.5.0, by April 30th. (BTW, I try to schedule deployments on a Monday so that I have an entire weekend prior to really get down and dirty with testing, last-minute tweaks and writing wikis.)
API Topic Management Dialog - Preview
Here's a preview of a new feature that I've added to DocProject. It's a dialog that allows a user to configure the apiFilter configuration element in Sandcastle's MRefBuilder configuration file, which is now included as a project item in the Presentation\Configuration folder. You can also edit the configuration file manually without tripping up the dialog, as long as your changes conform to the xml schema that Sandcastle requires, of course.
An API topic is included by checking its check box node in the tree view control and excluded by unchecking it. All topics are included (checked) by default if the apiFilter configuration section hasn't been used.
The dialog is accessed in the Tools Options Page for Active Projects by clicking the ellipses button of the Configuration > API Topic Management property. Notice in the image below that another new property has been added as well, MRefBuilder Configuration File Name, which specifies the name of the config file in the Presentation\Configuration directory to be used with the Sandcastle MRefBuilder program.
Figure 1: Finding the API Topic Management Dialog
Figure 2: The API Topic Management Dialog
Regular Expression Filter
Use the regular expression filter to include, exclude or locate topics by name using a managed-style regular expression. A link to an MSDN help reference on regular expressions is provided next to a link that pops-up a regular expression quick-help guide, which includes a few examples. You can also choose the regular expression's options.
The Find button will locate all matching topics by highlighting them.
The Apply button will locate all matching topics, highlight each of them, and ensure that the state of each check box matches the state of the Include matching topics check box.
An icon will appear next to the Find button after either button is clicked. The icon's tooltip displays the number of matching topics. The tooltip is automatically displayed for one and a half seconds after the search has completed. Clicking the icon repeatedly will cycle through all of the matching topics, selecting the current node and expanding nodes when necessary. The highlighted color and the icon's appearance differ depending upon whether you have clicked Find or Apply. Find was clicked in Figure 2 and the blue highlights in the tree view are the matching topics.
Clicking the icon to cycle through the matching topics can be useful. Use the spacebar to toggle (check or uncheck) a matching topic and click the icon to cycle to the next match. Using the icon and the Find button instead of the Apply button will allow you to manually choose from all of the matched topics the ones that should be toggled.
Category Filter
Use the category filter to include or exclude topics by API category. For instance, to include all interfaces you would first check the Interfaces checkbox and ensure that no other category is checked. Then make sure that the Include matching topics checkbox is checked to include matching topics (unchecking it will exclude matching topics). Press the Apply button to apply the changes.
Like the regular expression filter, an icon will appear next to the Apply button after it's clicked. The icon's tooltip displays the number of matching topics. The tooltip is automatically displayed for one and a half seconds after the search has completed. Clicking the icon repeatedly will cycle through all of the matching topics, selecting the current node and expanding nodes when necessary.
How It All Works
When you open the dialog it takes some time to fill the tree view, but the dialog remains responsive to user interaction and displays a marquee progress bar in the mean time. In the background, DocProject is actually using the Sandcastle build engine to run a build of the TOC only (the engine was refactored a bit to provide this new functionality).
Basically, all of the steps up to the Build Assembler are executed in the background. The toc.xml and reflection.xml files that are created by the the Sandcastle build steps are then parsed into the tree view nodes. The toc.xml file is used for the layout of the tree view and the reflection.xml file provides most of the information used to construct each topic node. And for each topic node a look-up into the existing apiFilter configuration section of the Presentation\Configuration\MRefBuilder.config file is performed to discover whether the topic should be included or excluded (checked or unchecked, respectively), by default.
A temporary copy of each node is made at the time they are created. When you modify the tree view, the copy of each node is preserved for the OK and Cancel functionality of the dialog.
Clicking OK on the dialog doesn't apply the changes, however. You must click OK on Visual Studio's Options dialog to commit the changes. This allows you to edit the API Topic Management property multiple times before you finally save your changes. If you click OK on the property's dialog and then reopen it, modify something, and click Cancel, then only the changes that you made since the last time the dialog was opened will be canceled. The original changes for which you clicked OK last will be retained.
When you finally save the changes by clicking OK on the Options dialog, they are written directly to the MRefBuilder.config file, overwriting any existing configuration of the apiFilter element. Other existing elements and content in the configuration file is left unmodified.
One stipulation to use this feature is that you must build all of the source projects once before the dialog will display their topics. If you only build some of the source projects then the dialog will only display those API topics. If you clean the solution or otherwise delete any of the output assemblies from the source projects then the corresponding API topics will not be displayed. In other words, the source assemblies have to be present in order for the dialog to display their topics. Showing the dialog does not automatically build all of the source projects so you must build the projects before using the dialog. If you close Visual Studio after building a source project and then reopen it, the assembly will still be there so you won't need to build the project again to use the API Topic Management dialog for a referencing DocProject or DocSite.
Performance
The MRefBuilder utility is very quick, even for the 13-assembly library that I use to test DocProject's performance. But the dialog, for the same library, does take a considerable amount of time to fill the tree view, and that's what the dialog is doing for the majority of the time that you wait for it to be ready. In testing it took about 5 minutes for the dialog to be ready for 2715 API topics. However, that number only represents API topics, not other topics such as All Members, so the number presented as the API Topic Management property value is somewhat lower than the total number of topics that end up in the compiled help.
I already see room for improvement in the code base to increase the performance of the dialog, and I was even thinking about possibly adding nodes-on-demand functionality, but I'm going to save those changes for a subsequent release. The idea behind the on-demand behavior, BTW, is that expanding a node for the first time will cause its children to be loaded at that time instead of all nodes being loaded when the dialog opens. However, I have some doubts if that would be useful since the filters will require all nodes to be loaded before they will function properly. Obviously, I still have some things to work out for that feature (which is why I'm not adding it now).
To improve the performance I plan on using an XmlReader to parse the topics instead of an XmlDocument and I plan on using a lot more custom iterators in C# too (i.e., the yield statement).
The performance of the tree view and the filters after all of the topics have been loaded is actually quite good; as is the process that writes the changes to disc when you click OK on the Options dialog.
DocProject's Past, Present and Future
DocProject hasn't been around that long but it has accumulated over 800 downloads between all of its versions, up to the 1.4.0 RC. Currently, DocProject is downloaded about 10 times per day, on average.
I have some plans for the future of DocProject, feature-wise, but nothing is set in stone. Feedback is a must for me. Please let me know what features you want to see in DocProject and I will consider them.
Why doesn't DocProject provide any content-based features?
Currently, DocProject is built for automating Sandcastle from within Visual Studio 2005, with extensibility built-in to provide an open-ended environment for developers to control the help-build process. All of the Sandcastle presentation files (xml transformations and configuration files) are included in DocProjects and DocSites so that users can configure Sandcastle the way they want without being restricted to GUIs. In other words, DocProject is not focused on content-based features. DocProject relies on Sandcastle and you, the user, to develop and configure your own documentation. This has always been the most important guideline that I followed since I started developing the first beta.
Recently I received a request from a user to include some GUI support for things such as adding namespace summaries and customizing the default copyright, header and footer of the HTML topic files generated by Sandcastle (see work item #9717 and vote if you're interested). Although I have considered adding these features in past releases, I always ended up siding with the idea that DocProject is better off without content-related features, for now. What I mean is that DocProject has always been geared toward making it easier to build documentation using Sandcastle and for users to be able to develop their documentation using Sandcastle, directly. I've purposefully avoided creating interfaces for users to manage the HTML topic files that Sandcastle generates in favor of simply including the xml transformations and configuration files as project items so that users could configure Sandcastle themselves. The reasons are simple:
- Sandcastle is not RTW yet and there are changes between CTP releases, naturally.
- I consider adding content-based features a major version change from not having any content-based features and I'd rather wait for the first DocProject RTW to correspond with the first Sandcastle RTW before any major changes are implemented in DocProject.
- I didn't want to be bogged down with presentation-related issues while I was developing and stabilizing DocProject's feature-base for building documentation, which I consider to be more important now since all Sandcastle files are visible to users and are not modified by DocProject in any way. So users are completely free to configure Sandcastle however they want and I don't have to worry about supporting end-user/DocProject configuration mistakes or issues with the appearance of HTML files that Sandcastle generates.
- It seems a bit ridiculous to me for DocProject to offer a textbox so users could edit an HTML header, for example, inside of the Visual Studio 2005 IDE when they can use the XML editor to edit the entire shared content XML file.
- I can't help but think that it should be all or nothing. A topic designer must be included or else DocProject should simply rely on Sandcastle and DocProject users to produce the documentation, as it currently does. Without a topic designer, any UI support for presentation would just be a hack, IMO.
Planned Features
I plan to research some content-based features, including the ability to add namespace summaries and maybe even a basic topic designer for the next major release of DocProject.
In preparation for the next release of Sandcastle, which I read will include a new presentation design (A. Raman, MSDN Forums, Apr 20, 2007; Last Post), I've already begun to find places in the code base that I want to refactor. I'm also going to start testing DocProject in the Orcas CTPs so that it will be ready for Orcas documentation, as long as Sandcastle supports it.
I'd also like to add a command-line build feature (work item #9788) and an MSBuild task (work item #9787). I've already thought about how that will work and started refactoring the code base in 1.5.0 in preparation for the changes required to support these features. Some of the changes were included in 1.5.0 simply because they were easy enough to do.
I've begun to create a command-line executable named, docop (DocProject Operator) that will eventually automate the build process of a DocProject or DocSite outside of the Visual Studio 2005 IDE. The plan is to run the MSBuild task using the MSBuild API, which I started playing around with last weekend.
If you are interested in any of the features that I mentioned then please follow the corresponding link and vote on the work item in CodePlex. Thanks!