In this post, I'll introduce you to my new OSS project called White Tie and I'll describe how it relates to Rxx a bit.

I'll also introduce you to a new feature of another OSS project of mine, Visual Studio Settings Switcher.  This new feature allows you to easily share VS settings files with your solution, and OSS contributors to easily take advantage of it.

White Tie and VS Settings Switcher provide a set of features that I've specifically intended to be used by OSS developers, though anyone can benefit from them really.

I know that some of the ideas in these projects are probably not new, but perhaps the way in which I've implemented those ideas as features is new enough, and simple enough, to make them better than competing projects.

Your feedback is welcome.

White Tie

I've recently created the White Tie project on CodePlex.  I like to think of it as "formal buildware for use after 6PM".

White Tie is basically just a couple of unintrusive MSBuild files that are imported by your VS projects.  It's primarily intended for OSS developers who use Visual Studio (or MSBuild) to build their projects into NuGet packages, and optionally use Code Contracts, run static analysis and like to copy all build output into a local aggregate directory.  There are also work items currently under consideration to build reference documentation via Sandcastle and to enable remote deployment scenarios, for example.

White Tie makes building, packaging and deployment easier by offering sensible, out-of-the-box configuration, though it's also highly customizable by adding particular MSBuild properties and items to your project file.  (Read the Configuration documentation for details.)

White Tie is delivered as a NuGet package, so it's really easy to integrate into your projects.  Simply build your project in Release mode and a NuGet package will be generated automatically.  No configuration is necessary for existing projects, in general, though read the note at the bottom here for details about build errors caused by missing assembly-level attributes in new projects.

One of White Tie's key features is the NuGetFlavor item group.  By adding project references as NuGetFlavor items, rather than as ProjectReference items, White Tie will automatically include the referenced projects' output into the primary project's NuGet package.

For example, let's say that you have a primary class library project called Project A, which targets the full .NET Framework 4.5.1.  If you add a NuGet reference to White Tie, then whenever Project A is built in Release mode (by default), White Tie will generate a NuGet package for Project A.  Now let's say that you also have a portable class library project called Project B, which targets a portable profile that includes Windows 8 and Windows Phone 8.  Project B is intended to be the same as Project A, only targeting different platforms.  You might refer to Project B as a "flavor" of Project A.  Now you can simply edit the project file of Project A to add a NuGetFlavor item that points to the project file of Project B.  Whenever you build Project A in Release mode, White Tie ensures that Project B is built as well, and it also includes the portable library built by Project B into the NuGet package that it builds for Project A, and in the correct target framework subfolder.

White Tie offers several more features and configuration options.  Take a look at the documentation to get started, and let me know what you think.

White Tie's Origins and Future

White Tie is something that I've been working on for a few years, in one form or another.  Much of the work began as part of the Rxx project, such as its support for NuGet, though some features even predate Rxx.

Since Rxx it has evolved into something a bit more useful, though I've only been using it internally for my own OSS projects and business-related products (under the name DaveSexton.Build).  I finally decided recently that if I was ever going to "officially" release Rxx again, I'd have to bring its build system and project configurations up-to-date first.  White Tie is basically the best of DaveSexton.Build, formalized and revised (or "reimagined", if you prefer advertising jargon) to take advantage of NuGet 2.8 features, including complete support for portable libraries.  My transition to White Tie is not yet complete, and there are currently some road blocks, but I'm committed to getting it working fully.

Note that this isn't a guarantee that I'm going to build another NuGet package for Rxx, although it's certainly a large step in the right direction.  Not only have I checked in updates to Rxx's source code to include references to White Tie already, but I've also begun to think about and make changes to the portable library configurations that Rxx will target in the future.  But even after that, I'm still going to have to comb over all of its APIs before I'll release an "official" package again.  I have OCD and I haven't reviewed its source code or work items thoroughly since Rx 2.0 was released, so my confidence level in Rxx is (unreasonably, perhaps) lower than it should be.  :)

Visual Studio Settings Switcher

I've recently released version 1.2 of the Visual Studio Settings Switcher project on CodePlex, also available in the Visual Studio Gallery.

The latest release adds a new feature that I think OSS developers really want: Solution Settings Files

If you have an OSS project, VS Settings Switcher makes it really easy to generate a Visual Studio settings file that you can check into source control along with your solution, for your OSS contributors to use.  This settings file is in the typical VS settings file format.  It's also XML, which makes it easy to edit in Visual Studio, which you should definitely do before checking it in.  I highly recommend removing all of the extraneous settings that nobody wants to be shared in OSS projects, like the positions of your VS tool windows and other environment settings.  Typically, you'll want to reduce this file to only a small subset, such as your code formatting settings for your project's programming language.

This feature is also great for those of us that contribute to OSS projects.  It makes applying an OSS project's Visual Studio settings a breeze, without any manual intervention required.  If a solution has a .vssettings file with the same name and in the same directory, then VS Settings Switcher will automatically apply its settings whenever your open that solution.  And when you close that solution or open a different solution later, your original settings are automatically restored.

Spread the word: It's a good idea now to check in a partial .vssettings file with your solution!

I just published the first release of my Settings Switcher extension for Visual Studio 2012 on CodePlex.

Yesterday, I was looking for a quick solution to switch between Visual Studio settings files (.vssettings) without using the Tools > Import and Export Settings… dialog.  The dialog’s great, but it’s a bit slow to use a wizard every time I want to switch settings.  Sometimes I like to jump back and forth between different solutions quickly during the day, like when I’m working on various open source projects with different code formatting requirements.  I wanted a quicker way to load settings after opening a solution.

I found this blog post, which had a link to a tip on Sara Ford’s blog showing how easy it is to create a command button in the IDE to switch between settings by writing a simple macro.

Unfortunately, Visual Studio 2012 doesn’t support macros.

So I decided to create a Visual Studio extension.  The original premise was simply to show a drop-down list containing all of the .vssettings files and when one is selected those settings are applied.  That turned out to be a useful feature of the extension; however, while I was developing it I realized that I could add a few more useful features.  For example, I added a button to format every code file in the solution according to the currently selected settings.

But the primary feature, IMO, is that when you close a solution or exit Visual Studio, the extension saves a reference to the current settings file in the solution’s user options file (.suo).  The next time that you open that solution, the associated settings are automatically applied by the extension.  “Automate everything” is my motto :)

This extension should be useful to open source developers.  Typically we work on projects across multiple teams, with different code formatting requirements.  The next time you fork a project you can simply open the solution, edit your settings in the normal Tools > Options… dialog to meet the requirements of the team, then export the settings to a new file using the Export Current Settings button on the Settings Switcher toolbar provided by the extension.  And that’s it!  Every time you open that particular solution, its associated settings are applied automatically.

Project coordinators should also consider checking in a .vssettings file to their repositories.  This will make it easier for interested developers to set up their environment for working on your project.  A developer would simply need to copy the .vssettings file that you provided to the directory that Visual Studio uses to export settings files on their system (Tools > Options > Environment > Import and Export Settings).  Then follow the instructions above to associate the settings with the solution.

I’ve got some additional features that I plan on adding in the future, time permitting.  Some of them are very community-oriented.  We’ll see how that goes.

Feedback is appreciated.  Feel free to request a feature, report a bug or start a discussion.

 

It’s time to start thinking reactively!  What have you been waiting for?

Maybe you were waiting for this:
Yesterday, Microsoft announced that Reactive Extensions (Rx) is now an open source project.  The complete source code for Rx 2.0 is being hosted at http://rx.codeplex.com under the Apache License 2.0.  After an extensive prerelease period and two “officially” released versions, Rx is finally being opened up to the community for a look inside the monadic machinery that helps developers to compose asynchronous queries with ease.  Not only that, but now you can submit bug fixes and implement features yourself.*

Maybe you were waiting for this:
Interactive Extensions (Ix) for .NET and JavaScript are open as well.  They are included under the same source control root as Rx.  I’ve decided to form a habit:  Whenever I create a new project, before I even write a single line of code, I’ll add references to the Rx Package and Ix Package from NuGet.  (Note:  I just started a discussion about the future of distribution, so those links may change.)

Maybe you were waiting for this:
The Rx Team has included brand new libraries to support C/C++.  They are aptly named Rx++ and Ix++.

Or maybe this:
The Rx Team has included new “Rx binding” examples, which are basically just specialized examples illustrating how to apply Rx and LINQ to specific domains.  They’ve included Tx, which is a set of code samples showing how to use LINQ to Events; e.g., real-time queries from trace logs for ETW, Windows Events and SQL Server Extended Events.  Another Rx binding example they’ve included is LINQ2Charts, which “allows developers to use LINQ to create/change/update charts in an easy way and avoid having to deal with XML or other underneath data structures”.

They’re also looking for new examples of Rx bindings, so let them know if you’ve got any.

Or this?
Rx already has an open source community building libraries and tools around it.  And now that Rx itself is open source, that should help to strengthen the existing open source Rx community by exposing it to more developers.  So even if you can’t/won’t contribute code directly to Rx, either due to complexity or legal constraints or whatever, then at least consider contributing code and/or feedback to one of the existing open source projects that enhance Rx.

Seriously, are you still considering how to develop asynchronous applications without Rx?  Are you also considering how to simplify data extraction from a SQL Server table without using SQL?  Or concurrently poke your eyes out with the pointy ends of a Metro-style button?

My (Additional) Impressions

A primary key to success for any open source library is to be extremely useful.  Another key to success is to be unique.  Rx is both, so I have no doubt that it will flourish in the open source world.

Rx goes a long way to make asynchronous programming easy.  It’s a solid library built around core principles that hides much of the complexity of controlling and coordinating asynchrony within any kind of application.  Opening it will help to lower the learning curve and increase the adoption rate of this amazing library, enabling developers to create complex asynchronous queries with relative ease and without any spaghetti code left over.  If asynchronous spaghetti code were a disease, Rx is the cure.

The more functionality a library offers, the more it becomes imperative to be able to peruse the source code.  Understanding user error and library bugs can be very difficult without source code, especially given the inherent complexity of asynchronous programming and the many factors of Rx’s public contracts, which must be considered while developing and debugging.  Capturing every last detail about Rx operators in documentation may be overly cumbersome or even impossible.  Opening Rx can help to make building and debugging complex asynchronous queries much easier.

Opening Rx allows the community to be less entitled and more empowered.  Developers using Rx already identify bugs and request features by posting them in the Rx forum, so it makes perfect sense to allow them to fix bugs and implement features themselves in a timely manner and share their work directly with the community.

Opening Rx to the community will help to tighten their relationship.  It will help to focus the flow of feedback to Microsoft, enabling more informed decisions about the shape of features and the future of Rx.

I’ve noticed the trend at Microsoft to move very useful products into open source projects on CodePlex.  I’ve been wondering whether Rx would take this route as well.  There are many great uses for Rx and its reach will certainly grow even more as an open source project.  The communities around Microsoft’s many open source projects are continuously growing and Rx certainly deserves to benefit from community involvement as much as any other product.

*After jumping through a legal hoop or two.  Read the Rx project home page for details.

If you’re not familiar with the Rxx project yet, it's something I’ve been working on for several months now along with James Miles.

Rxx is a library of unofficial reactive LINQ extensions supplementary to Microsoft's Reactive Extensions for .NET (Rx).  Rxx is developed entirely in C# and targets the .NET Framework 4.0, Silverlight 4.0 and Windows Phone 7 (WP7).

Why should you use Rx and Rxx?

Well, if you’re doing any kind of programming that involves asynchrony or concurrency, such as what is commonly found in UI layers, business layers, data access layers, middle tier services – basically everywhere, then you’ll definitely find Rx to be very useful.  Rxx adds a whole bunch of useful features on top of Rx and the .NET Framework Class Library (FCL), making it quite easy to introduce common patterns of asynchrony into any .NET program through the use of IObservable<T> and LINQ.

Rxx provides the following features.  See the Documentation for details.

1. Many IObservable<T> extension methods and IEnumerable<T> extension methods.
2. Many useful types such as CommandSubject, ListSubject, DictionarySubject, ViewModel, ObservableDynamicObject, Either<TLeft, TRight>, Maybe<T> and others.
3. Various interactive labs that illustrate the runtime behavior of the extensions in Rxx.  Individual labs' source code included.

The latest release of Rxx is now available:

• CodePlex
  http://rxx.codeplex.com/releases/view/69545
• NuGet
  http://nuget.org/List/Packages/Rxx
  http://nuget.org/List/Packages/Rxx_Experimental

New features for Rxx 1.2 include:

1. Compatible with Microsoft's Ix Experimental library.
2. UI extensions for WPF and Silverlight, including AnonymousCommand, CommandSubject, a Subscription XAML markup extension for binding UI elements to observables, EventSubscription trigger and a reactive view model infrastructure.  (Download the labs application for examples.)
3. N-ary Zip and CombineLatest combinators.
4. Several parser updates, including new operators, non-greedy (lazy) quantifiers and major performance and memory improvements, such as avoiding stack overflows due to recursion in quantifiers.
5. Cursor types and extensions (Rx and Ix).
6. ListSubject and DictionarySubject.
7. Consume extensions that generalize the producer/consumer pattern over observables.
8. ApplicationSettingsBase extensions.
9. ObservableSyndication for RSS 2.0 and Atom 1.0.
10. ObservableFile and additional ObservableDirectory extensions.
11. Stream, FileStream and TextReader extensions.

More details can be found in the latest release notes: http://rxx.codeplex.com/wikipage?title=Release%20Notes

We’d really appreciate your feedback.  Please let us know about your experiences with Rxx by starting a new discussion or submitting an issue.  Thanks!

In my first DocProject Roadmap post I described my preliminary plans for DocProject 1.x and later versions.  In this post I'm going to discuss the project's current status and some updates to my plans for future versions.

1.11.0 RC

DocProject 1.11.0 RC will be deployed within a week after the upcoming Sandcastle release.  It contains several new features and bug fixes, and also a few breaking changes that I'll be sure to mention on the release page.

I expect a few changes to Sandcastle that may prompt the addition of a few new features to DocProject before the next release (e.g., support for Help 1.x and Help 2.x project management and, possibly, the ability to include raw HTML topics using Topic Explorer), but since I can't start working on them until Sandcastle is released, there's obviously going to be a delay.

A week should be more than enough time though since I've taken some time off from paid employment to work on my open source projects - a sacrifice that I can barely afford, but working on my own projects is more fun than writing line of business apps and I just couldn't resist after losing my only client anyway (they lost their biggest client first and had to suspend my services, maybe indefinitely).

Universal Installer

The next version of DocProject, 1.11.0 RC, can install DocProject for Visual Studio 2005 and 2008 simultaneously, including all supported Express editions.  The code base for DocProject 1.x and DocProject 2008 are now identical and as a result DocProject 2008 now uses the 1.x assemblies.

Ending Phase I

I've done a lot of refactoring in 1.11.0 RC, but I still plan to deploy it as the final release candidate (RC) for DocProject 1.x, with compiler optimizations enabled and without any debug symbol databases (.pdb).

Feedback is a must in this release so that I can iron out showstoppers and maybe even add some mission-critical features that people have been waiting for.  If you're a fan of DocProject and you use VS 2005 then now would be the time to get in your suggestions for DocProject 1.x since most of my focus in the next phase of development is going to be on Visual Studio 2008.

Unfortunately I didn't get to add some of the features to 1.11.0 RC as I had planned, but they may be added to 1.12.0 or, more likely, DocProject 2008 Beta 3.

First Production Release

I still expect 1.12.0 to be the first production release of DocProject, but now it will target both Visual Studio 2005 and 2008.  To support this I've reorganized source control, once again, and included the source for other projects as well, such as DocToMaml.  I've also decided not to package the source code with the installer anymore, starting with 1.11.0 RC.

My plans for DocProject 1.x after the first production release are to provide subsequent releases for bug fixes and small feature updates only, targeting both VS 2005 and 2008.  The major features that I have been planning will make their first appearance in DocProject 2008 Beta 3.

DocProject 2008

After I deploy 1.11.0 RC, development will continue on both 1.12.0 RTW and DocProject 2008 Beta 3 simultaneously, although the most notable of new features will only be included in DocProject 2008 Beta 3.  Some of these features are described below.

Please let me know which features you're most interested in so that I can prioritize.

Visual Studio Package

DocProject currently provides an Add-In that only uses VS automation, but I'd like to provide more integration to create a better Topic Explorer, Topic Editor and Topic Designer, and even a preliminary MAML editor.  I've done some initial research and have already come up with a plan to start development.

This is an exciting task to work on but I don't plan on the first release being extremely stable as this will be my first VS package; although, I do plan on reusing a fair amount of the automation code since it has already been tested.  I'll also incorporate much of DocProject's current architecture, including its high level of extensibility and its "openness".

Business logic, infrastructure and utilities will probably end up being mostly the same, but I do want to take this as an opportunity to update some of the designs that DocProject currently uses, like change buffering.

Standalone UI

The current DocProject External UI program relies on DocProjects and DocSites that have been created in Visual Studio.  For help authors that just-so-happen to be developers, this isn't a problem.  But for anyone that isn't a developer, the free Visual Studio Express edition works well but doesn't make much sense since it's obviously not designed just for building documentation, and certainly not for authoring it.

DocProject 2008 Beta 3 will use the new Visual Studio 2008 Shell to provide a tool that has all of DocProject's features, but without requiring Visual Studio to be installed.  I haven't decided yet whether I'm going to use integrated or isolated mode, but I'm leaning towards integrated.

This should be quite easy to create since DocProject is becoming a VS package anyway.  A blocking issue though is whether MSBuild is provided by the Shell redistributable.  If not, then I may just have to continue to deploy the External UI instead.  I'm also not sure whether the Shell provides C# and VB.NET source code editors, but if not it's no big deal since they aren't available in the current External UI either.  (The code editors would be used to edit the project's build process component.  I think this would be a great feature to have in the standalone app, if possible, but it's not a requirement.)

My ultimate goal for the standalone UI is to create a program that works with or without Visual Studio.  It must provide the ability to create new DocProjects and DocSites from existing MSBuild project templates, so they work in Visual Studio as well.  It will also provide all of DocProject's existing and upcoming features, including support for UI extensibility; e.g., providers can register new tool bars and tool windows.

Continuous Integration

A project option will be available to enable support for continuous integration.  When enabled, DocProject will copy all dependencies into the project, including DocProject and Sandcastle assemblies, configuration files, and the HTML Help Workshop program, allowing them to be easily committed to source control.  After a DocProject or DocSite is checked-in, another team member can get the latest version to their computer or to a build server and all the dependencies will be updated as well, without requiring any installation.

The only blocking issue of which I'm aware seems to be that Help 2.x requires installation.  DocProject does not use any of the command-line tools that are installed by the VS SDK, however, it does use the MS Help COM interfaces.  Unfortunately, to get the libraries and COM objects installed you must install the VS SDK, but to install the SDK you must first install Visual Studio.  I'll continue to research this when the time comes to see if I can find a workaround, but so far it doesn't look good.

Continuous Builds

I mentioned this in my previous roadmap post.  The idea is to detect only the changes made to APIs and XML documentation and have a web server that continuously rebuilds a live DocSite so that the updates are visible immediately to all team members.

This feature will require dependency detection and differencing algorithms for both API references and XML documentation.  One question is whether these algorithms will perform better than building entire documentation sets from scratch, but I believe that they certainly will in the scenarios where it really matters.

The goal is that for projects with many large assemblies there should be little delay to see the updates in documentation, at least compared to the very long delay that ensues when rebuilding entire documentation sets from scratch, which is currently the only option with Sandcastle.

To use this feature you would start be creating a new DocSite project and probably selecting some option in the New Project Wizard to indicate that you want continuous build support.  You would also select a VS solution file or an MSBuild project that the DocSite will use to automatically build sources.  In the last wizard step you would add all of the sources to the project as project references, like normal.  When finished configuring the project as usual, you would deploy it to an intranet web server that all team members can access over HTTP.

The live DocSite would trigger a rebuild using MSBuild at a configurable time interval.  This way MSBuild will detect updates automatically and the DocSite will always have the latest versions of all sources.  The DocSite will monitor changes to the sources and continuously rebuild only those topics that were affected.  It will also update the DocSiteContents.xml and DocSiteIndex.xml files, as well as the full-text search index.

One problem might be getting the latest source code onto the server so that the DocSite can rebuild it.  In a team scenario it's probably safe to assume that some sort of source control system is in use.  I'm not sure if it's possible, but if source control systems provide an event model that could be used to automatically get the latest version to the web server after each check-in, that would solve the problem.  Otherwise, manually getting the latest version to the server may be required.  It may be possible to access the source code over a network share (to a build server perhaps) so that it doesn't have to be stored on the web server, although I'm not sure if MSBuild works with UNC.

Using external sources instead of project references may be another option, but replacing the assemblies on a network share automatically might be just as difficult.  It could be done manually or, as with the solution for project references, using source control events to run a build after each (labeled?) check-in.

Localization

I've already put a few hours into researching localization support in DocProject.  What I discovered, and I'm not surprised, is that if I want an implementation that isn't a hack I'll have to add some new features to support dynamic changes to various configuration files during builds.  Anticipating this problem, I had decided to defer localization support into DocProject 2008.

I suspect that these new support features will be useful on their own as well, much in the same way that extension methods in C# 3.0 were probably added to support LINQ yet they are even useful by themselves.  The new features will add to DocProject's already high level of extensibility and will be customizable via the DocProject configuration file.

My goal for localization support is for a single DocProject or DocSite to produce multiple compiled help files for different locales in a single build.  This must be accomplished without affecting the current feature-set of DocProject, including the Sandcastle Build Component drop-down properties and the fact that you can currently change almost anything that you want in the BuildAssembler configuration files.  So in other words, the localization feature must be completely transparent when it's not being used.

All Sandcastle content items and user-written XML documentation comments will be localizable.  Once implemented, I plan to ask the community to create language packs for content items in various locales (Unfortunately, I only know English, and not so well ;).  A language pack will be a .zip file that contains one or more Sandcastle content documents and an XML manifest file, which describes the contents and rules for locale inheritance.  DocProject 2008 will be able to locate the packs in its installation directory, by default, or a user-configured location.

With language packs, DocProject can offer out-of-the-box localization for multiple languages so that help authors can simply check a box and build their project to produce multiple compiled help files, one per language.  Localized XML documentation comments can be added to the project's Help\Comments\[locale] folder (e.g., Help\Comments\en-GB) and it will be merged automatically with XML documentation comments from the source code (how it's merged will depend upon the Merge XML documentation project option, in the same exact way that it works now in the current version of DocProject).  Likewise, localized conceptual topics (MAML) may be added to the project's Help\Topics\[locale] folder.

Instead of trying to write down the many concepts related to implementing localization support that I have echoing around between my ears, I'll just post my notes in their raw form.  If you want more details, please let me know.

1. add content item mappings to DocProject config file: 
  <item name="..." presentationNames="Visual Studio 2005, Sandcastle Hana" source="options-property-name"/> 
  - Items that are added here are automatically removed from the content item documents when a project is 
    first created.
  - Before each build a dynamic content item document is generated in the working directory for all of the 
    items that are configured here.
  - Note that a configChange must be added to register the dynamic content item document. 
2. Add a wizard step, before the Create Shared Content step, that allows a user to specify the values for 
   each of the pre-defined_enum_names: 
  - Documentation set name
  - Running header text
  - Copyright
  - Company name
  - Default locale (combox box) 

3. Add a Localization Management dialog. 
  - Choose the different locales in which distinct documentation sets will be generated by selecting from a 
    drop down list, which will add the selection to another list.
    - one locale may be selected as the "working" locale for the dialog; UI elements will apply only to the 
      "working" locale.
    - a tree view, like solution explorer, shows only the folders that may be localized within the project; 
      e.g., Help\Topics, Help\Comments, etc.
    - one node may be selected at a time.
    - for the selected locale/node, a button can be clicked that will prompt the user to choose a locale from 
      which to import files, with the option of specifying whether the import should be performed recursively.
  - Choose a default locale (corresponds to the wizard step option indicated above).
  - A single build will produce all of the documentation sets in each locale.
  - The BuildContext should have a property that returns the current LCID that's being built and another that 
    returns the list of all LCIDs that are being built.
  - Output will be stored in sub folders named with the locale (e.g., bin\Debug\sp-SP\ or Help\bin\sp-SP).
  - Local input will be stored in sub folders named with the locale (e.g., Help\Topics\sp-SP\, 
    Help\Comments\sp-SP\, Help\Presentation\Style\Content\sp-SP, Help\Presentation\Shared\Content\sp-SP).
  - The default locale's content will be stored at the root;  e.g., Help\Topics\, Help\Comments\, etc. so the 
    build process for the default locale will work the same way that it does now.
  - Source input can be managed in the same way that version management works. 
4. Support dynamic updates to config files at build time. 
  - Take all of the <add> elements in the <configChanges> element and move them under a sub element named, 
    import.
  - Add a new sub element named build that will contain <add> elements for changes to be applied during 
    builds.
  - replacements for build changes will work the same as for import changes, although they will also allow 
    string replacement tokens: {0} {1} {2} ...
    - The values of replacement tokens will be defined by attributes on the <add> element: value0="..." 
      value1="..." value2="..."
    - The values of the replacement attributes are strings that identify the names of public properties on 
      the current derived implementation of the BuildSettings class during builds; e.g., 
      value0="DocumentationSetName"
    - Single-valued properties are converted to strings using Convert.ToString().
    - Properties of System.Collections.IEnumerable implementations are iterated and each element is handled 
      recursively.  Single-valued elements are converted to strings and IEnumerables are iterated.  
      The replacement is executed once for each iteration, recursively.
  - When a build starts the configuration files that will be used are copied to the working directory and 
     then the build changes are applied.
  - Build assembler will use the config files in the working directory. 
5. Support dynamic localization of config files at build time.
  - Add a new sub element to <configChanges> named, localization.
  - replacements for localization changes will work the same as for build changes, including replacement 
    tokens, expect that replacements are specified as properties of the current BuildContext object.
    - BuildEngine implementations should be able to override a method to create a BuildContext 
      implementation, such as SandcastleBuildContext, so that properties such as Lcid and Locale are dynamic.
  - When a localized build starts (not the BuildStarting event but the first step of a localized build, which 
    is executed repeatedly within a single build - once for each locale), the configured localization 
    changes are applied, for the current local, to the working copy of the build assembler config files.  
    Build assembler must then be executed against the localized copy.  (The original copy must not be 
    replaced.) 
** The explanations above were created before this example.  This example illustrates the _better_ approach 
when there are conflicts in the notes above since it's more thought out.
 
- DaveSexton.DocProject.dll.config -
Sandcastle presentation configuration section

<presentations>
  <presentation name="..."
                ... all of the attributes that currently appear in a presentation's add element ... >
    <options>
      <add name="RunningHeaderText" type="System.String" displayName="Running header text"
           description="The text that will appear at the top of each topic." category="Configuration" 
           sort="80" />  <!-- the specified type must be convertible to and from System.String 
                              (serialization is not supported) -->
    </options>
    <content>
      <item name="runningHeaderText" value="RunningHeaderText"/>
            <!-- value is the name of a dynamic project option here, but it can be any property that 
                 exists on a DocProjectOptions implementation -->
    </content>
    <newProjectWizard>
      <step type="type name" assembly="assembly name"/>
            <!-- steps are inserted before the Create Shared Content step in the order that
                 they are added here -->
    </newProjectWizard>
  </presentation> 
  ... more presentations
</presentations> 
<configChanges>
  <import>
    ... all of the stuff that is currently in the configChanges section ... 
    Note that the default comments folder should be changed from Comments\*.xml to Comments\default\*.xml 
    since comments should be segregated by locale; 'default' is used since the default locale can be changed.
  </import>
  <build>
    <add name="assign default reference links locale" ....
         xpath="find ResolveReferenceLinksComponent2 locale attribute"
         setAttribute="{0}"
         value0="Locale" />  <!-- Locale refers to a property (that must be added) on the 
                                  SandcastleSettings class -->
  </build>
  <localization>
    <add name="assign reference links locale" ....
         xpath="find ResolveReferenceLinksComponent2 locale attribute"
         replaceWith="{0}"
         value0="Locale" required="true" />
    <!-- Locale refers to a property on the SandcastleBuildContext class (this class does not currently 
         exist; it will derive from BuildContext) -->
    <!-- The required attribute indicates that append should be used if no node is matched by xpath; 
         although, I'm not sure if I'll be able to determine programmatically whether xpath is selecting an 
         attribute or an element --> 
    <add name="assign localized comments" ....
         xpath="find CopyFromIndexComponent with index name='comments' and select 
                files='Comments\default\*.xml'"
         replaceWith="Comments\{0}\*.xml"
         value0="Locale" /> 
    <add name="assign shared content items" ....
         xpath="find all of the shared content component's content items that start 
                with ..\..\Help\Presentation\"
         replaceWith="..\..\Help\Presentation\content\{0}\{1}" value0="Locale" value1="Regex:[^\\]+.xml"
         collection="true" />  <!-- the collection attribute will indicate whether xpath is expected to 
         return a node collection, in which case this change is applied to each node -->
  </localization>
</configChanges>

I've just deployed Auto-Input Protection 2.0 Beta to CodePlex.  (Finally!)

It was changed from a Production release to a Beta release because there are so many breaking changes, although I'm confident that it works since I've been using it on my blog for the last week or two.  I've even fixed a few bugs during that time.

The release page lists the breaking changes and most of the new features.

Here are a couple that aren't mentioned:

• All of the provider collection elements in the web.config file are now optional.  (It used to be that just the <filters> element was optional.)
• CrossHatchAutoInputProtectionFilterProvider is now built-in.  Instead of using a diagonal cross hatch each time it selects a HatchStyle value at random, although you can configure it to use a single style if you want.
• The ProviderHelper class is now public.  It's useful for parsing configuration values from strings into other types.
• An ASP.NET Web Application project, for testing AIP, is part of the source code provided by the installer.
• You now have the choice of using the ASP.NET cache or session state to store challenges on the server.  With out-of-process session state, AIP can be used in web farm scenarios.

Security

This release fixes a security flaw that was reported for AIP 1.0.0.  If you find any other flaws please don't hesitate to report them to me so that I can try to fix them.

Note that due to the flaw I've added some new behaviors to help secure AIP; however, it may not be appropriate for immediate use in some scenarios.  A timeout is now implemented that will cause validation to fail if a user does not respond in a timely manner (30 seconds).

If you are using the control on a blog (like me) or any other page that contains reading material or data entry fields, then you can increase the default timeout by setting the ValidationTimeout property on the AIP web control.

Alternatively, although not recommended, you can disable this behavior entirely by setting the ValidationKeepAlive property to true on the AIP web control.  Doing so will cause unused challenges to remain on the server indefinitely, which will increase the amount of memory needed upon each request.  (An unused challenge is one that is requested but never answered.)  If you do decide to disable this timeout (by enabling ValidationKeepAlive) then I recommend setting the PersistenceMode property (new to the AutoInputProtection class) to a value of Session so that at least when a user's session expires their unused challenges will too.  You can set this property easily on the autoInputProtection element in your web.config file:

<autoInputProtection persistenceMode="Session"/>

When you enable session state persistence you should also add the new AutoInputProtectionSessionRequestHandler to the configuration file instead of AutoInputProtectionRequestHandler.  Refer to the docs for more information.

Documentation

The first batch of preliminary docs were built for this release.  Naturally, I used DocProject 1.10.1 RC and Sandcastle so that I could automatically generate reference documentation from my triple-slash code comments and write conceptual documentation using MAML.

The AIP installer merges the MS Help 2 docs (.HxS) into Visual Studio 2005 and 2008 automatically.  The HTML Help 1.x docs (.chm) is provided on the release page as a separate download.

A bit off-topic...

For the DocProject users out there, I've learned a few things about Help 2.x since I built the documentation for AIP and I plan to write a tutorial that describes how to:

• Add DocSet attributes to your topics so that your documentation appears when filters are applied in Document Explorer, including custom filters.
• Set the home and default pages.
• Use the Help Integration Wizard to produce a Merge Module for your .HxS file that can be added to a Setup Project.  The wizard automatically generates the required collection-level files and allows you to specify titles, IDs and custom filters that will be installed automatically.  I'll also describe a few showstoppers that you may run into as well.

There's already documentation online for some of this stuff, but I plan to write this tutorial with DocProject in mind.

In the AIP solution there are two merge modules: one for VS 2005 and one for VS 2008.  Both are referenced by the installer project.  I didn't include the DocProject that I used in the solution simply because AIP was written in Visual Studio 2008 and I was using DocProject 1.10.0 RC, which requires Visual Studio 2005.

I've just released the first beta version of ContractN, which is a fully managed programming-by-contract framework that uses the CLR's built-in support for aspect-oriented programming (AOP), which provides a way to seamlessly and intuitively describe the required pre- and post-conditions of a type's contract.

I got the idea based on this blog post by Sasha Goldshtein; although, Sasha's implementation and that of ContractN have little in common other than their purpose.  I liked the way Sasha used dynamically-generated code, but other than that I wasn't satisfied having to create the extra stuff that was required to enforce conditions.

Advantages of ContractN

The real benefit of ContractN, IMO, is that there's no overhead in terms of setting up your classes or their consumers, and you don't have to worry about breaking encapsulation or having to create any supporting interfaces or classes.  You just need to derive an object from ContractN.ProgrammingByContract. (It's a shame that this is required though - I hope AOP is eventually introduced as a C# language feature; although, I suspect that the CLR would have to participate as well to alleviate the need for subclassing.)

ProgrammingByContract is an abstract base class that sets up the AOP infrastructure, which is used to intercept method invocations and define conditions as attributes.  After defining a class that inherits from ProgrammingByContract, simply add the appropriate attributes wherever you'd like to enforce certain conditions on the public contract of your type.  Here's a really simple C# 3.0 example:

public class Person : ContractN.ProgrammingByContract
{
  [InRequired, OutRequired]
  public string Name { get; set; }
}

The example illustrates that there are some built-in conditional attributes for basic argument checking, such as InRequiredAttribute, OutRequiredAttribute and ReturnRequiredAttribute, which throw an exception when an argument (or in the case of the latter two, output parameters and a return value) are null.  I plan to add more conditions in the future, and the framework was designed to be flexible enough so that you can define your own conditions as well.

Custom Conditions

To define a new condition, create a class that derives from either PreConditionBaseAttribute, PostConditionBaseAttribute or ConditionBaseAttribute and override the abstract members.  Usage should be intuitive depending upon the implementation that you need, but you can look at the source code for InRequiredAttribute (beta 1) as an example.

Dynamic Conditions

I decided to take a different approach than Sasha in terms of how dynamic conditions are implemented.  I wanted to take advantage of static type checking instead of hard-coding constraints as strings.  PreConditionAttribute and PostConditionAttribute allow you to specify a method name on the decorated type that will be invoked automatically whenever any public method is called.  That also includes constructors and accessors for properties and events.  (Note that these attributes are also valid on a property, constructor, method or event.)  The method signature can vary depending upon your needs; generally it will be a parameterless instance method, though static methods are allowed as well.  The types of arguments that are accepted encapsulate information about a particular invocation.  For example, you could define a method that accepts an argument typed as ContractN.PropertyCall and the method will be invoked automatically whenever a property get or set accessor is invoked, pre- or post-invocation depending upon the attribute used.  Within these methods you can write code that will actually be part of the class that it's meant to constrain, which maintains encapsulation, makes it easy to share code between constraints and allows access to state with compiler support.

Here's a more sophisticated example than above.  It makes use of pre- and post-conditions described above and also implements ad-hoc conditions by providing an implementation of ICondition:

[PreCondition("TestStatic")]
[PreCondition("TestInstance")]
class AdvancedFeatures : ProgrammingByContract, ICondition
{
    #region Public Properties
    public string TestProperty { get; set; }
    #endregion

    #region Methods
    private static void TestStatic(PropertyCall info)
    {
        Console.WriteLine("Pre Static Property: {0}", info);
    }

    private static void TestStatic(MethodCallBase info)
    {
        Console.WriteLine("Pre Static Default: {0}", info);
    }

    private void TestInstance(MethodCallBase info)
    {
        Console.WriteLine("Pre Instance Default: {0}", info);
    }

    public void NoOperation()
    {
    }
    #endregion

    #region IPreCondition Members
    void IPreCondition.Apply(ConstructorCall info)
    {
        Console.WriteLine("Pre: {0}", info);
    }

    void IPreCondition.Apply(MethodCall info)
    {
        Console.WriteLine("Pre: {0}", info);
    }

    void IPreCondition.Apply(PropertyCall info)
    {
        Console.WriteLine("Pre: {0}", info);
    }

    void IPreCondition.Apply(EventRegistrationCall info)
    {
        Console.WriteLine("Pre: {0}", info);
    }
    #endregion

    #region IPostCondition Members
    void IPostCondition.Apply(ConstructorCall info, MethodCallReturn returnInfo)
    {
        Console.WriteLine("Post: {0}", info);
    }

    void IPostCondition.Apply(MethodCall info, MethodCallReturn returnInfo)
    {
        Console.WriteLine("Post: {0}", info);
    }

    void IPostCondition.Apply(PropertyCall info, MethodCallReturn returnInfo)
    {
        Console.WriteLine("Post: {0}", info);
    }
    
    void IPostCondition.Apply(EventRegistrationCall info, MethodCallReturn returnInfo)
    {
        Console.WriteLine("Post: {0}", info);
    }
    #endregion
}

When the above class is used as follows:

AdvancedFeatures features = new AdvancedFeatures();
features.TestProperty = "A test value";
features.NoOperation();

The console output is:

Pre Static Default: ContractN.ConstructorCall
Post: ContractN.ConstructorCall
Pre: ContractN.PropertyCall
Pre Instance Default: ContractN.PropertyCall
Pre Static Property: ContractN.PropertyCall
Post: ContractN.PropertyCall
Pre: ContractN.MethodCall
Pre Instance Default: ContractN.MethodCall
Pre Static Default: ContractN.MethodCall
Post: ContractN.MethodCall

One of the really neat uses for the PreConditionAttribute that I've found is to have ObjectDisposedException automatically thrown instead of having to check for it everywhere.  Here's an example that builds off of the Person class in my first example:

[PreCondition("NotDisposed")]
public class Person : ContractN.ProgrammingByContract, IDisposable
{
    [InRequired, OutRequired]
    public string Name { get; set; }

    private bool disposed;

    private void NotDisposed()
    {
        if (disposed)
            throw new ObjectDisposedException(this.GetType().FullName);
    }

    public void Dispose()
    {
        disposed = true;
    }
}

After the Dispose method is called, public members are no longer accessible to external types.  All calls will result in ObjectDisposedException automatically!

Conclusion

ContractN is more of a hobby project for me now than anything else, but it has potential and I'd really like to see it blossom into something useful.  If you have any suggestions for improvements please let me know and I'll consider them for the next beta release.  (I'm going to establish a timeline on-the-fly as I get ideas for features.)  I have some interesting ideas for new conditions and I hope to blog about them in the future, so don't wander too far off if you're interested...

Microsoft's Visual Studio Gallery was announced to the public today.  It provides a one-stop-shop for Visual Studio add-ins, packages, item and project templates, macros, controls, coffee makers, toasters, etc.  I think it's a great idea.  Google can only help so much here (have you tried searching for "visual studio add-ins"? ;)

Anyone can register their free or paid products, so if you've got one, let the world know!

I've added DocProject under the Build, Documentation and Web categories.  It has its own little HTML-formatted corner of the site too, which is a cool feature I think.  Take a look at it here.  It already has over 150 unique views!

Currently, the site does not host products so I simply link back to CodePlex (although I love CodePlex so I have no intention of leaving anytime soon).

From my few short discussions with the site's program manager, Anthony, I'd say that they have some good ideas to work with for the future.  Features that'll make locating products and adding new products easier.  So remember to look back occasionally for updates in the next few months.

As you may have already heard, Sandcastle [1] was officially released to the web (RTW) through CodePlex on January 15, 2008.  The license that was chosen is the Microsoft Public License (Ms-PL) [2], but keep an eye on it because it may change [3].  There are still a few unanswered questions that I have [3] about how Microsoft's work on Sandcastle will continue and how it will affect DocProject, but things are looking very positive.

In this blog post I'd like to express a few of my goals and ideas for DocProject and maybe even try to establish some preliminary timeline.  And of course, if you have any thoughts your feedback will be welcomed :)

The End of Phase 1

With the help of community feedback and my propensity for being anti-social in favor of sitting in a dark room, programming, DocProject's long initial development phase is nearing an end.  The next release, which you can read about in my blog (DocProject 1.10.0 RC Preview [4]), brings together a few long-awaited features that will hopefully start to make DocProject into a useful help authoring tool (HAT) [5].  I'm going to try for Monday, January 21st, to release 1.10.0 RC, and I'm hoping that the worst-case scenario will be the following Friday.

From there I expect to deploy only one more Release Candidate that contains various outstanding bug fixes and feature requests and then I'll freeze DocProject's feature set for Visual Studio 2005.  The following deployment, 1.12.0, should therefore be the first DocProject Production release.  Expect that in March, 2008.

Visual Studio 2005 vs. Visual Studio 2008

I expect to have my own licensed copy of VS 2008 Standard edition sometime before the end of February, but until then I've decided that I will not be concentrating on any new deployments for DocProject 2008 Beta, although I'm very excited about using VS 2008 and I'll certainly jump on that bandwagon as soon as I can.

So for now, you can expect development to continue like normal for VS 2005.  But once I start concentrating on VS 2008, I'm probably going to freeze the feature set for VS 2005, distribute the first Production release of DocProject 1.x, and then switch gears for .NET 3.5 and, possibly, VS Package development.  I intended to eventually release DocProject as a true package, hopefully with its own editors and actual project types [6] (i.e., DocProject and DocSite Templates [7] that are recognized by Visual Studio as distinct project types, with their own property pages, etc.).

After DocProject 1.x goes into stabilization (the end of phase 1) I'll certainly provide support for it and help people whenever I can if they have questions about how to modify the source code or how to add additional features themselves.  Although, the only official deployments that you'll see will probably be major releases that fix a large number of bugs.

Well that's my plan anyway because in the last couple of DocProject 2008 releases I've realized that maintaining two versions of the project is almost like double the work; however, since Visual Studio 2008 will allow me to target the .NET 2.0 Framework I may be able to work on both versions of DocProject simultaneously, without much additional effort.  If that's true then hopefully I won't feel the need to freeze the DocProject 1.x feature set.  Although, technically I can't really freeze DocProject's feature set anyway since it's open source ;)

DocProject Documentation

I plan to distribute compiled help for DocProject 1.x that consists of both API reference and conceptual documentation after the first Production release.  I'd say that a week or two should be enough time to author useful, preliminary documentation.  After that I'll probably just add content here and there and deploy whenever I reach certain milestones that I define on the fly.

A .chm will be downloadable from CodePlex and the installer will present you with the option to have DocProject's help content (.HxS) merged with Visual Studio's help.  I'm even considering hosting a DocSite as an online reference and as a great working example of DocProject's and Sandcastle's capabilities.

Phase 2

The next phase will pickup with the development of DocProject 2008, targeting Visual Studio 2008 and the .NET 3.5 Framework.  The timeline will probably start sometime in March, 2008.  From there I hope to deploy on a monthly basis as I've done with DocProject for the past year+ (since December, 2006 as a matter of fact).

Planning

Although DocProject, in the next release [4], will provide first-class support for building mixed reference and conceptual documentation, unbounded filtering capabilities, much improved performance and a mostly finalized API, I still see lots of room for improvement.  Namely, in the area of content-based features such as for authoring topics and localization.  Also, here's a few of the outstanding tasks [8] that I'm going to look into for DocProject 2008: 

• Running DocProject in areas of heightened security (e.g., Vista with UAC enabled).
• SQL Server full-text search provider for the DocSite templates.
• Microsoft Word and Adobe PDF output.
• The ability to annotate comments with developer notes. 
• Automatic token replacement.
• Some new DocSite skins, maybe including one that makes judicious use of SilverLight.  (Documentation might start looking better than the software it documents, if I achieve my goal, that is ;)
• Change control and better support for team scenarios; e.g., the ability to have a server that continuously builds live documentation to keep it up-to-date.
• The development of a simple community wiki for the DocSite templates, which I guess we can start calling the "DocSite Community Wiki" feature (sure, some might say it's a rip-off of the MSDN Community Content Wiki idea, and I guess they'd be correct ;).  This should be very useful in team scenarios for large, volatile, hosted documentation sets.
• The ability to register multiple Build Process Components [9] per project, with a management UI as well.  I may even turn the Sandcastle/Deployment [10] engine into a BPC that can be added to any DocProject or DocSite.
• Other various user-defined tasks (bug fixes and feature requests).

I have even more ideas that I've been tossing around for a while, so I hope to aggregate them into a blog post and eventually add them as work items in CodePlex.  And if you have any ideas for DocProject features please let me know!

Development

Development will target Visual Studio 2008 and the .NET 3.5 Framework, and as I mentioned previously, I'm definitely considering rewriting areas of DocProject as a true VS Package with its own custom editors and project types.  I'm also considering using WPF for the interfaces.  There will be a learning curve but it's one that I'm going to have to accept eventually (and I really do want to learn).

Collaboration

I expect to continue collaborating with community members to find bugs and DocProject's potential for new features.  It seems to me that the experience of DocProject users has been positive, so I'd like to continue working on DocProject in the future in the same way as I've been for the last year.  But if you feel differently please let me know.  Even though it's open source I'd still like to provide an acceptable level of service and support in my free time since the time that I put into helping end-users is also time that I'm putting into DocProject (even if it's just thought), which helps to make a better product even for me to use.

Extensibility

It might also be worth noting that I'm going to maintain the "openness" of DocProject by continuing to introduce new public APIs and areas for extensibility whenever possible.  With the limited feedback that I've received, I'd say that DocProject's high level of extensibility was a success.  A few people have mentioned to me that their businesses have created custom build engine providers [11] and build process components, and I've even used BPCs to help people debug issues in the past.  Actually, my favorite feature of DocProject might actually be the build process component, which, incidentally, doesn't really have anything in particular to do with documentation.

Deployment

It's hard to predict when the first Production release for DocProject 2008 will be deployed, but if I can use the past year as a metric then I'd say another year from now is certainly reasonable, and possibly even longer than might be required.  To be honest, I think the summer of 2008 is a reasonable goal as long as I keep my eyes on the prize.

Phase 3

At the start of phase 3 I hope to have already achieved my major goals for DocProject, which includes much better integration into Visual Studio and additional features that will make authoring documentation extremely simple and, in many cases, automated.  Thanks to Sandcastle I believe that this goal is attainable.

For the future I think I'm aiming for much better external support so that non-Visual Studio users will be able to benefit from DocProject's features as well.  The team-based features that I mentioned previously might get pushed into this phase, but we'll see how it goes.  And finally, I'd like to hear your ideas for where you'd like to see DocProject go.  After I'm able to meet most of my own goals I'd like to brainstorm with the community to come up with some new and innovative ideas to be a part of DocProject in the future.

References

[1] Sandcastle on CodePlex, http://www.codeplex.com/Sandcastle

[2] Microsoft Public License (Ms-PL), http://opensource.org/licenses/ms-pl.html

[3] CTP, License and Source Code, http://www.codeplex.com/Sandcastle/Thread/View.aspx?ThreadId=20557

[4] Dave Sexton's Blog: DocProject 1.10.0 RC Preview, http://davesexton.com/blog/blogs/blog/archive/2008/01/15/docproject-1-10-0-rc-preview.aspx

[5] Help authoring tool. (2008, January 9). In Wikipedia, The Free Encyclopedia. Retrieved 11:22, January 17, 2008, from http://en.wikipedia.org/w/index.php?title=Help_authoring_tool&oldid=183180723

[6] DocProject Work Items: VS 2008 Shell and Extensibility, http://www.codeplex.com/DocProject/WorkItem/View.aspx?WorkItemId=14029

[7] DocProject Components, http://www.codeplex.com/DocProject/Wiki/View.aspx?title=DocProject+Components

[8] DocProject Work Items, Advanced List, http://www.codeplex.com/DocProject/WorkItem/AdvancedList.aspx

[9] DocProject's Build Process, Build Process Components, http://www.codeplex.com/DocProject/Wiki/View.aspx?title=Build+Process#BuildProcessComponent

[10] DocProject Sandcastle/Deployment Plug-In, http://www.codeplex.com/DocProject/Wiki/View.aspx?title=Sandcastle+Deployment+Plugin

[11] DocProject tutorial: Creating a Build Engine Provider, http://www.codeplex.com/DocProject/Wiki/View.aspx?title=Creating+a+Build+Engine+Provider

Somebody from websecurity.com.ua has contacted me regarding a vulnerability of Auto-Input Protection (AIP).  I was able to confirm that reusing an older web form to bypass the AIP web control, even if the CAPTCHA image is no longer valid, is in fact a vulnerability of AIP 1.0.0.

See the original article here:  http://www.websecurity.com.ua/1568/

Although I believe that Security Through Obscurity [1] is not really security at all, one of the purposes of this control is to prevent unwanted SPAM such as automated comments in blogs, and in my experience it works as intended even though this vulnerability has always existed.  In most cases I think the potential risk of the control being bypassed is outweighed by the fact that its use is not related to security.  I think it's probably used mostly to prevent the nuisance of SPAM in blogs (that's how I use it) and forums, and to prevent unwanted user accounts from being generated, which may still be an appropriate use even with its current vulnerability as long as it works for now.

However, I certainly don't recommend AIP 1.0.0 to be used for the authorization of banking transactions or the arming of a nuclear weapon :)

The Attacks

Other than the obvious uses, AIP should be able to counter the following threats to itself (probably not an exhaustive list):

1. Attempts to sequester many auto-generated images for pattern analysis by the bot's owner.
2. Attempts to validate the same challenge repeatedly to learn, in an automated way, about the particular CAPTCHA algorithm being used.
3. Attempts to repeatedly use the same challenge to bypass security.

AIP 1.0.0 does not provide protection for all of the points above.  If you can think of anything else that should make this list please let me know.

The Problem

In an attempt to remedy the situation I realized that it doesn't seem possible to have a 100% secure image-based CAPTCHA implementation because, at the very least, an implementation must be able to verify the submitted text and therefore must keep track of it either on the client or the server:

Client: Raw Text
When included in a response as raw text, such as in a query string or POST data, there is an obvious security flaw since bots can just find the "answer" in the response itself.  (AIP doesn't do this!)

Client: Encrypted Text
When encrypted in POST data, such as part of ASP.NET Control State (AIP does this in 1.0.0), there is an added level of security however it's still fundamentally flawed without taking extra precautions because the same page can be cached by a client and used repeatedly to bypass CAPTCHA protection, even using the same bitmap text and even when the CAPTCHA image has already been invalidated.

Server: Raw Text With Client Identity
When stored on the server, AIP must be able to identify the user agent posting the request as the same agent that requested the CAPTCHA challenge to begin with.  Regardless of whether the text is stored in-proc or out-of-proc, the easiest way to identify the user agent in ASP.NET is to use session state; however, this adds an additional requirement for a client: it must now respond with a session ID each time that it makes a request.  Commonly, a session ID is stored in a cookie on the client's machine; alternatively, it can be part of the request's query string instead.  But in both cases this can make the CAPTCHA susceptible to a Cross-site Request Forgery (CSRF) attack [2].  Though overall I think this can be the most secure solution with the help of absolute expirations.

The Solution

So here's what I've come up with to make AIP more secure.  The improved implementation will be available soon as part of the 2.0.0 Beta release.  Please let me know if you have any thoughts of your own about how security can be beefed up and I'll definitely consider it.

The main security points are:

1. Configurable timeouts for requesting the image and responding with an answer.
2. The same "user" that makes the original request for a page that contains the AIP web control must also be the one to respond with an answer.
3. Only one CAPTCHA test can be active at any given time, per "user".
4. Once an answer is given, the test is invalidated and cannot be used again (i.e., the current in-memory test cannot be used again, regardless of whether the answer was correct or not, but the actual text on the bitmap can still be used again if it happens to be generated, randomly, for a different request altogether).

Here's an outline of the implementation details, with the securest settings available all being enabled:

1. Store the CAPTCHA text on the server instead of in the response and associate it with a value that uniquely identifies the user agent that made the request.
2. Replace the last challenge, if one exists, so that only one at a time may be active.
3. Invalidate the CAPTCHA image once it's requested for the first time after being generated (may not be appropriate in all situations).
4. Minimize the amount of time in which a client has to request the image from the cache after it's been generated.
5. Minimize the amount of time in which a client has to respond to a CAPTCHA challenge (may not be appropriate in all situations).
6. Remove the CAPTCHA text from the server when an answer is submitted, regardless of whether it was correct or not.

1. Store the CAPTCHA text on the server and identify it by the user agent's ID.
The purpose is to ensure that only the same user that requested the CAPTCHA challenge can subsequently validate the CAPTCHA text.  However, this is not a RESTful [3] solution and it may not be 100% secure either since a CSRF attack [2] may be still possible due to the way Session state is implemented in ASP.NET (not to imply that there's a better alternative ;).

2. Replace existing CAPTCHA text.
The purpose of replacing the last challenge is to ensure that each user can only have one active challenge at a time, minimizing the chances that a bot will be able to scoop up multiple images to evaluate them asynchronously, without at least responding with an answer first.  It also cleans up server memory, with no reason to keep track of old tests.

3. Invalidate the CAPTCHA image after it's been requested once.
AIP 1.0.0 uses a sliding expiration for the image.  For added security, the expiration should not be sliding though - once the image is requested for the first time it should be removed from the cache.  But instead of assuming that this is the best approach for every case I've added a boolean property named, BitmapCacheKeepAlive that specifies whether the bitmap should be kept alive or expire immediately upon the first request.  The default value is false, meaning that once the first request is made the image is invalidated.  Requesting the image will result in the same message that appears when the sliding timeout has expired.  When the value is true, the sliding expiration will be used.  Note that if the value is false and an image is never requested then it will be removed from the cache automatically in the timeout specified by the BitmapCacheTimeoutSeconds property.

4. Minimize challenge request time.
This is already implemented in AIP 1.0.0 with the use of a property named, BitmapCacheTimeoutSeconds on the AIP web control. 

When a page with the AIP web control is requested an image is generated on the server and the timeout is started.  When the browser reads the page it will discover an HTML image tag and attempt to download it, but it must request the image from the server within the configured timeout period or else the image's key will be invalid.

This feature can be verified by using the same image URL twice in a row, after waiting for the sliding timeout to expire (the default is 15 seconds).  The first time a page is requested that contains the web control a URL will be generated and the image will be visible in the browser.  Copy the URL from the source and browse to it directly after 15 seconds; the result is an image containing the message: * AIP image not found in cache *.  If you're frequently receiving this message or if users report that your web pages take a while to load then increase the value of the control's BitmapCacheTimeoutSeconds property to give browsers more time to request the image.

5. Minimize challenge response time.
This has been implemented in AIP 2.0.0 as a timeout just like the bitmap request timeout, configurable via a property on the AIP web control named, BitmapTextTimeoutSeconds.  The default value is 30 seconds.  Another property on the AIP web control named, BitmapTextKeepAlive enables or disables the timeout.  The default value is false, indicating that the timeout will be used.  When true the challenge doesn't expire until an attempt is made.

Minimizing the response time may be appropriate for sign-in screens and pages with little information to read, but it's inappropriate in long documents and large data-input forms.  If you're using the control on a page that has a lot of reading material, such as a blog, or with many data entry fields, such as a long user membership form, then you should probably set BitmapTextKeepAlive to true; otherwise, increase the value of BitmapTextTimeoutSeconds to something that is more realistic than 30 seconds so that users will have time to read or enter data before submitting the form.

Alternatively, for a more secure solution, you can put the control on a separate web page by itself and show it after the form is submitted, but before the posted data is saved to your database.  This way you can keep the timeout short; maybe even shorter than 30 seconds, which should help to protect your site against the Human Solvers [4] attack and services such as the one provided by www.captchakiller.com.

6. One shot only.
Regardless of whether the first answer submitted is correct or not, the challenge is invalidated and cannot be used again.  If the value is wrong then a new challenge must be requested.  This feature, along with the new server-side storage, should fix the vulnerability in AIP 1.0.0 discovered by websecurity.com.ua.

How To Exploit AIP 2.0.0 Beta

I don't believe that the solution outlined above is 100% secure.  However, I don't think that any solution really is, but this new implementation should be enough to prevent hackers and spammers from bypassing the control's security when it's configured properly, depending upon the context in which it's being used.

To exploit any vulnerabilities in AIP 2.0.0 a malicious client can't just cache the page anymore with the new security measures in place since the session ID that was used to make the request is required in the response as well, in the form of a cookie or as a query string (depending upon how session state is configured in the site).  But even if the cookie value is obtained, somehow, it will eventually expire due to the session timing out.  If it's used before the timeout, only one challenge may be active at any time, per user, so it's possible that the test is no longer valid anyway.  And once the first answer is provided the challenge is invalidated automatically, regardless of whether the answer is correct or not.

In order for the AIP 2.0.0 web control to be breached, assuming that there aren't more vulnerabilities of which I'm unaware, a malicious client would have to:

1. request the CAPTCHA image within the configured timeout period after the initial request is made for the page that contains the AIP web control.
2. get the answer to the challenge.
3. obtain a valid session cookie (or query string if the website is configured to use cookieless session state).
   1. The session ID must belong to the owner of the session that made the initial request for the page with the AIP web control (not the request for the image).  The owner could be the bot itself or it could possibly use a spoofed or hijacked session.
4. submit the session information with POST data containing the correct answer to the challenge:
   1. within the configured challenge timeout period.
   2. before the user that actually owns the session submits their answer (assuming that the bot is using a spoofed or hijacked session).
   3. and before the user that actually owns the session requests a new challenge (again, assuming misrepresentation).

And finally, once an answer has been submitted the test cannot be used again.

Conclusion

This is still a work in progress.  I'll provide more information, probably in the form of a WIKI on CodePlex, around the time of deployment.  Also, I'd appreciate feedback from anybody that runs tests against the AIP web control to verify its security.  If you find a flaw, please let me know and I'll try to correct it.

References

[1] Security through obscurity. (2007, November 29). In Wikipedia, The Free Encyclopedia. Retrieved 20:40, December 12, 2007, from http://en.wikipedia.org/w/index.php?title=Security_through_obscurity&oldid=174670068

[2] Cross-site request forgery. (2007, December 11). In Wikipedia, The Free Encyclopedia. Retrieved 19:33, December 12, 2007, from http://en.wikipedia.org/w/index.php?title=Cross-site_request_forgery&oldid=177107398

[3] Representational State Transfer. (2007, December 12). In Wikipedia, The Free Encyclopedia. Retrieved 19:34, December 12, 2007, from http://en.wikipedia.org/w/index.php?title=Representational_State_Transfer&oldid=177439704

[4] CAPTCHA. (2007, December 11). In Wikipedia, The Free Encyclopedia. Retrieved 19:33, December 12, 2007, from http://en.wikipedia.org/w/index.php?title=CAPTCHA&oldid=177256271#Human_solvers