April 02, 2008

Auto-Input Protection 2.0 Beta Is Now Available

I've just deployed Auto-Input Protection 2.0 Beta to CodePlex.  (Finally!)

It was changed from a Production release to a Beta release because there are so many breaking changes, although I'm confident that it works since I've been using it on my blog for the last week or two.  I've even fixed a few bugs during that time.

The release page lists the breaking changes and most of the new features.

Here are a couple that aren't mentioned:

  • All of the provider collection elements in the web.config file are now optional.  (It used to be that just the <filters> element was optional.)
  • CrossHatchAutoInputProtectionFilterProvider is now built-in.  Instead of using a diagonal cross hatch each time it selects a HatchStyle value at random, although you can configure it to use a single style if you want.
  • The ProviderHelper class is now public.  It's useful for parsing configuration values from strings into other types.
  • An ASP.NET Web Application project, for testing AIP, is part of the source code provided by the installer.
  • You now have the choice of using the ASP.NET cache or session state to store challenges on the server.  With out-of-process session state, AIP can be used in web farm scenarios.

This release fixes a security flaw that was reported for AIP 1.0.0.  If you find any other flaws please don't hesitate to report them to me so that I can try to fix them.

Note that due to the flaw I've added some new behaviors to help secure AIP; however, it may not be appropriate for immediate use in some scenarios.  A timeout is now implemented that will cause validation to fail if a user does not respond in a timely manner (30 seconds).

If you are using the control on a blog (like me) or any other page that contains reading material or data entry fields, then you can increase the default timeout by setting the ValidationTimeout property on the AIP web control.

Alternatively, although not recommended, you can disable this behavior entirely by setting the ValidationKeepAlive property to true on the AIP web control.  Doing so will cause unused challenges to remain on the server indefinitely, which will increase the amount of memory needed upon each request.  (An unused challenge is one that is requested but never answered.)  If you do decide to disable this timeout (by enabling ValidationKeepAlive) then I recommend setting the PersistenceMode property (new to the AutoInputProtection class) to a value of Session so that at least when a user's session expires their unused challenges will too.  You can set this property easily on the autoInputProtection element in your web.config file:

<autoInputProtection persistenceMode="Session"/>

When you enable session state persistence you should also add the new AutoInputProtectionSessionRequestHandler to the configuration file instead of AutoInputProtectionRequestHandler.  Refer to the docs for more information.


The first batch of preliminary docs were built for this release.  Naturally, I used DocProject 1.10.1 RC and Sandcastle so that I could automatically generate reference documentation from my triple-slash code comments and write conceptual documentation using MAML.

The AIP installer merges the MS Help 2 docs (.HxS) into Visual Studio 2005 and 2008 automatically.  The HTML Help 1.x docs (.chm) is provided on the release page as a separate download.

A bit off-topic...

For the DocProject users out there, I've learned a few things about Help 2.x since I built the documentation for AIP and I plan to write a tutorial that describes how to:

  • Add DocSet attributes to your topics so that your documentation appears when filters are applied in Document Explorer, including custom filters.
  • Set the home and default pages.
  • Use the Help Integration Wizard to produce a Merge Module for your .HxS file that can be added to a Setup Project.  The wizard automatically generates the required collection-level files and allows you to specify titles, IDs and custom filters that will be installed automatically.  I'll also describe a few showstoppers that you may run into as well.

There's already documentation online for some of this stuff, but I plan to write this tutorial with DocProject in mind.

In the AIP solution there are two merge modules: one for VS 2005 and one for VS 2008.  Both are referenced by the installer project.  I didn't include the DocProject that I used in the solution simply because AIP was written in Visual Studio 2008 and I was using DocProject 1.10.0 RC, which requires Visual Studio 2005.

December 12, 2007

AIP 1.0.0 Bypassed

Somebody from websecurity.com.ua has contacted me regarding a vulnerability of Auto-Input Protection (AIP).  I was able to confirm that reusing an older web form to bypass the AIP web control, even if the CAPTCHA image is no longer valid, is in fact a vulnerability of AIP 1.0.0.

See the original article here:  http://www.websecurity.com.ua/1568/

Although I believe that Security Through Obscurity [1] is not really security at all, one of the purposes of this control is to prevent unwanted SPAM such as automated comments in blogs, and in my experience it works as intended even though this vulnerability has always existed.  In most cases I think the potential risk of the control being bypassed is outweighed by the fact that its use is not related to security.  I think it's probably used mostly to prevent the nuisance of SPAM in blogs (that's how I use it) and forums, and to prevent unwanted user accounts from being generated, which may still be an appropriate use even with its current vulnerability as long as it works for now.

However, I certainly don't recommend AIP 1.0.0 to be used for the authorization of banking transactions or the arming of a nuclear weapon :)

The Attacks

Other than the obvious uses, AIP should be able to counter the following threats to itself (probably not an exhaustive list):

  1. Attempts to sequester many auto-generated images for pattern analysis by the bot's owner.
  2. Attempts to validate the same challenge repeatedly to learn, in an automated way, about the particular CAPTCHA algorithm being used.
  3. Attempts to repeatedly use the same challenge to bypass security.

AIP 1.0.0 does not provide protection for all of the points above.  If you can think of anything else that should make this list please let me know.

The Problem

In an attempt to remedy the situation I realized that it doesn't seem possible to have a 100% secure image-based CAPTCHA implementation because, at the very least, an implementation must be able to verify the submitted text and therefore must keep track of it either on the client or the server:

Client: Raw Text
When included in a response as raw text, such as in a query string or POST data, there is an obvious security flaw since bots can just find the "answer" in the response itself.  (AIP doesn't do this!)

Client: Encrypted Text
When encrypted in POST data, such as part of ASP.NET Control State (AIP does this in 1.0.0), there is an added level of security however it's still fundamentally flawed without taking extra precautions because the same page can be cached by a client and used repeatedly to bypass CAPTCHA protection, even using the same bitmap text and even when the CAPTCHA image has already been invalidated.

Server: Raw Text With Client Identity
When stored on the server, AIP must be able to identify the user agent posting the request as the same agent that requested the CAPTCHA challenge to begin with.  Regardless of whether the text is stored in-proc or out-of-proc, the easiest way to identify the user agent in ASP.NET is to use session state; however, this adds an additional requirement for a client: it must now respond with a session ID each time that it makes a request.  Commonly, a session ID is stored in a cookie on the client's machine; alternatively, it can be part of the request's query string instead.  But in both cases this can make the CAPTCHA susceptible to a Cross-site Request Forgery (CSRF) attack [2].  Though overall I think this can be the most secure solution with the help of absolute expirations.

The Solution

So here's what I've come up with to make AIP more secure.  The improved implementation will be available soon as part of the 2.0.0 Beta release.  Please let me know if you have any thoughts of your own about how security can be beefed up and I'll definitely consider it.

The main security points are:

  1. Configurable timeouts for requesting the image and responding with an answer.
  2. The same "user" that makes the original request for a page that contains the AIP web control must also be the one to respond with an answer.
  3. Only one CAPTCHA test can be active at any given time, per "user".
  4. Once an answer is given, the test is invalidated and cannot be used again (i.e., the current in-memory test cannot be used again, regardless of whether the answer was correct or not, but the actual text on the bitmap can still be used again if it happens to be generated, randomly, for a different request altogether).

Here's an outline of the implementation details, with the securest settings available all being enabled:

  1. Store the CAPTCHA text on the server instead of in the response and associate it with a value that uniquely identifies the user agent that made the request.
  2. Replace the last challenge, if one exists, so that only one at a time may be active.
  3. Invalidate the CAPTCHA image once it's requested for the first time after being generated (may not be appropriate in all situations).
  4. Minimize the amount of time in which a client has to request the image from the cache after it's been generated.
  5. Minimize the amount of time in which a client has to respond to a CAPTCHA challenge (may not be appropriate in all situations).
  6. Remove the CAPTCHA text from the server when an answer is submitted, regardless of whether it was correct or not.

1. Store the CAPTCHA text on the server and identify it by the user agent's ID.
The purpose is to ensure that only the same user that requested the CAPTCHA challenge can subsequently validate the CAPTCHA text.  However, this is not a RESTful [3] solution and it may not be 100% secure either since a CSRF attack [2] may be still possible due to the way Session state is implemented in ASP.NET (not to imply that there's a better alternative ;).

2. Replace existing CAPTCHA text.
The purpose of replacing the last challenge is to ensure that each user can only have one active challenge at a time, minimizing the chances that a bot will be able to scoop up multiple images to evaluate them asynchronously, without at least responding with an answer first.  It also cleans up server memory, with no reason to keep track of old tests.

3. Invalidate the CAPTCHA image after it's been requested once.
AIP 1.0.0 uses a sliding expiration for the image.  For added security, the expiration should not be sliding though - once the image is requested for the first time it should be removed from the cache.  But instead of assuming that this is the best approach for every case I've added a boolean property named, BitmapCacheKeepAlive that specifies whether the bitmap should be kept alive or expire immediately upon the first request.  The default value is false, meaning that once the first request is made the image is invalidated.  Requesting the image will result in the same message that appears when the sliding timeout has expired.  When the value is true, the sliding expiration will be used.  Note that if the value is false and an image is never requested then it will be removed from the cache automatically in the timeout specified by the BitmapCacheTimeoutSeconds property.

4. Minimize challenge request time.
This is already implemented in AIP 1.0.0 with the use of a property named, BitmapCacheTimeoutSeconds on the AIP web control. 

When a page with the AIP web control is requested an image is generated on the server and the timeout is started.  When the browser reads the page it will discover an HTML image tag and attempt to download it, but it must request the image from the server within the configured timeout period or else the image's key will be invalid.

This feature can be verified by using the same image URL twice in a row, after waiting for the sliding timeout to expire (the default is 15 seconds).  The first time a page is requested that contains the web control a URL will be generated and the image will be visible in the browser.  Copy the URL from the source and browse to it directly after 15 seconds; the result is an image containing the message: * AIP image not found in cache *.  If you're frequently receiving this message or if users report that your web pages take a while to load then increase the value of the control's BitmapCacheTimeoutSeconds property to give browsers more time to request the image.

5. Minimize challenge response time.
This has been implemented in AIP 2.0.0 as a timeout just like the bitmap request timeout, configurable via a property on the AIP web control named, BitmapTextTimeoutSeconds.  The default value is 30 seconds.  Another property on the AIP web control named, BitmapTextKeepAlive enables or disables the timeout.  The default value is false, indicating that the timeout will be used.  When true the challenge doesn't expire until an attempt is made.

Minimizing the response time may be appropriate for sign-in screens and pages with little information to read, but it's inappropriate in long documents and large data-input forms.  If you're using the control on a page that has a lot of reading material, such as a blog, or with many data entry fields, such as a long user membership form, then you should probably set BitmapTextKeepAlive to true; otherwise, increase the value of BitmapTextTimeoutSeconds to something that is more realistic than 30 seconds so that users will have time to read or enter data before submitting the form.

Alternatively, for a more secure solution, you can put the control on a separate web page by itself and show it after the form is submitted, but before the posted data is saved to your database.  This way you can keep the timeout short; maybe even shorter than 30 seconds, which should help to protect your site against the Human Solvers [4] attack and services such as the one provided by www.captchakiller.com.

6. One shot only.
Regardless of whether the first answer submitted is correct or not, the challenge is invalidated and cannot be used again.  If the value is wrong then a new challenge must be requested.  This feature, along with the new server-side storage, should fix the vulnerability in AIP 1.0.0 discovered by websecurity.com.ua.

How To Exploit AIP 2.0.0 Beta

I don't believe that the solution outlined above is 100% secure.  However, I don't think that any solution really is, but this new implementation should be enough to prevent hackers and spammers from bypassing the control's security when it's configured properly, depending upon the context in which it's being used.

To exploit any vulnerabilities in AIP 2.0.0 a malicious client can't just cache the page anymore with the new security measures in place since the session ID that was used to make the request is required in the response as well, in the form of a cookie or as a query string (depending upon how session state is configured in the site).  But even if the cookie value is obtained, somehow, it will eventually expire due to the session timing out.  If it's used before the timeout, only one challenge may be active at any time, per user, so it's possible that the test is no longer valid anyway.  And once the first answer is provided the challenge is invalidated automatically, regardless of whether the answer is correct or not.

In order for the AIP 2.0.0 web control to be breached, assuming that there aren't more vulnerabilities of which I'm unaware, a malicious client would have to:

  1. request the CAPTCHA image within the configured timeout period after the initial request is made for the page that contains the AIP web control.
  2. get the answer to the challenge.
  3. obtain a valid session cookie (or query string if the website is configured to use cookieless session state).
    1. The session ID must belong to the owner of the session that made the initial request for the page with the AIP web control (not the request for the image).  The owner could be the bot itself or it could possibly use a spoofed or hijacked session.
  4. submit the session information with POST data containing the correct answer to the challenge:
    1. within the configured challenge timeout period.
    2. before the user that actually owns the session submits their answer (assuming that the bot is using a spoofed or hijacked session).
    3. and before the user that actually owns the session requests a new challenge (again, assuming misrepresentation).

And finally, once an answer has been submitted the test cannot be used again.


This is still a work in progress.  I'll provide more information, probably in the form of a WIKI on CodePlex, around the time of deployment.  Also, I'd appreciate feedback from anybody that runs tests against the AIP web control to verify its security.  If you find a flaw, please let me know and I'll try to correct it.


[1] Security through obscurity. (2007, November 29). In Wikipedia, The Free Encyclopedia. Retrieved 20:40, December 12, 2007, from http://en.wikipedia.org/w/index.php?title=Security_through_obscurity&oldid=174670068

[2] Cross-site request forgery. (2007, December 11). In Wikipedia, The Free Encyclopedia. Retrieved 19:33, December 12, 2007, from http://en.wikipedia.org/w/index.php?title=Cross-site_request_forgery&oldid=177107398

[3] Representational State Transfer. (2007, December 12). In Wikipedia, The Free Encyclopedia. Retrieved 19:34, December 12, 2007, from http://en.wikipedia.org/w/index.php?title=Representational_State_Transfer&oldid=177439704

[4] CAPTCHA. (2007, December 11). In Wikipedia, The Free Encyclopedia. Retrieved 19:33, December 12, 2007, from http://en.wikipedia.org/w/index.php?title=CAPTCHA&oldid=177256271#Human_solvers

April 05, 2007

2 in 1: DocProject for Sandcastle 1.4.0 and AIP Beta 1

I've just published 2 open source deployments in one month, one of which is a brand new project: Auto-Input Protection (AIP). Both deployments are available on CodePlex.

DocProject 1.4.0 Release Candidate Is Now Available

DocProject drives the Sandcastle help generation tools using the power of Visual Studio 2005. Choose from various project templates that build compiled help version 1.* or 2.* for all project references. DocProject facilitates the administration and development of project documentation with Sandcastle, allowing you to use the integrated tools of Visual Studio 2005 to customize Sandcastle's output.

Download DocProject from CodePlex, which comes with the complete source code (C#). Try it out and let me know what you think.

Here are some of the new features in 1.4.0:

  • DocProject's Sandcastle build engine now uses the Sandcastle March CTP transformations.
  • DocProjects and DocSites can now build Html Help 1.*, Help 2.*, neither or both via a single build.
  • The New Project wizard has been extended for the Sandcastle build engine, allowing you to choose the type of help that your project will build (see previous feature).
  • DocSites use AJAX to improve client-side performance.
  • DocSite's index can be filtered, client-side.
  • The tools options page, "Dave Sexton's Tools", has been renamed to "DocProject" and was split into separate sub categories.
  • A status notification and progress bar now appear during help builds.
  • Improved error handling, including logging to the Application event log.
  • Several bug fixes.

Use a DocSite template to build an AJAX-enabled ASP.NET Web Application for your compiled help, which includes an interactive TOC, filterable index, breadcrumbs, an auto-generated header and footer, and a link to download the compiled help file. The auto-generated website has been tested for compatibility with the IE7, Firefox and Opera web browsers only.

Example DocSite
Figure 1: Example DocSite shown in Internet Explorer 7 on Windows Vista; Sandcastle vs2005 presentation.

Auto-Input Protection Beta 1 for ASP.NET Is Now Available

AIP is an extensible ASP.NET web control that provides CAPTCHA protection for your blogs, forums, wikis and websites, greatly reducing the likelihood of unwanted form submission from automated spam and hacks.

Want to see it in action? Look no further than in the comments section of my own blog posts. :)

Download AIP from CodePlex, which comes with the complete source code (C#). Try it out and let me know what you think.

Add the AIP web control to a web page with full designer support and only a few modifications to your web.config file:

Default AIP Web Control

Modify the AIP web control's presentation with a custom template and modify the CAPTCHA image that it generates by changing the bitmap and filter providers' settings in your web.config file:

Customized AIP Web Control

Easily implement custom CAPTCHA algorithms that produce randomized patterns with a custom bitmap provider and filter providers:

Customized AIP Web Control - Cross Hatch 1 Customized AIP Web Control - Cross Hatch 2 Customized AIP Web Control - Cross Hatch 3 Customized AIP Web Control - Cross Hatch 4