Parsing HTML in Microsoft C#

Most data on the Web is stored in the Hypertext Markup Language (HTML) format. There are many times that you might want to parse HTML in your C# application. However, the .NET framework does not provide an easy way to parse HTML. Evidence of this is the numerous questions posted by C# programmers looking for an easy way to parse HTML.

The Microsoft .NET framework includes extensive support for Extensible Markup Language (XML). However, although XML and HTML look very similar, they are not very compatible. Consider the following major differences between XML and HTML:

• XML requires end tags.
• All XML attribute values must be fully quoted with either single or double quotes.
• XML tags must be properly nested.
• XML tag names are case sensitive.
• XML does not allow duplicate attributes.
• Empty attributes are not allowed in XML.

Let's look at one of these examples in code to illustrate the difference. In XML, every beginning tag must have an ending tag. The following HTML would cause problems for a XML parser.

<p>This is line 1<br>
This is line 2</p>

This is just one of many differences. Of course, you can require HTML to be written in such a way that it is compatible with XML. The preceding HTML could be rewritten as in the following example.

<p>This is line 1<br/>
This is line 2</p>

Both an XML parser and any modern browser could understand this. Unfortunately, this is not a viable solution because you do not control the sources of HTML. You will want your program to be able to process HTML from any source.

The Solution

Because of this, I found it necessary to write my own HTML parser. In this article, I will show you how my HTML parser was constructed, and how you can use this parser with your own applications. I will begin by showing you the main components that make up the HTML parser. I will conclude this article by showing a simple example that uses the HTML parser.

The HTML parser consists of the following four classes:

• Attribute—The attribute class is used to hold an individual attribute inside an HTML tag.
• AttributeList—The attribute list holds an individual HTML tag and all of its attributes.
• Parse—Holds general text parsing routines.
• ParseHTML—The main class that you will interface with; the ParseHTML class is fed the HTML that you would like to parse.

I will now show you how each of these classes functions, and how you will use them. I will begin with the Attribute class.

The Attribute Class

The Attribute class is used to store individual HTML attributes. The source code for the Attribute class can be seen in Listing 1. The following HTML tag demonstrates attributes:

<img src="picture.gif" alt="Some Picture">

The above HTML tag has two attributes named "src" and "alt". The values of these two attributes are "picture.gif" and "Some Picture", respectively.

The Attribute class consists of three properties named "name", "value", and "delim". The "name" property stores the name of the attribute. The "value" property stores the value held by the property. And finally, the "delim" property holds the character that was used to delimit the value, if any. This property will either hold a quote ("), an apostrophe ('), or nothing at all, depending on what was used to delineate the value.

The AttributeList Class

An HTML tag often consists of several attributes. The "AttributeList" class is used to hold a list of these attributes. The "AttributeList" class is shown in Listing 2. The "AttributeList" class consists of a name and a collection of attributes. The "AttributeList" name, stored in a property called "name", holds the name of the tag. When tags are returned to you from the parser, they will be in the form of "AttributeList" objects.

The AttributeList class makes use of the C# indices. You can access individual attributes both by numeric and string indicies. For example, if an attribute "src" were stored in the "AttributeList" object "theTag", you could access the "src" attribute in the following ways:

theTag[0]    // assuming "src" were the first attribute
theTag["src"]

Both of these methods could be used to access the attributes of the tag.