Implementing A Lexical Walker


There was a need for a mechanism for walking over the elements’ object grid once it was finished, to use the lexical analyzer to its fullest extent. This is where walkers come into play.
The walkers’ design is simple. Simply implement the base class WalkerBase and add code that does whatever you want whenever a certain element is stumbled upon. There are three basic implementations that come with nGineer – CodeWalker (persists the code back to a file), XmlWalker (which serializes the object grid to Xml) and HighlightedCodeWalker (on which webify is based).

The ILexicalElement interface

In order to enumerate over the elements and the elements contained in them, recursively, there was a need for a solid, unified interface that would be implemented by all lexical elements and return the instances’ child elements.
This is where ILexicalElement came into play, forcing each implementing element to return its child elements.

public interface ILexicalElement
ILexicalElement[] ChildElements { get; }

Constants (keywords, etc.) were returned using the Constant class and string values (characters, etc.) were returned using the StringValue class.

This would allow any iterating class to view the whole object graph, regardless of the objects which it is made up of.

The Basic Walker Behaviour

The abstract nGineer.LexicalGrammar.Walkers.LexicalWalkerBase class walks on each element in the following manner:

  1. Calls the BeginWalkXXX method (XXX being the type of element), which in turn calls the BeginWalkCommon method.
  2. Walks over all the child elements for that particular element.
  3. Calls the EndWalkXXX method (XXX being the type of element), which in turn calls the EndWalkCommon method.

This structure allows a deriving class to traverse over all the elements as easily as possible. The implementer should only focus on overriding the appropriate element methods.

Walker Examples

Several examples of walker implementations come with nGineer:

The Code Walker (nGineer.LexicalGrammar.Walkers.CodeWalker)

This walker simply overrides the BeginWalkStringValue and BeginWalkConstant, which are the only elements with actual text in them, than saves them to a StringBuilder. When done, the StringBuilder contains the original source file.

The Xml Walker (nGineer.LexicalGrammar.Walkers.XmlWalker)

This walker, in addition to working with Constants and StringValues, works with BeginWalkCommon and EndWalkCommon in order to create an xml element for each element.
As you may recall from the previous article, some classes have base classes because the deriving element “is a” base element. The IncludeInheritanceHierarchy tells the walker to either walk over base classes or not.

The Highlighted Code Walker (nGineer.LexicalGrammar.Walkers.HighlightedCode)

This is the most complicated walker, which creates an Html representation of the code, highlighting Keywords, Comments and the appropriate parts of pre-processing declarations.


If you have any questions or you think I have left something out, please contact me.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s