Dot Net Perls

Problem

Develop a method that will remove punctuation and whitespace and return a 'cleaned' (sanitized) version. The method must return a series of words separated by a 1-space delimiter. This output can easily be used by full-text indexes and other applications. The following table shows some example input and output.

C# Solution

You will probably have other requirements, but I present this code to show an approach to the problem. The following static method receives a string, and returns a string stripped of unwanted characters. Here we look at the method and its internals, and my explanations follow.

/// <summary>
/// Static class containing helper string methods.
/// </summary>
static public class StringUtil
{
    /// <summary>
    /// Strip the input string of characters that are not letters or digits.
    /// Replace characters with spaces and then strip all spaces
    /// in a sequence until there is only a single space.
    /// </summary>
    /// <param name="selText">The string you want to process (sanitize).</param>
    /// <returns>The new sanitized version of the string.</returns>
    static public string SanitizeString(string selText)
    {
        // We build up a new string with the StringBuilder, and keep track of spaces
        // with a bool variable.
        StringBuilder res = new StringBuilder();
        bool lastWasSpace = false;

        for (int i = 0; i < selText.Length; i++)
        {
            if (char.IsLetterOrDigit(selText[i]))
            {
                res.Append(selText[i]);
                lastWasSpace = false;
            }
            else if (char.IsWhiteSpace(selText[i]) || char.IsPunctuation(selText[i]))
            {
                // Replace any number of whitespace or punctuation characters
                // in a sequence into a single space.
                if (lastWasSpace == false)
                {
                    res.Append(' ');
                    lastWasSpace = true;
                }
            }
        }
        // Return the sanitized string.
        return res.ToString();
    }
}

• Method is static
  Static methods do not need to save state. This method has no state to save, so it is marked with the static keyword.
• Parses characters
  This is a very dumb parser--a quick and dirty thing. I understand that there are very advanced and nuanced ways to build finite state machines, lexical parsers, and tokenizers, but I didn't need those things
• It is fairly fast
  The method is relatively fast--it won't choke on a very large string. It uses the StringBuilder class as a fast way to append to a buffer.
• ... and flexible
  It has some flexibility due to using the char.Is* static methods. I would like to have a benchmark that set the StringBuilder's capacity in its constructor to a specific value, such as the length of the parameter string.

Conclusion

This code is highly effective and simple way of removing unwanted characters. It performs its task in time linear to the length of the string. I want to conclude by saying that regular expressions can be thought of as sledgehammers, and you sometimes want a pair of pliers, like this method.

Ads by The Lounge