`StringTokenizer`

Strings often contain many parts. With StringTokenizer we can separate these parts based on a delimiter. This class gets "tokens" from Strings.

This class should be avoided. Instead, using split() with a regular expression is a better approach. StringTokenizer exists for compatibility.

Example

We create a StringTokenizer with an input String containing three substrings separated by commas or spaces. With StringTokenizer we split its tokens apart.

Info HasMoreTokens() returns true if the StringTokenizer contains more tokens. The delimiter is known from the last call to nextToken.

Next NextToken() receives a String. Any of the characters in the string are considered possible delimiters. It returns the next token.

Finally The while-loop will continue iterating infinitely, until the hasMoreTokens() method returns false.

import java.util.StringTokenizer;

public class Program {
    public static void main(String[] args) {

        // Create tokenizer with specified String.
        StringTokenizer tokenizer = new StringTokenizer("123,cat 456");

        // Loop over all tokens separated by comma or space.
        while (tokenizer.hasMoreTokens()) {
            System.out.println(tokenizer.nextToken(", "));
        }
    }
}123
cat
456

`Split`

This is the recommended approach. We tokenize a string with the Split method. We can specify a character set, or use a non-word metacharacter like \s to indicate the separator.

Also Regular expression patterns, and groups, may be used instead of split and the StringTokenizer class.

public class Program {
    public static void main(String[] args) {

        final String value = "123,cat 456";

        // Tokenize the String with a regular expression in Split.
        String[] tokens = value.split("[,\\ ]");
        for (String token : tokens) {
            System.out.println(token);
        }
    }
}123
cat
456

Deprecated classes like StringTokenizer are not recommended in new programs. But knowing about them, and why they are not used, is helpful. It allows us to make better design decisions.

StringTokenizer is more confusing and verbose than split. In the split() program, fewer lines are used, fewer methods are called, and there is less syntax noise. Simpler is better.