Suffixarray

A linear search can be slow. Each element must tested each time. When element count multiplies, this becomes prohibitive. An index is needed.

With suffixarray, an index is built. We use byte slices (which can be based on strings). We call New to create an index, and Lookup to use that index to find positions.

An example

This example uses suffixarray with strings. It first converts the string literal to a byte slice. Then it calls New on the byte slice.

Start The new() method is called with a byte slice argument. Pass -1 to get all indexes found, and any other number to get a specific count.

Result The Lookup method returns an int slice. These are the positions of the indexes found.

Info Lookup returns an int slice, so we can use a for-range loop to iterate over its results.

package main

import (
    "fmt"
    "index/suffixarray"
)

func main() {
    // A byte string.
    value := []byte("cat dog cat cat bird")

    // Create a new suffixarray index.
    index := suffixarray.New(value)

    // Find all instances of this byte slice in the source slice.
    cats := index.Lookup([]byte("cat"), -1)
    fmt.Println(cats)

    // Display all the substrings starting at each index found.
    for i := range cats {
        fmt.Println(string(value[cats[i]:]))
    }

    // Find just one index.
    catsOne := index.Lookup([]byte("cat"), 1)
    fmt.Println(catsOne)

    // Find another index.
    birds := index.Lookup([]byte("bird"), -1)
    fmt.Println(birds)
}[12 8 0]
cat bird
cat cat bird
cat dog cat cat bird
[12]
[16]

Benchmark, Lookup versus loops

Does suffixarray perform faster than a simple for-loop search? Here I test suffixarray versus nested-for loops. Both versions find 3 results.

Version 1 This finds all instances of the bytes "cat" in the data. It uses suffixarray and the Lookup method.

Version 2 This uses nested for-loops to search for the bytes in each position of the string.

Result The suffixarray version is faster. But please note that the suffixarray.New method is not timed.

package main

import (
    "fmt"
    "index/suffixarray"
    "time"
)

func main() {

    // The data we are searching.
    value := []byte("cat dog cat cat bird")
    // The data we search for.
    test := []byte("cat")

    // Create index with suffixarray.
    index := suffixarray.New(value)

    t0 := time.Now()

    // Version 1: use suffixarray method.
    for i := 0; i < 1000000; i++ {
        cats := index.Lookup(test, -1)
        if len(cats) != 3 {
            return
        }
    }

    t1 := time.Now()

    // Version 2: build slice of indexes with for-loops.
    for i := 0; i < 1000000; i++ {
        results := []int{}
        for start := 0; start < len(value); start++ {
            equal := true
            for c := 0; c < len(test); c++ {
                if start + c >= len(value) {
                    break
                }
                if test[c] != value[start + c] {
                    equal = false
                    break
                }
            }
            if equal {
                results = append(results, start)
            }
        }
        if len(results) != 3 {
            return
        }
    }

    t2 := time.Now()
    // Benchmark results.
    fmt.Println(t1.Sub(t0))
    fmt.Println(t2.Sub(t1))
}214.4069ms,    Uses suffixarray, Lookup
290.3657ms,    For-loops

Summary

With suffixarray we achieve an algorithmic optimization of search. We can retrieve multiple matching indexes at once. This module implements optimizations on its own.