Duplicate chars

Often a string contains many characters, and some of these are duplicates. And sometimes we want to remove the duplicated data.

With a special method, we can remove duplicate characters, leaving only the first ones that appear. Many methods can be used, but a simple method with nested loops may suffice.

Consider this method

We have a Function that receives (and returns) a String. We create 2 character arrays as the method begins.

Info The table stores each character as we encounter it. We can see if a character has been encountered by scanning this table.

Start We build up our string in a Char array. We convert it to a string at the end of the method.

Then We loop over the argument string. We see if the character has been encountered before by scanning the table.

Finally When we are on a new Char we have not seen before, we add it to our result string and to the "encountered" table.

Module Module1

    Function RemoveDuplicateChars(ByVal value As String) As String
        ' This table stores characters we have encountered.
        Dim table(value.Length) As Char
        Dim tableLength As Integer = 0

        ' This is our result.
        Dim result(value.Length) As Char
        Dim resultLength As Integer = 0

        For i As Integer = 0 To value.Length - 1
            Dim current As Char = value(i)
            Dim exists As Boolean = False

            ' Loop over all characters in the table of encountered chars.
            For y As Integer = 0 To tableLength - 1
                ' See if we have already encountered this character.
                If current = table(y) Then
                    ' End the loop.
                    exists = True
                    y = tableLength
                End If
            Next

            ' If we have not encountered the character, add it.
            If exists = False Then
                ' Add character to the table of encountered characters.
                table(tableLength) = current
                tableLength += 1

                ' Add character to our result string.
                result(resultLength) = current
                resultLength += 1
            End If
        Next

        ' Return the unique character string.
        Return New String(result, 0, resultLength)
    End Function

    Sub Main()

        ' Test the method we wrote.
        Dim test As String = "having a good day"
        Dim result As String = RemoveDuplicateChars(test)
        Console.WriteLine(result)

        test = "areopagitica"
        result = RemoveDuplicateChars(test)
        Console.WriteLine(result)
    End Sub

End Modulehaving ody
areopgitc

Notes, performance

The algorithm is not as fast on all strings as possible. But it can handle long strings because it only needs to check up to 128 ASCII characters for duplicates.

Note If your strings are not ASCII then this will not apply and the algorithm may be slower.

Tip Consider a 1000 character string. If we loop over it to check for duplicates each time, we could have a slow operation.

But With the "encountered" table, we would have a limit of 128 ASCII characters, making the operation faster.

This duplicate-char removal algorithm is not optimal for all strings. And a specialized method would work better for many programs.