Remove Duplicates From ListUse the distinct function, along with map, to remove duplicate elements from lists.
This page was last reviewed on Apr 21, 2023.
Remove duplicates. A list contains strings: chair, table, table. It has a duplicate string. With Scala we have many ways to remove duplicates from our list.
With distinct, a method on the List type, we eliminate duplicates and retain a list's order. Other approaches are possible. We can convert the list to a set: this also dedupes a list.
An example. Let us begin with this example. We create a constant, immutable List of furniture strings. The list has two instances of the string "table."
Detail We invoke the distinct function on the furniture list. A new list, with duplicates removed, is returned.
// Create a list of strings. val furniture = List("chair", "bed", "table", "table", "couch") // Get distinct strings. // ... This removes duplicates but retains order. val result = furniture.distinct // Print results. println(furniture) println(result)
List(chair, bed, table, table, couch) List(chair, bed, table, couch)
ToSet, toList. Here we use toSet and toList to strip duplicate Ints. We create a list that has six Ints, and two duplicate Ints. We then remove those duplicates.
Note ToSet converts the list to a set. Duplicates are removed because a set cannot store duplicates.
Note 2 We use ToList to convert the set back into a list. The ordering may be changed by the set.
// Create a list of Ints. val ids = List(10, 10, 1, 2, 3, 3) println(ids) // Convert list to set. // ... Duplicate elements are removed at this step. val set = ids.toSet println(set) // Convert set to list. val ids2 = set.toList println(ids2)
List(10, 10, 1, 2, 3, 3) Set(10, 1, 2, 3) List(10, 1, 2, 3)
Map, distinct. With the map method we can transform all elements into a standard form. This may result in duplicates. With distinct we can remove the dupes.
Here We use a lambda expression to map all strings to uppercase forms. A more complex method could be applied.
val codes = List("abC", "Abc", "ABC", "xyz", "XyZ") println(codes) // Convert all strings to uppercase. // ... Then get distinct strings. val result = codes.map(_.toUpperCase()).distinct println(result)
List(abC, Abc, ABC, xyz, XyZ) List(ABC, XYZ)
Sets, maps. Some collections enforce unique elements. These never need to have duplicates removed—duplicates never occur. Sets and maps cannot store duplicates.
A summary. Removing duplicates from a list is an essential task in programming. It helps us understand basic manipulations of lists. And it is useful in real programs.
Dot Net Perls is a collection of tested code examples. Pages are continually updated to stay current, with code correctness a top priority.
Sam Allen is passionate about computer languages. In the past, his work has been recommended by Apple and Microsoft and he has studied computers at a selective university in the United States.
This page was last updated on Apr 21, 2023 (edit).
© 2007-2023 Sam Allen.