Scala - List Remove Duplicates

Remove duplicates. A list contains strings: chair, table, table. It has a duplicate string. With Scala 3.3 we have many ways to remove duplicates from our list.

With distinct, a method on the List type, we eliminate duplicates and retain a list's order. Other approaches are possible. We can convert the list to a set: this also dedupes a list.

An example. Let us begin with this example. We create a constant, immutable List of furniture strings. The list has two instances of the string "table."

Detail We invoke the distinct function on the furniture list. A new list, with duplicates removed, is returned.

object Program { def main(args: Array[String]): Unit = { // Create a list of strings. val furniture = List("chair", "bed", "table", "table", "couch") // Get distinct strings. // ... This removes duplicates but retains order. val result = furniture.distinct // Print results. println(furniture) println(result) } }

List(chair, bed, table, table, couch) List(chair, bed, table, couch)

ToSet, toList. Here we use toSet and toList to strip duplicate Ints. We create a list that has six Ints, and two duplicate Ints. We then remove those duplicates.

Note ToSet converts the list to a set. Duplicates are removed because a set cannot store duplicates.

Note 2 We use ToList to convert the set back into a list. The ordering may be changed by the set.

List

object Program { def main(args: Array[String]): Unit = { // Create a list of Ints. val ids = List(10, 10, 1, 2, 3, 3) println(ids) // Convert list to set. // ... Duplicate elements are removed at this step. val set = ids.toSet println(set) // Convert set to list. val ids2 = set.toList println(ids2) } }

List(10, 10, 1, 2, 3, 3) Set(10, 1, 2, 3) List(10, 1, 2, 3)

Map, distinct. With the map method we can transform all elements into a standard form. This may result in duplicates. With distinct we can remove the dupes.

Here We use a lambda expression to map all strings to uppercase forms. A more complex method could be applied.

object Program { def main(args: Array[String]): Unit = { val codes = List("abC", "Abc", "ABC", "xyz", "XyZ") println(codes) // Convert all strings to uppercase. // ... Then get distinct strings. val result = codes.map(_.toUpperCase()).distinct println(result) } }

List(abC, Abc, ABC, xyz, XyZ) List(ABC, XYZ)

Sets, maps. Some collections enforce unique elements. These never need to have duplicates removed—duplicates never occur. Sets and maps cannot store duplicates.

Set

Map

A summary. Removing duplicates from a list is an essential task in programming. It helps us understand basic manipulations of lists. And it is useful in real programs.

Dot Net Perls is a collection of pages with code examples, which are updated to stay current. Programming is an art, and it can be learned from examples.

Donate to this site to help offset the costs of running the server. Sites like this will cease to exist if there is no financial support for them.

Sam Allen is passionate about computer languages, and he maintains 100% of the material available on this website. He hopes it makes the world a nicer place.

This page was last updated on Dec 13, 2023 (edit).

Home

Changes