Invoke Seq.distinct to remove duplicates from a list. Convert from the seq back into a list or array.
Remove duplicates. A list contains numbers: 10, 20, 10, 30. There are two tens. But we only want distinct values in this list. With F# we can eliminate dupes.
With Seq functions, we can apply common operations to lists. The distinct function receives a list. It eliminates the second occurrence of values in the list.
An example. Let us begin. We have a list of popular drinks. But there are two "coffee" strings in the list. We then call Seq.distinct and pass the list to it.
Result: The Seq.distinct function returns a seq instance. The second "coffee" was removed, and no other changes were made.
Finally: We use Seq.toList and Seq.toArray to convert our sequence to other types. These may be more useful in other parts of the program.
F# program that uses distinct, removes duplicates// This list has a duplicate string.
let drinks = ["coffee"; "beer"; "tea"; "coffee"; "milk"]
// Use Seq.distinct to remove duplicates.
// ... The order is retained.
let result = Seq.distinct drinks
printfn "%A" result
// We can convert from the Seq to a list or array.
let resultList = Seq.toList result
let resultArray = Seq.toArray result
printfn "%A" resultList.Length
printfn "%A" resultArray.Length
Output
seq ["coffee"; "beer"; "tea"; "milk"]
4
4
DistinctBy. Sometimes duplicate-removing (dedupe) can be augmented with transformations. For example we can trim whitespace from strings before comparing them.
Here: We use distinctBy and pass it a lambda expression. The lambda returns a trimmed string by calling Trim.
Result: The seq has no duplicates. The leading and trailing whitespace is ignored by distinctBy.
F# program that uses distinctBy, ignores whitespace// Has some duplicates if we ignore whitespace.
let codes = [" ABC "; "DEF"; "ABC"; "XYZ"; " XYZ"]
// Use Seq.distinctBy to trim strings before removing duplicates.
let result = Seq.distinctBy (fun (a : string) -> a.Trim()) codes
printfn "%A" result
Output
seq [" ABC "; "DEF"; "XYZ"]
Count by value. This function counts elements by value—if a sequence has two "Red" values it returns 2. It uses "where" to create an intermediate sequence with only matching elements.
And: With the pipeline operator, it returns the length of the sequence, which is the count of matching elements in the sequence.
Tip: This does not remove duplicates. But it tells us whether duplicates are present for a certain value.
F# program that counts instances in list// This count function accepts two arguments: value and list.
// ... It computes a Seq where the value equals the argument.
// It returns the length of that sequence.
let count value list =
Seq.where (fun y -> y = value) list
|>Seq.length
// Has two red values.
let colors = ["Blue"; "Red"; "Red"; "Orange"]
// Count blue values.
let total1 = count"Blue" colors
printfn "Blue = %A" total1
// Count red values.
let total2 = count"Red" colors
printfn "Red = %A" total2
Output
Blue = 1
Red = 2
Int array. This program uses a pipelined expression to remove duplicate elements from an Int array. The result is a new Int array.Array
Description: We use Seq.ofArray to treat the array as a sequence. We then call distinct, and then toArray to return an array.
F# program that removes duplicates from Int array
let numbers = [|10; 10; 20; 30; 40; 40|]
// Use ofArray to use an array as a sequence.
// ... Use distinct to eliminate duplicates.
// Convert back into an Int array.
let result =
Seq.ofArray numbers
|> Seq.distinct
|> Seq.toArray
printfn "%A" result
Output
[|10; 20; 30; 40|]
A review. With sequences in F#, we use prebuilt logic to process data. And we can combine operations into more powerful, sequence-based ones. This is powerful. It is elegant.
