Compress

Compression trades time for space. It is used often on files and even on web pages. With the compress package in Go we can use gzip on a string.

With NewWriter, we create a new gzip writer. And then we can write compressed bytes to a file with Write. We must close the file when we are done.

First example

Here we have a string. We want to write a compressed version of this string to a file on the disk. We create a file with os.Create.

Then We use NewWriter to create a gzip writer targeting that file we created. We call Write() to write bytes.

Tip We convert the string into a byte value. This can be done with a simple cast expression.

package main

import (
    "compress/gzip"
    "fmt"
    "os"
)

func main() {
    // Some text we want to compress.
    original := "bird and frog"

    // Open a file for writing.
    f, _ := os.Create("C:\\programs\\file.gz")

    // Create gzip writer.
    w := gzip.NewWriter(f)

    // Write bytes in compressed form to the file.
    w.Write([]byte(original))

    // Close the file.
    w.Close()

    fmt.Println("DONE")
}
DONE
bird and frog

Decompress

Here we decompress data from a file on the disk. We first read in the file with os.Open. Then we create a gzip Reader with the gzip.NewReader call.

Start We make a new empty byte slice. The data returned by Read will be stored in this region of memory.

Next We Read the data and it is decompressed. We can then convert the byte slice to a string and display it.

package main

import (
    "compress/gzip"
    "fmt"
    "os"
)

func main() {
    // Open the gzip file.
    f, _ := os.Open("C:\\programs\\file.gz")

    // Create new reader to decompress gzip.
    reader, _ := gzip.NewReader(f)

    // Empty byte slice.
    result := make([]byte, 100)

    // Read in data.
    count, _ := reader.Read(result)

    // Print our decompressed data.
    fmt.Println(count)
    fmt.Println(string(result))
}13
bird and frog

`BestCompression`

We can adjust the GZIP algorithm to emphasize speed or the output's compressed file size. Here we build up a string that contains enough characters to show a difference.

And We then compress the string with NewWriterLevel and gzip.BestSpeed. Then we use BestCompression.

Result The output file that uses BestCompression is 4 bytes smaller on the disk than the version that uses BestSpeed.

package main

import (
    "compress/gzip"
    "fmt"
    "os"
)

func main() {
    test := "this is an example string for testing compression levels"
    test += " here is some more example text"
    test += " cat 123 bird 456 UPPERCASE TEXT"
    test += " __ punct __ // punct"

    // Write with BestSpeed.
    fmt.Println("BESTSPEED")
    f, _ := os.Create("C:\\programs\\file-bestspeed.gz")
    w, _ := gzip.NewWriterLevel(f, gzip.BestSpeed)
    w.Write([]byte(test))
    w.Close()

    // Write with BestCompression.
    fmt.Println("BESTCOMPRESSION")
    f, _ = os.Create("C:\\programs\\file-bestcompression.gz")
    w, _ = gzip.NewWriterLevel(f, gzip.BestCompression)
    w.Write([]byte(test))
    w.Close()
}BESTSPEED
BESTCOMPRESSION      file-bestspeed.gz: 138 bytes
file-bestcompression.gz: 134 bytes

Compress file

This program reads in a file. It uses ioutil.ReadAll to get all the bytes from the file. It then creates a new file by replacing the extension with "gz" and writing it.

Note By decompressing file.gz with a program like 7-Zip, we can be sure that the correct data was compressed.

Note 2 This program could be used to compress files. You can change the paths and the extensions to handle a specific case.

package main

import (
    "bufio"
    "compress/gzip"
    "fmt"
    "io/ioutil"
    "os"
    "strings"
)

func main() {
    // Open file on disk.
    name := "file.txt"
    f, _ := os.Open("C:\\programs\\" + name)

    // Create a Reader and use ReadAll to get all the bytes from the file.
    reader := bufio.NewReader(f)
    content, _ := ioutil.ReadAll(reader)

    // Replace txt extension with gz extension.
    name = strings.Replace(name, ".txt", ".gz", -1)

    // Open file for writing.
    f, _ = os.Create("C:\\programs\\" + name)

    // Write compressed data.
    w := gzip.NewWriter(f)
    w.Write(content)
    w.Close()

    // Done.
    fmt.Println("DONE")
}
DONE
here is some sample text
here is some sample text

A review

The compress package provides built-in gzip support. Other compression technologies are also supported. The level of compression can be chosen.