Home
Go
http.Get Examples: Download Web Pages
Updated Jun 22, 2023
Dot Net Perls
Http.Get. Go can be used to download web pages. This helps with validating a site works correctly (or for trying to get important data from the web page).
With the net http package, we have many helpful funcs for accessing files through HTTP. We do not need to write any low-level code to do this.
Http.Get example. Let us begin with this example program that uses http.Get. Please note we must import the "net/http" package at the top.
Start We use the Get() func to download the home page of Wikipedia—the HTTPS protocol is used here.
Important We require that the Body (a stream) is entirely closed before continuing. The defer keyword waits for the Body to close.
Tip We use the ioutil package to access the ReadAll method—this func is useful in many Go programs.
File
package main import ( "fmt" "io/ioutil" "net/http" ) func main() { // Get Wikipedia home page. resp, _ := http.Get("https://en.wikipedia.org/") // Defer close the response Body. defer resp.Body.Close() // Read everything from Body. body, _ := ioutil.ReadAll(resp.Body) // Convert body bytes to string. bodyText := string(body) fmt.Println("DONE") fmt.Println("Response length: ", len(bodyText)) // Convert to rune slice to take substring. runeSlice := []rune(bodyText) fmt.Println("First chars: ", string(runeSlice[0:10])) }
DONE Response length: 73727 First chars: <!DOCTYPE
Status. When a HTTP request is made, the response includes a status code. We can access the string containing the status with "Status."
And We can access the int code itself with StatusCode. This is probably easier to use when testing for a 404 error.
Tip When using http.Get(), a 404 error is not returned as an error in Go. It is returned as a successful download with a 404 code.
package main import ( "fmt" "net/http" "strings" ) func main() { // This file does not exist. resp, _ := http.Get("https://www.dotnetperls.com/robots.txt") defer resp.Body.Close() // Test status string. fmt.Println("Status:", resp.Status) if strings.Contains(resp.Status, "404") { fmt.Println("Is not found (string tested)") } // Test status code of the response. fmt.Println("Status code:", resp.StatusCode) if resp.StatusCode == 404 { fmt.Println("Is not found (int tested)") } }
Status: 404 Not Found Is not found (string tested) Status code: 404 Is not found (int tested)
Serve. With the ListenAndServe func, also part of the http module, we can build a simple web server in Go. Each response can have a special function with HandleFunc.
ListenAndServe
A review. Making sure your website is working correctly is an important job. And Go is well-suited to download pages and validate them.
Dot Net Perls is a collection of pages with code examples, which are updated to stay current. Programming is an art, and it can be learned from examples.
Donate to this site to help offset the costs of running the server. Sites like this will cease to exist if there is no financial support for them.
Sam Allen is passionate about computer languages, and he maintains 100% of the material available on this website. He hopes it makes the world a nicer place.
This page was last updated on Jun 22, 2023 (edit).
Home
Changes
© 2007-2025 Sam Allen