Learning HTTP caching in Go

Jan 19, 2015

When you are building a web app, you want to make sure that you are caching the app's resources. So if for example a specific URL always delivers the same image, you want to cache it in the browser to avoid unnecessary traffic and have a better web performance.

For the rest of this post I suppose you have the following web application. It generates an image when the /black/ URL is called. I would like to cache that image.

package main
    
import (
    "bytes"
    "flag"
    "image"
    "image/color"
    "image/draw"
    "image/jpeg"
    "log"
    "net/http"
    "strconv"
)
    
var root = flag.String("root", ".", "file system path")

func main() {
    http.HandleFunc("/black/", blackHandler)
    http.Handle("/", http.FileServer(http.Dir(*root)))
    log.Println("Listening on 8080")
    err := http.ListenAndServe(":8080", nil)
    if err != nil {
        log.Fatal("ListenAndServe:", err)
    }
}

func blackHandler(w http.ResponseWriter, r *http.Request) {
    m := image.NewRGBA(image.Rect(0, 0, 240, 240))
    black := color.RGBA{0, 0, 0, 255}
    draw.Draw(m, m.Bounds(), &image.Uniform{black}, image.ZP, draw.Src)
    
    var img image.Image = m
    writeImage(w, &img)
}
    
// writeImage encodes an image in jpeg format and writes it into ResponseWriter.
func writeImage(w http.ResponseWriter, img *image.Image) {
    
    buffer := new(bytes.Buffer)
    if err := jpeg.Encode(buffer, *img, nil); err != nil {
        log.Println("unable to encode image.")
    }
    
    w.Header().Set("Content-Type", "image/jpeg")
    w.Header().Set("Content-Length", strconv.Itoa(len(buffer.Bytes())))
    if _, err := w.Write(buffer.Bytes()); err != nil {
        log.Println("unable to write image.")
    }
}

How to cache?

This subject is new to me, I didn't know you could set some caching information in the HTTP headers (there is so much to learn :-) ).

This is what the HTTP header of /black/ looks like right now:

santiaago$ curl -I localhost:8080/black/
HTTP/1.1 200 OK
Content-Length: 1493
Content-Type: image/jpeg
Date: Sun, 18 Jan 2015 14:17:37 GMT

There are two things that can be done in order to cache this HTTP request:

Setting an etag:

An etag or entity tag is a mechanisms that HTTP provides to deal with caching. If you ask multiple times for the same resource you get the resource for free once it is cached.

The etag is what will identify a specific resource so it should be a unique key in your web app.

In Go this is how you could set the etag in the HTTP header:

func Handler(w http.ResponseWriter, r *http.Request) {
    key := "somekey"
    e := `"` + key + `"`
    w.Header().Set("Etag", e)
    ...

Reading the HTTP request:

Suppose you send the HTTP response to the client and he asks for the same resource again. In this case you need to read the HTTP header of the HTTP.request and check the If-None-Match field. This field will have the etag key value if the client has already asked for that resource.

If there is a match between the If-None-Match field and the key you generate in the server, there is no need to rebuild the image again as the browser already has it. In that case set the HTTP status to StatusNotModified ie 304 and return.

if match := r.Header.Get("If-None-Match"); match != "" {
    if strings.Contains(match, e) {
        w.WriteHeader(http.StatusNotModified)
        return
    }
}

Setting cache-control:

Setting cache-control to a specific date tells the client that once the date expires the cache should revalidate the resource. You can also set the cache-control value to other values. This resource is great to understand cache-control: A Beginner's Guide to HTTP Cache Headers

This is how you could set your cache in Go:

w.Header().Set("Cache-Control", "max-age=2592000") // 30 days

All together:

func blackHandler(w http.ResponseWriter, r *http.Request) {
    
    key := "black"
    e := `"` + key + `"`
    w.Header().Set("Etag", e)
    w.Header().Set("Cache-Control", "max-age=2592000") // 30 days
    
    if match := r.Header.Get("If-None-Match"); match != "" {
        if strings.Contains(match, e) {
            w.WriteHeader(http.StatusNotModified)
            return
        }
    }
    
    m := image.NewRGBA(image.Rect(0, 0, 240, 240))
    black := color.RGBA{0, 0, 0, 255}
    draw.Draw(m, m.Bounds(), &image.Uniform{black}, image.ZP, draw.Src)
    
    var img image.Image = m
    writeImage(w, &img)
}

Testing:

Once your handler is ready to cache, you want to test that everything is in place.

The first thing you can do is take a look at the Network section in Chrome Developer tools or any dev tools you have.

Ask for your resource, in this case localhost:8080/black/. You should get a Status 200 OK, then hit a refresh and check that the Status has now changed to 304 StatusNotModified. You can also notice that the latency of the second request is shorted than the first one.

network of /black request

If you take a look at the HTTP header of the response you should be able to check the Etag and Cache-Control values.

You can also do this with curl by running the following command:

santiaago$ curl -I localhost:8080/black/
HTTP/1.1 200 OK
Cache-Control: max-age=2592000
Content-Length: 1493
Content-Type: image/jpeg
Etag: "black"
Date: Sun, 18 Jan 2015 14:41:31 GMT

The final example is available in this gist

Some links that helped understand this:

Follow me at @santiago_arias to be notified about more posts like this.

Santiaago