Sunday, December 9, 2018

What hash does Go modules (vgo, go mod) use for verification?





As mentioned in a previous blog, git's hashing mechanism isn't great.  At the time of this writing, the replacement is still not fully ready, let alone used. In my humble opinion, securely verifying code should be a primary function of git.  Git historically has been indifferent on this point.  It's argued git shouldn't be as it isn't security focused, while frustratingly containing a built in function to do just that. This means secure infrastructure must be a layer separate and above git. Languages or other ecosystems that use git must build their own systems.  

Go had no standard means of addressing this issue until Go modules.  Thankfully, Go wisely knew not to delegate this vital security feature to git.  This also allows seamless employment of other versioning system like mercurial or bazaar.  

A project's `go.sum` file can reference an external project like this:
example.com/example v0.0.0-20171218180944-5ea4d0ddac55 h1:jbGlDKdzAZ92NzK65hUP98ri0/r50vVVvmZsFP/nIqo=
In this example, the project didn't have a manually created version number set as a git tag, so Go created one.  `5ea4d0ddac55` is the first part of the git commit SHA-1 hash sufficient in length to reasonably avoid collisions.  A hash is nice to have as a component of the version number, but Go does not use this for anything security related.  Why then use a hash for a version number?  Time isn't sufficient as version should be related to what the code is.  By including parts of the commit hash, small changes will result in a different version number.  

But the line doesn't stop with the version.  The line continues and references "h1".  What an improvement!  Go modules is already anticipating using different hashing methods in the future.  What is h1?  Go modules uses SHA-256 as a hashing algorithm for the verification of code and econdes the hash in Base64.  More specifically, Go modules compresses the whole directory as a `.zip` and then hashes the results.  The local cache directory for the results are kept in `$GOPATH/pkg/mod`.  The file in the cache ending in `.ziphash` contains the same hash as in the `go.sum` file in a given project.  

Here's a couple relevant links to how this is accomplished:

https://github.com/golang/go/blob/master/src/cmd/go/internal/dirhash/hash.go
https://github.com/golang/go/blob/master/src/cmd/go/internal/modcmd/verify.go

Thanks to Go's new module system, gophers no longer have to worry about what git is using. Go's verification mechanism is totally separate from external versioning tools and Go modules uses git only for versioning.

Thanks Go!

3 comments:

  1. Magnificent post, very informative. I wonder why the other specialists of this sector do not notice this. You should continue your writing. I’m confident, you have a great readers‘ base already!
    Tony Daniloo

    ReplyDelete
  2. This comment has been removed by a blog administrator.

    ReplyDelete