PSA: go.sum is not a lockfile.

[filippo@abyssdomain.expert]

PSA: go.sum is not a lockfile.

You never need to look at go.sum.

go.mod has everything you need.

https://words.filippo.io/gosum/?source=Mastodon

[risottobias@toot.risottobias.org]

@filippo "then what is it?"

best guess at what could go there?

"go.sum detects tampering in transit or in your local ~/go cache, for specific tagged versions of dependencies; it isn't locked to a particular version, it's the last time you pulled & computed hashes for those tags"

[michael@oldinternet.net]

@filippo omg thank you for this. I was *just* looking into this today, because we got some automated dependency notification that something was outdated because we had like, something v0.0.0-20201119 in go.sum even though the same package in go.mod was 0.39.0. I will link your post to say “we don't need to do anything”!

[diazona@techhub.social]

@filippo Interesting stuff

One nitpick, or perhaps more likely a misunderstanding on my part: I wasn't sure what you meant by lockfiles applying recursively, but if that's referring to pinning the indirect dependencies of the main package, then at least in Python, they *do* apply recursively - at least as far as I understand it, a lockfile is supposed to specify exact versions of every dependency all the way down the graph. The idea is that if you install the exact packages listed in the lockfile and no others in an empty environment, everything should work. Or did you mean something else by applying recursively?

[filippo@abyssdomain.expert]

@michael awesome, that’s exactly what I wrote it for!

You might want to reconsider using whatever generated that notification: if they get something so basic wrong, it’s unlikely they’ll be doing everything else right!

[filippo@abyssdomain.expert]

@diazona they don’t apply to dependents. Click on the linked post by Russ Cox for a full explanation.

[orsinium@fosstodon.org]

@filippo

when Russ Cox started publishing his research on packaging and minimal version selection, i was deep into Python packaging, writing my own package manager (DepHell), dependency resolution included. and i was honestly impressed. then we actually got it in Go and practice confirmed the theory: this is THE BEST package manager and dependency resolver ever made. if nothing else, i'd use Go just for that.

[filippo@abyssdomain.expert]

@risottobias expanded a bit the part that says what it's for!

[raito@nixos.paris]

@filippo but there's no hashes in gomod, is there?

[filippo@abyssdomain.expert]

@raito correct, go.mod has the versions, go.sum is a dumb mapping of versions to hashes.

[raito@nixos.paris]

@filippo so I feel like you almost never need to look at go.sum except if you do use a system that allows only fixed output fetches and then go.sum is useful to predict more or less that hash, right?

[diazona@techhub.social]

@filippo Yes I read that post; that's what helped me understand enough to formulate my question. It's about npm, and maybe npm does lockfiles differently, I wouldn't know, but what he's saying there does not accurately describe how any of the Python installers I'm familiar with handle lockfiles.

[filippo@abyssdomain.expert]

@raito yeah, cmd/go/internal/modfetch needs the hashes when downloading contents. But essentially no one reimplements that part.

[filippo@abyssdomain.expert]

@diazona if you add foo to xxx’s dependencies, and foo depends on bar, which version of bar is used?

[diazona@techhub.social]

@filippo Is this about lockfiles? If the installation is being done from a lockfile, then it's whichever version of bar is specified in the lockfile. Otherwise, depends on the resolver and how it's configured, but probably the latest available version of bar, if there's no further constraint on its version.

I'm not sure I see where you're going with this.

[raito@nixos.paris]

@filippo Well, the Nix ecosystem definitely had appetite for that but the usage of dirhash made that quite hard because AFAIK it's not easy to go from dirhash to some raw SHA-256 or similar.

[filippo@abyssdomain.expert]

@diazona what does “being done from a lockfile” mean in this context?

You are in xxx. You add foo. Which version of bar do you get? The latest or the one in foo’s lockfile?

In Go, you get the one in foo’s go.mod. Which is why I say go.mod applies to dependents like manifests and unlike lockfiles, despite having lockfile-like precision.

[filippo@abyssdomain.expert]

@raito yeah it’s a trade-off, dirhash OTOH avoids the need to stabilize the archive format, compression, and metadata including timestamps

[raito@nixos.paris]

@filippo yep, it's fairly understandable why dirhash was chosen; unfortunately, Go (among application ecosystems) is quite unique in having solved this problem. Someday, Nix will support extensible fixed-output primitives so that dirhash can be explained to Nix and then Go modules can be natively downloaded by Nix and fed to the rest of the Go packaging machinery.

[arianvp@functional.cafe]

@raito @filippo isn't NAR basically defunctionalization of dirhash?