PSA: go.sum is not a lockfile.

[filippo@abyssdomain.expert]

@diazona they don’t apply to dependents. Click on the linked post by Russ Cox for a full explanation.

[orsinium@fosstodon.org]

@filippo

when Russ Cox started publishing his research on packaging and minimal version selection, i was deep into Python packaging, writing my own package manager (DepHell), dependency resolution included. and i was honestly impressed. then we actually got it in Go and practice confirmed the theory: this is THE BEST package manager and dependency resolver ever made. if nothing else, i'd use Go just for that.

[filippo@abyssdomain.expert]

@risottobias expanded a bit the part that says what it's for!

[raito@nixos.paris]

@filippo but there's no hashes in gomod, is there?

[filippo@abyssdomain.expert]

@raito correct, go.mod has the versions, go.sum is a dumb mapping of versions to hashes.

[raito@nixos.paris]

@filippo so I feel like you almost never need to look at go.sum except if you do use a system that allows only fixed output fetches and then go.sum is useful to predict more or less that hash, right?

[diazona@techhub.social]

@filippo Yes I read that post; that's what helped me understand enough to formulate my question. It's about npm, and maybe npm does lockfiles differently, I wouldn't know, but what he's saying there does not accurately describe how any of the Python installers I'm familiar with handle lockfiles.

[filippo@abyssdomain.expert]

@raito yeah, cmd/go/internal/modfetch needs the hashes when downloading contents. But essentially no one reimplements that part.

[filippo@abyssdomain.expert]

@diazona if you add foo to xxx’s dependencies, and foo depends on bar, which version of bar is used?

[diazona@techhub.social]

@filippo Is this about lockfiles? If the installation is being done from a lockfile, then it's whichever version of bar is specified in the lockfile. Otherwise, depends on the resolver and how it's configured, but probably the latest available version of bar, if there's no further constraint on its version.

I'm not sure I see where you're going with this.

[raito@nixos.paris]

@filippo Well, the Nix ecosystem definitely had appetite for that but the usage of dirhash made that quite hard because AFAIK it's not easy to go from dirhash to some raw SHA-256 or similar.

[filippo@abyssdomain.expert]

@diazona what does “being done from a lockfile” mean in this context?

You are in xxx. You add foo. Which version of bar do you get? The latest or the one in foo’s lockfile?

In Go, you get the one in foo’s go.mod. Which is why I say go.mod applies to dependents like manifests and unlike lockfiles, despite having lockfile-like precision.

[filippo@abyssdomain.expert]

@raito yeah it’s a trade-off, dirhash OTOH avoids the need to stabilize the archive format, compression, and metadata including timestamps

[raito@nixos.paris]

@filippo yep, it's fairly understandable why dirhash was chosen; unfortunately, Go (among application ecosystems) is quite unique in having solved this problem. Someday, Nix will support extensible fixed-output primitives so that dirhash can be explained to Nix and then Go modules can be natively downloaded by Nix and fed to the rest of the Go packaging machinery.

[arianvp@functional.cafe]

@raito @filippo isn't NAR basically defunctionalization of dirhash?

[raito@nixos.paris]

@arianvp @filippo i think NAR is more stupid than dirhash

[diazona@techhub.social]

@filippo

"what does 'being done from a lockfile' mean in this context?": You have a program which is capable of installing packages. How do you tell it which packages to install? When I say "being done from a lockfile", I mean that the way you tell the program which packages to install is by giving it a lockfile. (As opposed to, say, giving it the name of a package.)

"You are in xxx. You add foo. Which version of bar do you get? The latest or the one in foo’s lockfile?": What does it mean to be "in" xxx? Based on previous messages, I thought xxx was a package?

P.S. not trying to be difficult here, I'm just genuinely confused

[brouhaha@mastodon.social]

@diazona @filippo
I don't understand why such a thing would be called a lockfile. I've only ever heard "lockfile" used to mean a file that is temporarily created to indicate that some resource should be considered "locked", and unavailable for use by e.g. another process running the same or a related program. In other words, using the filesystem to implement a mutex. (This method only works for programs that agree to the usage convention; the OS kernel doesn't enforce anything.)

[brouhaha@mastodon.social]

@diazona @filippo
But for your question, being "in xxx" where "xxx" is a package name suggests to me that they're trying to describe having the current working directory be either the package's source code directory, or the package's nstallation directory.

[diazona@techhub.social]

@brouhaha @filippo That's a different meaning of lockfile (or lock file).

In software package management, when you tell an installer to install one or more packages by name (possibly with some version constraints), it will choose versions of the packages you named and all their (direct and indirect) dependencies and install those versions. But you don't know which versions it's going to choose. There are thousands/millions/billions/... of possible ways to choose the version of every package that needs to be installed, and typically that includes a lot of choices that wouldn't even work. So people have developed tools that take as input a set of package names and optionally version constraints and spit out a list of one *specific* version of every package in the dependency graph. This is called "locking dependencies", and if the versions are recorded in a file, that is a lockfile. Developers can then test that specific combination of dependency versions and confirm it works.