How big is a Git commit?

[−]fiddlerwoaroof · 2026-10-11 Sun 05:49 UTC · link

It can get even smaller when loose objects get packed into packfiles because of delta compression: a whole bunch of identical files can be stored in minimal overhead over storing one copy.

[−]bewuethr · 2026-10-11 Sun 08:15 UTC · link

I think the 250 files all containing just "foo" are stored as a single blob, too, even before creating a packfile.

[−]fiddlerwoaroof · 2026-10-11 Sun 08:53 UTC · link

I forgot about that, delta compression still can save quite a bit when you have nearly-identical files.

[−]bewuethr · 2026-10-11 Sun 09:13 UTC · link

Exactly, a tiny change in a large file produces a full new blob, but is stored very efficiently as a delta in a packfile.

[−]gblargg · 2026-10-11 Sun 09:02 UTC · link

Right, packfile size is really what matters in the long-term. A tiny change to a file in the short term just adds the whole thing again (compressed), whereas a tiny change in a packfile shouldn't add much at all. Unless you've got very limited disk space, the long-term is the only size that matters.

[−]masklinn · 2026-10-11 Sun 09:59 UTC · link

> whereas a tiny change in a packfile shouldn't add much at all

Well to the extent that git is able to find a suitable delta, which is the difficult part.

[−]grumbelbart2 · 2026-10-11 Sun 06:55 UTC · link

Did you do —apparent-size?

[−]SushiHippie · 2026-10-11 Sun 10:23 UTC · link

The -b option is equivalent to '--apparent-size --block-size=1'

[−]nathanpankon · 2026-10-11 Sun 07:03 UTC · link

I played with making a git alternative that was streamlined according to workflow. I haven't completely abandoned it, but my core idea was that you can use ast path to compress the data further.

Like you said git is really efficient, and even though I went in with some criticism because of the work(mess) that is chromium and the deep hacking I did there to fix stuff between versions without forcing a complete rebuild

I came to the conclusion that as a compression alternative, my approach wasn't worth it. Git did a better job!

Anyway here's the attempt: https://github.com/pankon/gat

[−]Anduia · 2026-10-11 Sun 07:22 UTC · link

It is not public is it?

[−]nathanpankon · 2026-10-11 Sun 08:17 UTC · link

Good call, made it public :)

Thanks!

[−]Anduia · 2026-10-11 Sun 10:54 UTC · link

Very interesting! Maybe the AST part is more useful for diffs and merges than for compression.

[−]cocoto · 2026-10-11 Sun 07:33 UTC · link

> The du options are: s to summarize and b to display bytes.

Small nitpick: Use long options and your code samples become self explanatory!

[−]NetOpWibby · 2026-10-11 Sun 08:05 UTC · link

I always use long options when possible. I don’t need terseness, computers do. I wanna help future me figure out how to run something.

[−]analog_daddy · 2026-10-11 Sun 09:29 UTC · link

This is one of my biggest pet peeves! I hate reading shell scripts which uses small options. It is okay to use short options when interactively using shell, but there is no excuse to not spend time and try converting a small options shell script to use long options if its going to be read by someone else, especially if you anyway have to write comments explaining it. The biggest part i hate is that, some utilities don’t have long option counterpart!

[−]zetanor · 2026-10-11 Sun 11:20 UTC · link

Not everyone uses GNU coreutils.

[−]CodesInChaos · 2026-10-11 Sun 08:38 UTC · link

Does git support better compression algorithms than deflate, like zstd or brotli?

[−]masklinn · 2026-10-11 Sun 09:56 UTC · link

No. You can configure the compression level but that's at far as it goes.

You can configure loose object and pack compression separately (core.compression and pack.compression) so loose object compression could be switched to an other algorithm as it's completely local[1] (likely a low-compression zstd), but pack files are part of the network exchange, so things are a bit more complicated there. Also an annoyance there is that git uses raw zlib there is no identifying header or anything.

There are old threads on the mailing list, as well as a gitlab issue.

Although you can also tune compression per pack when you create them explicitly, on $DAYJOB's repository in order to speed up gc some I have .keep packs which segregate images uncompressed in their own packs, because there's little point wasting cycles on zlib-compressing jpegs and pngs.

[1]: unless you're using the "dumb http" protocol, or performing (n)fs-mediated clones

[−]CodesInChaos · 2026-10-11 Sun 10:43 UTC · link

I'm a bit surprised Github or some other big git hoster didn't push for such a protocol improvement. Since I assume that smaller and cheaper to create packs would reduce their costs noticeably.

[−]jamesblonde · 2026-10-11 Sun 10:21 UTC · link

Git was designed with assumptions about local file storage which make it challenge when storing data in a distributed file system. We built our distributed filesystem, HopsFS, to store data in S3. To support high performance git in FUSE, our writes hit remote NVMe disk(s) and asynchronously sync to S3 (you can replicate on NVMes or just do failure recovery). Previously, we used remote NVMe disks as a write-through cache, but that doesn't work with git, S3 latency is too high.