public inbox for git-commits@fedoraproject.org
help / color / mirror / Atom feed
From: Yaakov Selkowitz <yselkowi@redhat.com>
To: git-commits@fedoraproject.org
Subject: [flatpaks/flatpak-runtime] f45: Use zstandard for repodata compression
Date: Fri, 11 Sep 2026 14:42:58 GMT	[thread overview]
Message-ID: <178913777801.1.15140028773782217710.flatpaks-flatpak-runtime-6b602f3ba39d@fedoraproject.org> (raw)

            A new commit has been pushed.

            Repo   : flatpaks/flatpak-runtime
            Branch : f45
            Commit : 6b602f3ba39d68946424180c7847792507a14254
            Author : Yaakov Selkowitz <yselkowi@redhat.com>
            Date   : 2026-03-02T20:29:46-05:00
            Stats  : +6/-6 in 1 file(s)
            URL    : https://src.fedoraproject.org/flatpaks/flatpak-runtime/c/6b602f3ba39d68946424180c7847792507a14254?branch=f45

            Log:
            Use zstandard for repodata compression

With the EOL of EPEL 7, Fedora Koji has just switched to zstd-compressed
repodata, which is the new default in createrepo_c.  While there are at
least three Python implementations (pyzstd, zstd, and zstandard), zstandard
seems to be the most widely used.

https://pagure.io/flatpak-module-tools/c/62d74e7b5a3274631886a6500e2b947775a1a275

---
diff --git a/tools/util.py b/tools/util.py
index 4f63412..4822ca4 100644
--- a/tools/util.py
+++ b/tools/util.py
@@ -1,5 +1,4 @@
 from dataclasses import dataclass
-import gzip
 import hashlib
 from pathlib import Path
 import pickle
@@ -9,6 +8,7 @@ import subprocess
 import sys
 import xml.etree.ElementTree as ET
 import xml.sax
+import zstandard
 
 RELEASE = 'f40'
 ID_PREFIX = 'org.fedoraproject'
@@ -181,7 +181,7 @@ def foreach_file(repo_info: RepoInfo, cb):
     filelists_path = repo_info.get_metadata_file("filelists")
 
     handler = FilesMapHandler(cb)
-    with gzip.open(filelists_path, 'rb') as f:
+    with zstandard.open(filelists_path, 'rb') as f:
         xml.sax.parse(f, handler)
 
     done()
@@ -222,7 +222,7 @@ def foreach_package(repo_info: RepoInfo, cb):
 
     primary_path = repo_info.get_metadata_file("primary")
     handler = PackageMapHandler(cb)
-    with gzip.open(primary_path, 'rb') as f:
+    with zstandard.open(primary_path, 'rb') as f:
         xml.sax.parse(f, handler)
 
     done()
@@ -232,10 +232,10 @@ def get_repo_cacheable(name, generate):
     repo_info = RepoInfo.fetch()
     repo_hash = hashlib.sha256(repo_info.repomd_contents).hexdigest()
 
-    cache_path = os.path.join('out', name + ".gz")
+    cache_path = os.path.join('out', name + ".zstd")
 
     try:
-        with gzip.open(cache_path, 'rb') as f:
+        with zstandard.open(cache_path, 'rb') as f:
             old_repo_hash = f.read(64).decode('utf-8')
             if old_repo_hash == repo_hash:
                 start("Reading " + name)
@@ -249,7 +249,7 @@ def get_repo_cacheable(name, generate):
     data = generate(repo_info)
 
     start("Writing " + name)
-    with gzip.open(cache_path, 'wb') as f:
+    with zstandard.open(cache_path, 'wb') as f:
         f.write(repo_hash.encode('utf-8'))
         pickle.dump(data, f)
     done()

                 reply	other threads:[~2026-09-11 14:42 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=178913777801.1.15140028773782217710.flatpaks-flatpak-runtime-6b602f3ba39d@fedoraproject.org \
    --to=yselkowi@redhat.com \
    --cc=git-commits@fedoraproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox