public inbox for git-commits@fedoraproject.org
help / color / mirror / Atom feed
* [flatpaks/flatpak-runtime] f45: Use zstandard for repodata compression
@ 2026-09-11 14:42 Yaakov Selkowitz
  0 siblings, 0 replies; only message in thread
From: Yaakov Selkowitz @ 2026-09-11 14:42 UTC (permalink / raw)
  To: git-commits

            A new commit has been pushed.

            Repo   : flatpaks/flatpak-runtime
            Branch : f45
            Commit : 6b602f3ba39d68946424180c7847792507a14254
            Author : Yaakov Selkowitz <yselkowi@redhat.com>
            Date   : 2026-03-02T20:29:46-05:00
            Stats  : +6/-6 in 1 file(s)
            URL    : https://src.fedoraproject.org/flatpaks/flatpak-runtime/c/6b602f3ba39d68946424180c7847792507a14254?branch=f45

            Log:
            Use zstandard for repodata compression

With the EOL of EPEL 7, Fedora Koji has just switched to zstd-compressed
repodata, which is the new default in createrepo_c.  While there are at
least three Python implementations (pyzstd, zstd, and zstandard), zstandard
seems to be the most widely used.

https://pagure.io/flatpak-module-tools/c/62d74e7b5a3274631886a6500e2b947775a1a275

---
diff --git a/tools/util.py b/tools/util.py
index 4f63412..4822ca4 100644
--- a/tools/util.py
+++ b/tools/util.py
@@ -1,5 +1,4 @@
 from dataclasses import dataclass
-import gzip
 import hashlib
 from pathlib import Path
 import pickle
@@ -9,6 +8,7 @@ import subprocess
 import sys
 import xml.etree.ElementTree as ET
 import xml.sax
+import zstandard
 
 RELEASE = 'f40'
 ID_PREFIX = 'org.fedoraproject'
@@ -181,7 +181,7 @@ def foreach_file(repo_info: RepoInfo, cb):
     filelists_path = repo_info.get_metadata_file("filelists")
 
     handler = FilesMapHandler(cb)
-    with gzip.open(filelists_path, 'rb') as f:
+    with zstandard.open(filelists_path, 'rb') as f:
         xml.sax.parse(f, handler)
 
     done()
@@ -222,7 +222,7 @@ def foreach_package(repo_info: RepoInfo, cb):
 
     primary_path = repo_info.get_metadata_file("primary")
     handler = PackageMapHandler(cb)
-    with gzip.open(primary_path, 'rb') as f:
+    with zstandard.open(primary_path, 'rb') as f:
         xml.sax.parse(f, handler)
 
     done()
@@ -232,10 +232,10 @@ def get_repo_cacheable(name, generate):
     repo_info = RepoInfo.fetch()
     repo_hash = hashlib.sha256(repo_info.repomd_contents).hexdigest()
 
-    cache_path = os.path.join('out', name + ".gz")
+    cache_path = os.path.join('out', name + ".zstd")
 
     try:
-        with gzip.open(cache_path, 'rb') as f:
+        with zstandard.open(cache_path, 'rb') as f:
             old_repo_hash = f.read(64).decode('utf-8')
             if old_repo_hash == repo_hash:
                 start("Reading " + name)
@@ -249,7 +249,7 @@ def get_repo_cacheable(name, generate):
     data = generate(repo_info)
 
     start("Writing " + name)
-    with gzip.open(cache_path, 'wb') as f:
+    with zstandard.open(cache_path, 'wb') as f:
         f.write(repo_hash.encode('utf-8'))
         pickle.dump(data, f)
     done()

^ permalink raw reply related	[flat|nested] only message in thread

only message in thread, other threads:[~2026-09-11 14:42 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-11 14:42 [flatpaks/flatpak-runtime] f45: Use zstandard for repodata compression Yaakov Selkowitz

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox