[pacman-dev] Feature: -T0 flag as default ZSTDCOMPRESSION arg

Message ID 249cfd40-ebed-e2bf-3d79-7fe7e5af5d1a@fenste.de
State New
Headers show
Series [pacman-dev] Feature: -T0 flag as default ZSTDCOMPRESSION arg | expand

Commit Message

Frederik Enste Feb. 9, 2021, 10:10 p.m. UTC
Hello pacman devs,

today I installed a big package from the AUR on a clean system and 
forgot to adjust the makepkg.conf beforehand.
As makepkg.conf is set up to compresses all packages with zstd before 
installing them onto the system the compression took a while.

Because of that, I would like to add the -T0 flag to the list of default 
ZSTCOMPRESSION args.
This would allow zstd to spawn multiple compression flags (equal to the 
number of physical processors) and thus speed up package compression.

I have created a simple patch for this feature but as this is my first 
contribution to this project I am not entirely sure that I adjusted all 
necessary files:


 From 6a8d811fbced5a799d88ee357d4b0c694d79e0f7 Mon Sep 17 00:00:00 2001
From: Frederik Enste <frederik@fenste.de>
Date: Tue, 9 Feb 2021 22:43:56 +0100
Subject: [PATCH] added -T0 flag to default zstd compression

Signed-off-by: Frederik Enste <frederik@fenste.de>
---
  doc/makepkg.conf.5.asciidoc | 2 +-
  etc/makepkg.conf.in         | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)

Comments

Eli Schwartz Feb. 9, 2021, 10:19 p.m. UTC | #1
On 2/9/21 5:10 PM, Frederik Enste wrote:
> Hello pacman devs,
> 
> today I installed a big package from the AUR on a clean system and
> forgot to adjust the makepkg.conf beforehand.
> As makepkg.conf is set up to compresses all packages with zstd before
> installing them onto the system the compression took a while.

If you don't plan to distribute or archive the resulting package, you
could also just define PKGEXT=.pkg.tar and completely opt out of
compression.

> Because of that, I would like to add the -T0 flag to the list of default
> ZSTCOMPRESSION args.
> This would allow zstd to spawn multiple compression flags (equal to the
> number of physical processors) and thus speed up package compression.
> 
> I have created a simple patch for this feature but as this is my first
> contribution to this project I am not entirely sure that I adjusted all
> necessary files:
> 
> 
> From 6a8d811fbced5a799d88ee357d4b0c694d79e0f7 Mon Sep 17 00:00:00 2001
> From: Frederik Enste <frederik@fenste.de>
> Date: Tue, 9 Feb 2021 22:43:56 +0100
> Subject: [PATCH] added -T0 flag to default zstd compression
> 
> Signed-off-by: Frederik Enste <frederik@fenste.de>
> ---
>  doc/makepkg.conf.5.asciidoc | 2 +-
>  etc/makepkg.conf.in         | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/doc/makepkg.conf.5.asciidoc b/doc/makepkg.conf.5.asciidoc
> index 04cc5ea9..40129a01 100644
> --- a/doc/makepkg.conf.5.asciidoc
> +++ b/doc/makepkg.conf.5.asciidoc
> @@ -259,7 +259,7 @@ Options
>  **COMPRESSGZ=**"(gzip -c -f -n)"::
>  **COMPRESSBZ2=**"(bzip2 -c -f)"::
>  **COMPRESSXZ=**"(xz -c -z -)"::
> -**COMPRESSZST=**"(zstd -c -z -)"::
> +**COMPRESSZST=**"(zstd -c -z -T0 -)"::
>  **COMPRESSLZO**"(lzop -q)"::
>  **COMPRESSLRZ=**"(lrzip -q)"::
>  **COMPRESSLZ4=**"(lz4 -q)"::
> diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in
> index b13b1d5d..8d655bf8 100644
> --- a/etc/makepkg.conf.in
> +++ b/etc/makepkg.conf.in
> @@ -133,7 +133,7 @@ DBGSRCDIR="/usr/src/debug"
>  COMPRESSGZ=(gzip -c -f -n)
>  COMPRESSBZ2=(bzip2 -c -f)
>  COMPRESSXZ=(xz -c -z -)
> -COMPRESSZST=(zstd -c -z -q -)
> +COMPRESSZST=(zstd -c -z -q -T0 -)
>  COMPRESSLRZ=(lrzip -q)
>  COMPRESSLZO=(lzop -q)
>  COMPRESSZ=(compress -c -f)
Allan McRae Feb. 9, 2021, 11:30 p.m. UTC | #2
On 10/2/21 8:10 am, Frederik Enste wrote:
> Hello pacman devs,
> 
> today I installed a big package from the AUR on a clean system and
> forgot to adjust the makepkg.conf beforehand.
> As makepkg.conf is set up to compresses all packages with zstd before
> installing them onto the system the compression took a while.
> 
> Because of that, I would like to add the -T0 flag to the list of default
> ZSTCOMPRESSION args.
> This would allow zstd to spawn multiple compression flags (equal to the
> number of physical processors) and thus speed up package compression.
> 
> I have created a simple patch for this feature but as this is my first
> contribution to this project I am not entirely sure that I adjusted all
> necessary files:
> 

This list is about the development of pacman, not the packaging of
pacman for Arch Linux.   In the example makepkg.conf provided in the
pacman source, we provide simple examples.  Distributions are free to
adjust that as needed.

Allan

Patch

diff --git a/doc/makepkg.conf.5.asciidoc b/doc/makepkg.conf.5.asciidoc
index 04cc5ea9..40129a01 100644
--- a/doc/makepkg.conf.5.asciidoc
+++ b/doc/makepkg.conf.5.asciidoc
@@ -259,7 +259,7 @@  Options
  **COMPRESSGZ=**"(gzip -c -f -n)"::
  **COMPRESSBZ2=**"(bzip2 -c -f)"::
  **COMPRESSXZ=**"(xz -c -z -)"::
-**COMPRESSZST=**"(zstd -c -z -)"::
+**COMPRESSZST=**"(zstd -c -z -T0 -)"::
  **COMPRESSLZO**"(lzop -q)"::
  **COMPRESSLRZ=**"(lrzip -q)"::
  **COMPRESSLZ4=**"(lz4 -q)"::
diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in
index b13b1d5d..8d655bf8 100644
--- a/etc/makepkg.conf.in
+++ b/etc/makepkg.conf.in
@@ -133,7 +133,7 @@  DBGSRCDIR="/usr/src/debug"
  COMPRESSGZ=(gzip -c -f -n)
  COMPRESSBZ2=(bzip2 -c -f)
  COMPRESSXZ=(xz -c -z -)
-COMPRESSZST=(zstd -c -z -q -)
+COMPRESSZST=(zstd -c -z -q -T0 -)
  COMPRESSLRZ=(lrzip -q)
  COMPRESSLZO=(lzop -q)
  COMPRESSZ=(compress -c -f)