From patchwork Fri May 27 16:34:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Morten Linderud X-Patchwork-Id: 2060 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.30) for (single-drop); Fri, 27 May 2022 16:34:50 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id qH4OKqn9kGL+VgYAK+/4rw (envelope-from ) for ; Fri, 27 May 2022 16:34:49 +0000 Received: from lists.archlinux.org (lists.archlinux.org [IPv6:2a01:4f9:c010:9eb4::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id 565D3103E51C; Fri, 27 May 2022 16:34:48 +0000 (UTC) Received: from lists.archlinux.org (localhost [IPv6:::1]) by lists.archlinux.org (Postfix) with ESMTP id 293EC101C6E2; Fri, 27 May 2022 16:34:48 +0000 (UTC) X-Original-To: pacman-dev@lists.archlinux.org Delivered-To: pacman-dev@lists.archlinux.org Received: from mail.archlinux.org (mail.archlinux.org [95.216.189.61]) by lists.archlinux.org (Postfix) with ESMTPS id 0D12D101C6DA for ; Fri, 27 May 2022 16:34:46 +0000 (UTC) From: Morten Linderud DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=archlinux.org; s=dkim-rsa; t=1653669285; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=oUBxlf7fmGXyk4tCpy/CI3X4kOEwsPo9xVXAvdbve/8=; b=uFlBVZ07WijwmIIKL0p2fkooU1UDePTCOFxmmEd0kEsCZnEKB8eVgUQZgawAwMFRShzRxy bCmiPpw0VVG+kzhURi8Q61FZCpgoQ4/lU0s2QtOSjzQnRTxcZ5PlWNFS6RkKMwLjKc8Ufk Jq2B/DNrlM4bk3WqN6gOAlrx8bok10O4wLhGQGHTgIaRut8Q6Ddab58pDZ0fpyqc0EE1+k KM+39Oci7pQumm39d9lsWYwr8tri4T8P6zRoi2kXFHhCbBpx5UWwG8VsIktezEVB3zSqi+ Wx7YLVjm/uPyv75JxS2zuOHbxhwqCYCygFJDbjjKRuZEs+5GA1Y2tG8Ekcg3iEYsgVdPK+ g6gq628JMeqv4qeGgiUa3d9vDpCpQ/g91fwkbHPbTqfhs8pBTue+M61vXHZYsq3jRXJ+hz juov+V49DQnN2s+9W3pHkPa1hC4aMb27g4pknLLVimtCxZoUF+IAhxPtRyFbVJjJiE9Xxb fQeV+ipJVM+adHaRT/CVRklmrQDKu9K/gWbr0Bq9GbmQ2cE9N4oVa9EZbcj/fVfOtH7flA QydsnsT2f3ReVxC3D6FdM3H7DUqi7Kn1nKdhhltMrzKceVripSQxkkXuVZNokBX08ef1ao 23Yvg5Sv4bYqVMFYXfbbUpfTSudLXWoSU2WQ4CLwVU7OfekA48mhU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=archlinux.org; s=dkim-ed25519; t=1653669285; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=oUBxlf7fmGXyk4tCpy/CI3X4kOEwsPo9xVXAvdbve/8=; b=IKggyWHFieEImo5DlB2K9hRSzb5xgmwV0P83UbawdBz0cc+JRhM+jUIhhV9xFRePkTUTRL Rb/nEf5cDWLr0ZBg== To: pacman-dev@lists.archlinux.org Cc: Morten Linderud Subject: [PATCH] strip: Use dwz to compress dwarf headers Date: Fri, 27 May 2022 18:34:28 +0200 Message-Id: <20220527163428.1126160-1-foxboron@archlinux.org> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=pass header.d=archlinux.org header.s=dkim-rsa header.b=uFlBVZ07; dkim=pass header.d=archlinux.org header.s=dkim-ed25519 header.b=IKggyWHF; spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 2a01:4f9:c010:9eb4::1 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org; dmarc=pass (policy=none) header.from=archlinux.org X-Rspamd-Server: mail.archlinux.org X-Spamd-Result: default: False [-2.61 / 15.00]; DWL_DNSWL_MED(-2.00)[archlinux.org:dkim]; MID_CONTAINS_FROM(1.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DMARC_POLICY_ALLOW(-0.50)[archlinux.org,none]; R_MISSING_CHARSET(0.50)[]; RCVD_IN_DNSWL_MED(-0.40)[95.216.189.61:received,2a01:4f9:c010:9eb4::1:from]; MAILLIST(-0.20)[mailman]; R_DKIM_ALLOW(-0.20)[archlinux.org:s=dkim-rsa,archlinux.org:s=dkim-ed25519]; R_SPF_ALLOW(-0.20)[+ip6:2a01:4f9:c010:9eb4::1:c]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[pacman-dev@lists.archlinux.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f9::/32, country:DE]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_THREE(0.00)[3]; FROM_NEQ_ENVFROM(0.00)[foxboron@archlinux.org,pacman-dev-bounces@lists.archlinux.org]; DKIM_TRACE(0.00)[archlinux.org:+]; MIME_TRACE(0.00)[0:+]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; NEURAL_HAM(-0.00)[-1.000]; FORGED_SENDER_MAILLIST(0.00)[] X-Rspamd-Queue-Id: 565D3103E51C From: Morten Linderud `dwz` allows us to compress the DWARF files when building debug packages. Potentially shaving off some of the sizes of the symbols we distribute. https://sourceware.org/dwz/ A sample of packages built with dwz; pacman: Original debug info size: 1520kB Size after compression: 1252kB systemd: Original debug info size: 46692kB Size after compression: 41036kB Signed-off-by: Morten Linderud --- doc/makepkg.conf.5.asciidoc | 3 ++ scripts/libmakepkg/executable/dwz.sh.in | 38 +++++++++++++++++++ .../executable/sepdebugcrcfix.sh.in | 38 +++++++++++++++++++ scripts/libmakepkg/tidy/strip.sh.in | 33 ++++++++++++++-- 4 files changed, 109 insertions(+), 3 deletions(-) create mode 100644 scripts/libmakepkg/executable/dwz.sh.in create mode 100644 scripts/libmakepkg/executable/sepdebugcrcfix.sh.in diff --git a/doc/makepkg.conf.5.asciidoc b/doc/makepkg.conf.5.asciidoc index a0d9a6d4..2e40f27f 100644 --- a/doc/makepkg.conf.5.asciidoc +++ b/doc/makepkg.conf.5.asciidoc @@ -194,6 +194,9 @@ Options DEBUG_CXXFLAGS to their counterpart buildflags. Creates a separate package containing the debug symbols when used with `strip'. + *dwz*;; + Compress dwarf files inside the debug package. + *lto*;; Enable building packages using link time optimization. Adds the flags specified in LTOFLAGS to CFLAGS, CXXFLAGS and LDFLAGS (or diff --git a/scripts/libmakepkg/executable/dwz.sh.in b/scripts/libmakepkg/executable/dwz.sh.in new file mode 100644 index 00000000..a540697c --- /dev/null +++ b/scripts/libmakepkg/executable/dwz.sh.in @@ -0,0 +1,38 @@ +#!/usr/bin/bash +# +# dwz.sh - Confirm presence of dwz binary +# +# Copyright (c) 2011-2022 Pacman Development Team +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +[[ -n "$LIBMAKEPKG_EXECUTABLE_DWZ_SH" ]] && return +LIBMAKEPKG_EXECUTABLE_DWZ_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/option.sh" + +executable_functions+=('executable_dwz') + +executable_dwz() { + if check_option "dwz" "y"; then + if ! type -p dwz>/dev/null; then + error "$(gettext "Cannot find the %s binary required to compress dwarf files.")" "dwz" + return 1 + fi + fi +} diff --git a/scripts/libmakepkg/executable/sepdebugcrcfix.sh.in b/scripts/libmakepkg/executable/sepdebugcrcfix.sh.in new file mode 100644 index 00000000..6259ac32 --- /dev/null +++ b/scripts/libmakepkg/executable/sepdebugcrcfix.sh.in @@ -0,0 +1,38 @@ +#!/usr/bin/bash +# +# sepdebugcrcfix.sh - Confirm presence of sepdebugcrcfix binary +# +# Copyright (c) 2011-2022 Pacman Development Team +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +[[ -n "$LIBMAKEPKG_EXECUTABLE_SEPDEBUGCRCFIX_SH" ]] && return +LIBMAKEPKG_EXECUTABLE_SEPDEBUGCRCFIX_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/option.sh" + +executable_functions+=('executable_sepdebugcrcfix') + +executable_sepdebugcrcfix() { + if check_option "dwz" "y"; then + if ! type -p sepdebugcrcfix>/dev/null; then + error "$(gettext "Cannot find the %s binary required to fix CRC.")" "sepdebugcrcfix" + return 1 + fi + fi +} diff --git a/scripts/libmakepkg/tidy/strip.sh.in b/scripts/libmakepkg/tidy/strip.sh.in index 688bcf1b..8504227d 100644 --- a/scripts/libmakepkg/tidy/strip.sh.in +++ b/scripts/libmakepkg/tidy/strip.sh.in @@ -27,7 +27,7 @@ source "$LIBRARY/util/message.sh" source "$LIBRARY/util/option.sh" -packaging_options+=('strip' 'debug') +packaging_options+=('strip' 'debug' 'dwz') tidy_modify+=('tidy_strip') @@ -52,6 +52,26 @@ source_files() { | sort -zu | tr '\0' '\n' } +compress_dwarf_files(){ + binary_files="$@" + readarray dwz_files < <(find "$dbgdir" -type f -name \*.debug | LC_ALL=C sort) + size_before=$(du -sk "$dbgdir" | cut -f1) + + # The dwz options are taken from rpm/debugedit + LANG=C dwz --hardlink \ + --quiet \ + --relative \ + --low-mem-die-limit 10000000 \ + --max-die-limit 50000000 \ + ${dwz_files[@]} + + size_after=$(du -sk "$dbgdir" | cut -f1) + msg2 "$(gettext "Original debug info size: ${size_before}kB, size after compression: ${size_after}kB")" + + # Some debuggers might use the CRC to find relevant debug files instead of the build-id + LANG=C sepdebugcrcfix "$dbgdir" $binary_files &>/dev/null +} + strip_file() { local binary=$1; shift @@ -142,7 +162,8 @@ tidy_strip() { fi local binary strip_flags - find . -type f -perm -u+w -print0 2>/dev/null | while IFS= read -rd '' binary ; do + declare -a binary_files + while IFS= read -rd '' binary ; do local STRIPLTO=0 case "$(LC_ALL=C readelf -h "$binary" 2>/dev/null)" in *Type:*'DYN (Shared object file)'*) # Libraries (.so) or Relocatable binaries @@ -166,6 +187,12 @@ tidy_strip() { esac strip_file "$binary" ${strip_flags} (( STRIPLTO )) && strip_lto "$binary" - done + binary_files+=("$binary") + done < <(find . -type f -perm -u+w -print0 2>/dev/null) + + if check_option "debug" "y" && check_option "dwz" "y"; then + msg2 "$(gettext "Compressign dwarf files...")" + compress_dwarf_files ${binary_files[@]} + fi fi }