From patchwork Sat May 28 21:56:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Morten Linderud X-Patchwork-Id: 2063 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.30) for (single-drop); Sat, 28 May 2022 22:37:34 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id IJtBNS2kkmKlPgcAK+/4rw (envelope-from ) for ; Sat, 28 May 2022 22:37:33 +0000 Received: from lists.archlinux.org (lists.archlinux.org [IPv6:2a01:4f9:c010:9eb4::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id A0CF6104B5C1; Sat, 28 May 2022 22:37:31 +0000 (UTC) Received: from lists.archlinux.org (localhost [IPv6:::1]) by lists.archlinux.org (Postfix) with ESMTP id 44796102F3BD; Sat, 28 May 2022 22:37:31 +0000 (UTC) X-Original-To: pacman-dev@lists.archlinux.org Delivered-To: pacman-dev@lists.archlinux.org Received: from mail.archlinux.org (mail.archlinux.org [IPv6:2a01:4f9:c010:3052::1]) by lists.archlinux.org (Postfix) with ESMTPS id 0688D102E222 for ; Sat, 28 May 2022 21:56:17 +0000 (UTC) From: Morten Linderud DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=archlinux.org; s=dkim-rsa; t=1653774975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=8qtR7L++KmXT+NZUcNqwhN1TBUqQDnlIvUiyoarExBQ=; b=vURbAwP2ArJtvX3drnmbvy3nVI79GQsSviq7VNYkhlP2X3bX+AhtNJEY1RJI34OUrCxcEq izFREIR9so9xdynkhwbaWcC3DeO30b9PJHgB341JdkFLchRku9e8yllbRwUFbUHBKhOrog zyI6bOsTpzPZSq5GYeeWCqdGzJczFh0wkBqt3K7ebKZ69JFAh1tFjHPDKwFL1bEWw+i4S+ IJGP8yuZ/eR0Q2I7vPW6hvq+xy5n+CVkJvTLeE9+2y6OTW3kCANcXHxZp82xxEJNrZ5ix4 6QfWDNZGRwJ6yBFVHLRKlXG9tfsJOYlszQUUUW57YWZvBksKjM4lEn6kFi1Z20Z6HlXg4Z 2543boxCVoxB/ldKpHDVuOi/WBE9XrScdycoSGtpCXEEA8qjJKsA6u4jz9puQOLWgIm6RI e86dzSrMISakMrwcVjq1cf5Eq/Onh0tgK6OoB2F91F+iEFL0ElyWQS+k/pz7/rt+es9OUp Uf6KLdMuaCNovD+TRlgp0jZB0sW+Qu7CJHpwQEf0V9T+fB0N7BPdsEoaskuJtMqRPqIZOj XLGRHA0W92sMc8abPVV8m53hldqBsnDA1Gmg6LfUSy2bSBT8PpWA7MOUvSee+XiG8o2ODY 1uuFH1Z95Spr9mTgn7ikEzHH4Eya2mCixs3gXwTErTGxKe+vcnyKg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=archlinux.org; s=dkim-ed25519; t=1653774975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=8qtR7L++KmXT+NZUcNqwhN1TBUqQDnlIvUiyoarExBQ=; b=wvNYLelPzSCcSwNVz4XWqXFQELEB7DsKZWUYeQGKGgGHL0sEHHoBCpJxjJJHrdbYkP78+/ CIBFDffjbwlRAVAw== To: pacman-dev@lists.archlinux.org Cc: Morten Linderud Subject: [PATCH v2] strip: Use dwz to compress dwarf headers Date: Sat, 28 May 2022 23:56:10 +0200 Message-Id: <20220528215610.2668804-1-foxboron@archlinux.org> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=pass header.d=archlinux.org header.s=dkim-rsa header.b=vURbAwP2; dkim=pass header.d=archlinux.org header.s=dkim-ed25519 header.b=wvNYLelP; spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 2a01:4f9:c010:9eb4::1 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org; dmarc=pass (policy=none) header.from=archlinux.org X-Rspamd-Server: mail.archlinux.org X-Spamd-Result: default: False [-2.61 / 15.00]; DWL_DNSWL_MED(-2.00)[archlinux.org:dkim]; MID_CONTAINS_FROM(1.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DMARC_POLICY_ALLOW(-0.50)[archlinux.org,none]; R_MISSING_CHARSET(0.50)[]; RCVD_IN_DNSWL_MED(-0.40)[2a01:4f9:c010:3052::1:received,2a01:4f9:c010:9eb4::1:from]; MAILLIST(-0.20)[mailman]; R_SPF_ALLOW(-0.20)[+ip6:2a01:4f9:c010:9eb4::1:c]; R_DKIM_ALLOW(-0.20)[archlinux.org:s=dkim-rsa,archlinux.org:s=dkim-ed25519]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[pacman-dev@lists.archlinux.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f9::/32, country:DE]; DKIM_TRACE(0.00)[archlinux.org:+]; RCVD_COUNT_THREE(0.00)[3]; FROM_NEQ_ENVFROM(0.00)[foxboron@archlinux.org,pacman-dev-bounces@lists.archlinux.org]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; NEURAL_HAM(-0.00)[-0.997]; FORGED_SENDER_MAILLIST(0.00)[] X-Rspamd-Queue-Id: A0CF6104B5C1 From: Morten Linderud `dwz` allows us to compress the DWARF files when building debug packages. Potentially shaving off some of the sizes of the symbols we distribute. https://sourceware.org/dwz/ A sample of packages built with dwz; pacman: Original debug info size: 1520kB Size after compression: 1252kB systemd: Original debug info size: 46692kB Size after compression: 41036kB Signed-off-by: Morten Linderud --- doc/makepkg.conf.5.asciidoc | 3 ++ scripts/libmakepkg/executable/dwz.sh.in | 38 ++++++++++++++++ .../executable/sepdebugcrcfix.sh.in | 38 ++++++++++++++++ scripts/libmakepkg/tidy/strip.sh.in | 44 +++++++++++++++++-- 4 files changed, 120 insertions(+), 3 deletions(-) create mode 100644 scripts/libmakepkg/executable/dwz.sh.in create mode 100644 scripts/libmakepkg/executable/sepdebugcrcfix.sh.in diff --git a/doc/makepkg.conf.5.asciidoc b/doc/makepkg.conf.5.asciidoc index a0d9a6d4..2e40f27f 100644 --- a/doc/makepkg.conf.5.asciidoc +++ b/doc/makepkg.conf.5.asciidoc @@ -194,6 +194,9 @@ Options DEBUG_CXXFLAGS to their counterpart buildflags. Creates a separate package containing the debug symbols when used with `strip'. + *dwz*;; + Compress dwarf files inside the debug package. + *lto*;; Enable building packages using link time optimization. Adds the flags specified in LTOFLAGS to CFLAGS, CXXFLAGS and LDFLAGS (or diff --git a/scripts/libmakepkg/executable/dwz.sh.in b/scripts/libmakepkg/executable/dwz.sh.in new file mode 100644 index 00000000..a540697c --- /dev/null +++ b/scripts/libmakepkg/executable/dwz.sh.in @@ -0,0 +1,38 @@ +#!/usr/bin/bash +# +# dwz.sh - Confirm presence of dwz binary +# +# Copyright (c) 2011-2022 Pacman Development Team +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +[[ -n "$LIBMAKEPKG_EXECUTABLE_DWZ_SH" ]] && return +LIBMAKEPKG_EXECUTABLE_DWZ_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/option.sh" + +executable_functions+=('executable_dwz') + +executable_dwz() { + if check_option "dwz" "y"; then + if ! type -p dwz>/dev/null; then + error "$(gettext "Cannot find the %s binary required to compress dwarf files.")" "dwz" + return 1 + fi + fi +} diff --git a/scripts/libmakepkg/executable/sepdebugcrcfix.sh.in b/scripts/libmakepkg/executable/sepdebugcrcfix.sh.in new file mode 100644 index 00000000..6259ac32 --- /dev/null +++ b/scripts/libmakepkg/executable/sepdebugcrcfix.sh.in @@ -0,0 +1,38 @@ +#!/usr/bin/bash +# +# sepdebugcrcfix.sh - Confirm presence of sepdebugcrcfix binary +# +# Copyright (c) 2011-2022 Pacman Development Team +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +[[ -n "$LIBMAKEPKG_EXECUTABLE_SEPDEBUGCRCFIX_SH" ]] && return +LIBMAKEPKG_EXECUTABLE_SEPDEBUGCRCFIX_SH=1 + +LIBRARY=${LIBRARY:-'@libmakepkgdir@'} + +source "$LIBRARY/util/message.sh" +source "$LIBRARY/util/option.sh" + +executable_functions+=('executable_sepdebugcrcfix') + +executable_sepdebugcrcfix() { + if check_option "dwz" "y"; then + if ! type -p sepdebugcrcfix>/dev/null; then + error "$(gettext "Cannot find the %s binary required to fix CRC.")" "sepdebugcrcfix" + return 1 + fi + fi +} diff --git a/scripts/libmakepkg/tidy/strip.sh.in b/scripts/libmakepkg/tidy/strip.sh.in index 688bcf1b..1365b77b 100644 --- a/scripts/libmakepkg/tidy/strip.sh.in +++ b/scripts/libmakepkg/tidy/strip.sh.in @@ -27,7 +27,7 @@ source "$LIBRARY/util/message.sh" source "$LIBRARY/util/option.sh" -packaging_options+=('strip' 'debug') +packaging_options+=('strip' 'debug' 'dwz') tidy_modify+=('tidy_strip') @@ -52,6 +52,37 @@ source_files() { | sort -zu | tr '\0' '\n' } +compress_dwarf_files(){ + binary_files="$@" + dwz_opts="" + readarray dwz_files < <(find "$dbgdir" -type f -name \*.debug | LC_ALL=C sort) + + # dwz multifile attempts to remove common symols found in all ELF files and + # throw them into one file. Replacing the references with .gnu_debugaltlink + local fullver=$(get_full_version) + dwz_multifile_name="${pkgbase}-${fullver}-${CARCH}" + if [ ${#dwz_files[@]} -gt 1 ]; then + dwz_opts="-m .dwz/${dwz_multifile_name}" + fi + mkdir -p "$dbgdir/.dwz" + + # The dwz options are taken from rpm/debugedit + (cd "$dbgdir" && + LANG=C dwz --hardlink \ + --quiet \ + --relative \ + --low-mem-die-limit 10000000 \ + --max-die-limit 50000000 \ + $dwz_opts \ + ${dwz_files[@]} 2> /dev/null) + + # Remove dir if empty + rmdir "$dbgdir/.dwz" 2> /dev/null + + # Some debuggers might use the CRC to find relevant debug files instead of the build-id + LANG=C sepdebugcrcfix "$dbgdir" $binary_files &>/dev/null +} + strip_file() { local binary=$1; shift @@ -142,7 +173,8 @@ tidy_strip() { fi local binary strip_flags - find . -type f -perm -u+w -print0 2>/dev/null | while IFS= read -rd '' binary ; do + declare -a binary_files + while IFS= read -rd '' binary ; do local STRIPLTO=0 case "$(LC_ALL=C readelf -h "$binary" 2>/dev/null)" in *Type:*'DYN (Shared object file)'*) # Libraries (.so) or Relocatable binaries @@ -166,6 +198,12 @@ tidy_strip() { esac strip_file "$binary" ${strip_flags} (( STRIPLTO )) && strip_lto "$binary" - done + binary_files+=("$binary") + done < <(find . -type f -perm -u+w -print0 2>/dev/null) + + if check_option "debug" "y" && check_option "dwz" "y"; then + msg2 "$(gettext "Compressing dwarf files...")" + compress_dwarf_files ${binary_files[@]} + fi fi }