From patchwork Thu Jan 2 17:31:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 1427 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id AF0A615FDE596 for ; Thu, 2 Jan 2020 17:31:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.3 (2019-12-06) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1, MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_NONE=-0.0001,SPF_HELO_NONE=0.001, T_DMARC_POLICY_NONE=0.01 autolearn=ham autolearn_force=no version=3.4.3 X-Spam-BL-Results: [127.0.9.0] Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Thu, 2 Jan 2020 17:31:42 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 73E7817BD03839; Thu, 2 Jan 2020 17:31:34 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: luna) by orion.archlinux.org (Postfix) with ESMTPSA id 199C417BD0381A; Thu, 2 Jan 2020 17:31:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=orion; t=1577986290; bh=8YtzIo9eJazKx3NYB/FPpGoKeCpjpC+Q1wAuvGU6zZQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc; b=MqvHwYgG1ROINEOd9B4l+n/WRnVQ+StLZxyDF02aVRd83ozSa5I0d5DmZVjDZwaGF 14KknzUvJ2IiO4C6qUCuAf37Id3gs5h7Bl8wQSRiGxVMoafM75J076W959O/DnwN4P 5bpCLeSc77gZueFLCQG3wayT0px77tYWzVSW4AQ5pD58Reb8fI2zwXkwal7vfc+TVX cnVbplpLWvVqbSmuNQtmtCdOhXq3/13czuaNtktULlgbBwQtzfidgVVXg3q8N/a6Y/ QretS4sBcTjTGcDUJ3CnUNJMvhKShUdo/dqauLDUjX9LHPdpcmXitojoEG+Bwzj4RG 0rnvBu3vBysk0Wma83/9EhFc/XTC8vUF5Eq64cIwpYsPVTRgt28ZYLhLh3OPzircI5 xAftrf+cWpNuiSRbJHsWfCZgrRPvZqXXMIlRbdHe2sGO8pfinb+DceqyNDXBuwrAbz /xPO8IlFaBDRCuCnIKAsh6bsGN0vaZ2XALSYkczR8rjUAW9g+glAEhJlLu0+eWQ3GP aL9+b/RYsz9wf8OB+PRAN3jn2nDFoOvbXC9PB9czYeqQGl79OJGMVrcjpf9xvLCNQE NWBnT3OozmyF0jqaQ208bUblrIyVbaJTwk/3MboOxmG3NOktBRsOzYSAlfbXm+THoI QhdqOytNIWJ5g+Uzrjipz3q4= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 0321E2BE61; Thu, 2 Jan 2020 17:31:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1577986290; bh=8YtzIo9eJazKx3NYB/FPpGoKeCpjpC+Q1wAuvGU6zZQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc; b=XnZGKzzpBiZhVabchkFuAdU+qVlF8d9xoSJR/BACGnVn13KNCoxUg+f8lqj2HVxsN 2b7Q0PNyb4o+k/tEkVt5QMuSX17vyNy5dpPT6lWOC/R0wMXBHTNHSLox8C/JIyTBsr b1BhMRDpG0PD6BZK5mKWxl5KcJ6N3L+Ezug427xHzg/D2qmKVQFocCaX8JSE6009Aw uWEdEeEAiZGfKCRx3uKL1yWcX0o3PNLP6tASCDwIvsXjwgcJ2I92nj1f0xKTZm2f1A LhEJst72btaLfodfIE3SINr4ENT8TCxcqeos7M2a2AHOoVCozW7PZeh9A3bQMIx7GA buFVlwhVmCOfl6Z5y1ijP4o2ArunXL+FsJYuXnGy3SFeywBrbETMAl+7xIauYKRBxw +dvXIjqaJqfnEiFOj2oq64ydoilctwxKoXP35jCSc4vEYp9ZlxWf77e/3fbyUXaq+l z+kz2BIi6jPDPmrchFpw1e3xaCmSC8/VEIRy/SmJIry6q7ngKay0uo7qe+5L7e4QnE haKZ3uFlH8h1NdqI5IH/7yrtf/LnswrYJbavbN1ujUB57r+vZqbHbWFFiQ3Yep98+W 8BuniBoOi3oWEkMu4M6RIUYI8iD7MNz9HKwmyFr3SWf7BDkPoKMPiFi2qKzla4x2y2 zHcVj/yA/jwhLMqSsGqIKGp8= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 629BF2BE5F for ; Thu, 2 Jan 2020 17:31:27 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Thu, 2 Jan 2020 17:31:27 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 3DFA317BD0380B; Thu, 2 Jan 2020 17:31:26 +0000 (UTC) Received: from didactylos.attlocal.net (unknown [IPv6:2600:1700:57f0:ca20:763a:c795:fcf6:91ea]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: eschwartz) by orion.archlinux.org (Postfix) with ESMTPSA id AC43317BD03809; Thu, 2 Jan 2020 17:31:25 +0000 (UTC) To: arch-projects@archlinux.org Date: Thu, 2 Jan 2020 12:31:13 -0500 Message-Id: <20200102173113.409799-2-eschwartz@archlinux.org> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200102173113.409799-1-eschwartz@archlinux.org> References: <20200102173113.409799-1-eschwartz@archlinux.org> MIME-Version: 1.0 Subject: [arch-projects] [devtools] [PATCH 2/2] makechrootpkg: use the chroot database to find checkpkg packages X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Eli Schwartz via arch-projects Reply-To: Arch Linux projects development discussion Cc: Eli Schwartz Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" We don't want to check against the current version known to the host system, because that will be incorrect in a wide variety of situations, including: - the build host hasn't done a full system upgrade yet - we're building against staging, and want to see the delta between different staging versions - we're building against extra, but the host runs testing which carries changes we don't want to visualize right now - the chroot has a configured database not available to the host, and the package is only available there Essentially, it's rarely 100% correct to run checkpkg on the host, but we already have a database we *know* is correct, and that is the one we just built the package against. So let's use that. This also fixes a bug in the current logic, where in order to try downloading fresh databases, we work in a non-cached temporary working database to download the package files, but then let checkpkg default to comparing packages in the system database. Since we are explicitly trying to compare against packages that differ from the host's pacman database, we need to pass the package files as options to checkpkg, using the additional modes added in commit c14338c0fe71a74f5e56b4f3af7c548fe0928e15 Signed-off-by: Eli Schwartz --- makechrootpkg.in | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/makechrootpkg.in b/makechrootpkg.in index dfea18c..bfa69be 100644 --- a/makechrootpkg.in +++ b/makechrootpkg.in @@ -366,11 +366,7 @@ if arch-nspawn "$copydir" \ "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" \ /chrootbuild "${makepkg_args[@]}" then - pkgnames=() - for pkgfile in "$copydir"/pkgdest/*; do - pkgfile=${pkgfile##*/}; - pkgnames+=("${pkgfile%-*-*-*}"); - done + mapfile -t pkgnames < <(sudo -u "$makepkg_user" bash -c 'source PKGBUILD; printf "%s\n" "${pkgname[@]}"') move_products else (( ret += 1 )) @@ -388,29 +384,29 @@ else if (( run_checkpkg )); then msg "Running checkpkg" - # sync off-site databases for up-to-date queries - trap 'rm -rf $dbpath; cleanup' EXIT INT TERM QUIT - dbpath=$(mktemp -d --tmpdir makechrootpkg-database.XXXXXXXXXX) - mkdir -p "$dbpath" - pacman -Sy --dbpath "$dbpath" --logfile /dev/null + mapfile -t remotepkgs < <(pacman --config "$copydir"/etc/pacman.conf \ + --dbpath "$copydir"/var/lib/pacman \ + -Sddp "${pkgnames[@]}") - # query current package locations - remotepkgs=($(pacman -Sddp --dbpath "$dbpath" --logfile /dev/null "${pkgnames[@]}")) - if (( $? )); then + if ! wait $!; then warning "Skipped checkpkg due to missing repo packages" exit 0 fi # download package files if any non-local location exists for remotepkg in "${remotepkgs[@]}"; do - [[ $remotepkg == file://* ]] && continue - msg2 "Downloading current versions" - pacman --noconfirm -Swdd --dbpath "$dbpath" --logfile /dev/null "${pkgnames[@]}" - break + if [[ $remotepkg != file://* ]]; then + msg2 "Downloading current versions" + arch-nspawn "$copydir" pacman --noconfirm -Swdd "${pkgnames[@]}" + mapfile -t remotepkgs < <(pacman --config "$copydir"/etc/pacman.conf \ + --dbpath "$copydir"/var/lib/pacman \ + -Sddp "${pkgnames[@]}") + break + fi done msg2 "Checking packages" - sudo -u "$makepkg_user" checkpkg --rmdir --warn + sudo -u "$makepkg_user" checkpkg --rmdir --warn "${remotepkgs[@]/#file:\/\//}" fi true fi