From patchwork Thu Jan 2 17:31:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 1426 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id 2DCFD15FDE58E for ; Thu, 2 Jan 2020 17:31:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.3 (2019-12-06) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1, MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,SPF_HELO_NONE=0.001, T_DMARC_POLICY_NONE=0.01 autolearn=ham autolearn_force=no version=3.4.3 X-Spam-BL-Results: [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Thu, 2 Jan 2020 17:31:41 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 197D017BD03815; Thu, 2 Jan 2020 17:31:30 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: luna) by orion.archlinux.org (Postfix) with ESMTPSA id C958817BD03810; Thu, 2 Jan 2020 17:31:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=orion; t=1577986289; bh=S11Qed/lm/mKzcyC/qsiplaatu+D3hzX5KMGt1ooOHo=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc; b=O97kj6PUQcWaBm/9xVREm1TcvsSZmW2KITvOHovqB1p90PlxoZCtfLU9Z1wCyo5Yc OaYqkGqU+bU0tDeo+NdJHkRRHEJ6XhhkVEf419ZtJ142kB6Jtr/7lXVodiT05j1ezZ 98em97RV0ejvYL9RLJyBOuiQ2FuJwR69UgvOMm/YM6RXRtL3/3hfAqeB/HzhmogbMJ MyPE6U0O5/GdaScQ7btB0eDWBjo7AXWslRqWWNxYWptEXL/lXelirrvmdijRwixK5/ zoJC1Y71t32w6lD3bpujEowWomTOtwTYxq7KrLK2zaZwuwpB/IK3pM8ruDRaP5ezBz XYvmIa6dUpXlNC2mxGfYrkBo8tsoVH6gFsEdXDlC2legcqyNVZL37bFENxWlZTShNo H5qA9GOXn4QYcA1HPuD0+6N94h8X9xaCQIg3dIta/5m/32ClCfE3U5JOsJjbwKF0bn 01CgN7YOsnFBU3c3gQKRaYVz55wBHRYEZVcP+7N7V6qrC5Up+5wMEYJ6n7dAwkoF5c /ZVfC0CQKvv701v1Ad/iSpzYCYsRdilh6oPZpW4mViaX8HPXF17fHh6+DwolTkWXOb MkvF2v42dMluQMC+GGSzQwKMDv9TtPA8kkm5HgSO2m0ftzyy95EgYupFrQG6SUFsP6 0JUXWzmOSwxNeknAC8e1FFQE= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id B91822BE5F; Thu, 2 Jan 2020 17:31:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1577986289; bh=S11Qed/lm/mKzcyC/qsiplaatu+D3hzX5KMGt1ooOHo=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc; b=bKIRKzt1k+4bSaWhlTa2kqzrRxiSnrzonYHeiuSZp4BkXFmvm/66teCmNRCowhznl PMpQlxtd7BzCayvL0MyLscWF3jvab66dnNqlkn4tGboEHgEitKd43RMBOV0MoX3dks ztlZpsRfeefNJtp4H+txuAwdkM69RxHcLbW+Efm/s/s4rUGNgxi2PIsNjZ5U1S5O6Y xUEkYu0ayL3zUWJzEyRIpS4aw39uiGdeBF9v0he/S0mp/Q6oaBRFqQlpN+AtNnocyd ECy43QclbXa/IjyZ7iWXnGjBtD++8OkNQjcUyOmzbL2G7colh+kTyg0so+VR3YWgs9 ydmC54d8AEbgh2ZvY2Dr0tTl6VQhhZdLFr2upP9tUodwwXibY4Nbc0s2weH4BD9Sxm qRDRhNVqtu8boFpGTNCq4OeKjZJOYvt82SufGRPG08UlapQM3Hv8R6AqD9FowlcvPp eNpw/L1OZ+xCuVLcfQAx7QEa+ggS/PGZ1twyuxRCQQVTwUyohR9Fl8B4gk8xXNk0LE 55BovOPtDekBsVXwn/5JmyFycEgNmKMKUaO+amHV8CJx42HuHNMA52HsI+Yr2SSuS9 6MYj7G7DSOCwmrL4uKH6b02630wDig4R/ih1eBT/PqbQTcLTpyhz/Xjs+w5OAVBDXc JgAGCfqOob6g8euV/0IU/IZ8= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 84FFA2BE5A for ; Thu, 2 Jan 2020 17:31:26 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by luna.archlinux.org (Postfix) with ESMTPS for ; Thu, 2 Jan 2020 17:31:26 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 7F60E17BD03808; Thu, 2 Jan 2020 17:31:25 +0000 (UTC) Received: from didactylos.attlocal.net (unknown [IPv6:2600:1700:57f0:ca20:763a:c795:fcf6:91ea]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: eschwartz) by orion.archlinux.org (Postfix) with ESMTPSA id E4A9017BD03807; Thu, 2 Jan 2020 17:31:24 +0000 (UTC) To: arch-projects@archlinux.org Date: Thu, 2 Jan 2020 12:31:12 -0500 Message-Id: <20200102173113.409799-1-eschwartz@archlinux.org> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Subject: [arch-projects] [devtools] [PATCH 1/2] checkpkg: fix support for http:// url comparisons X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Eli Schwartz via arch-projects Reply-To: Arch Linux projects development discussion Cc: Eli Schwartz Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" Broken in commit c14338c0fe71a74f5e56b4f3af7c548fe0928e15. Signed-off-by: Eli Schwartz --- checkpkg.in | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/checkpkg.in b/checkpkg.in index 374c004..d7c0948 100644 --- a/checkpkg.in +++ b/checkpkg.in @@ -95,8 +95,10 @@ for _pkgname in "${pkgname[@]}"; do if (( $# )); then case $1 in + *://*) + pkgurl=$1 ;; /*|*/*) - pkgurl=file://$(readlink -m "$1") ;; + pkgurl=$(readlink -m "$1") ;; *.pkg.tar*) pkgurl=$1 ;; '') From patchwork Thu Jan 2 17:31:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 1427 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id AF0A615FDE596 for ; Thu, 2 Jan 2020 17:31:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.3 (2019-12-06) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1, MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_NONE=-0.0001,SPF_HELO_NONE=0.001, T_DMARC_POLICY_NONE=0.01 autolearn=ham autolearn_force=no version=3.4.3 X-Spam-BL-Results: [127.0.9.0] Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Thu, 2 Jan 2020 17:31:42 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 73E7817BD03839; Thu, 2 Jan 2020 17:31:34 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: luna) by orion.archlinux.org (Postfix) with ESMTPSA id 199C417BD0381A; Thu, 2 Jan 2020 17:31:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=orion; t=1577986290; bh=8YtzIo9eJazKx3NYB/FPpGoKeCpjpC+Q1wAuvGU6zZQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc; b=MqvHwYgG1ROINEOd9B4l+n/WRnVQ+StLZxyDF02aVRd83ozSa5I0d5DmZVjDZwaGF 14KknzUvJ2IiO4C6qUCuAf37Id3gs5h7Bl8wQSRiGxVMoafM75J076W959O/DnwN4P 5bpCLeSc77gZueFLCQG3wayT0px77tYWzVSW4AQ5pD58Reb8fI2zwXkwal7vfc+TVX cnVbplpLWvVqbSmuNQtmtCdOhXq3/13czuaNtktULlgbBwQtzfidgVVXg3q8N/a6Y/ QretS4sBcTjTGcDUJ3CnUNJMvhKShUdo/dqauLDUjX9LHPdpcmXitojoEG+Bwzj4RG 0rnvBu3vBysk0Wma83/9EhFc/XTC8vUF5Eq64cIwpYsPVTRgt28ZYLhLh3OPzircI5 xAftrf+cWpNuiSRbJHsWfCZgrRPvZqXXMIlRbdHe2sGO8pfinb+DceqyNDXBuwrAbz /xPO8IlFaBDRCuCnIKAsh6bsGN0vaZ2XALSYkczR8rjUAW9g+glAEhJlLu0+eWQ3GP aL9+b/RYsz9wf8OB+PRAN3jn2nDFoOvbXC9PB9czYeqQGl79OJGMVrcjpf9xvLCNQE NWBnT3OozmyF0jqaQ208bUblrIyVbaJTwk/3MboOxmG3NOktBRsOzYSAlfbXm+THoI QhdqOytNIWJ5g+Uzrjipz3q4= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 0321E2BE61; Thu, 2 Jan 2020 17:31:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1577986290; bh=8YtzIo9eJazKx3NYB/FPpGoKeCpjpC+Q1wAuvGU6zZQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc; b=XnZGKzzpBiZhVabchkFuAdU+qVlF8d9xoSJR/BACGnVn13KNCoxUg+f8lqj2HVxsN 2b7Q0PNyb4o+k/tEkVt5QMuSX17vyNy5dpPT6lWOC/R0wMXBHTNHSLox8C/JIyTBsr b1BhMRDpG0PD6BZK5mKWxl5KcJ6N3L+Ezug427xHzg/D2qmKVQFocCaX8JSE6009Aw uWEdEeEAiZGfKCRx3uKL1yWcX0o3PNLP6tASCDwIvsXjwgcJ2I92nj1f0xKTZm2f1A LhEJst72btaLfodfIE3SINr4ENT8TCxcqeos7M2a2AHOoVCozW7PZeh9A3bQMIx7GA buFVlwhVmCOfl6Z5y1ijP4o2ArunXL+FsJYuXnGy3SFeywBrbETMAl+7xIauYKRBxw +dvXIjqaJqfnEiFOj2oq64ydoilctwxKoXP35jCSc4vEYp9ZlxWf77e/3fbyUXaq+l z+kz2BIi6jPDPmrchFpw1e3xaCmSC8/VEIRy/SmJIry6q7ngKay0uo7qe+5L7e4QnE haKZ3uFlH8h1NdqI5IH/7yrtf/LnswrYJbavbN1ujUB57r+vZqbHbWFFiQ3Yep98+W 8BuniBoOi3oWEkMu4M6RIUYI8iD7MNz9HKwmyFr3SWf7BDkPoKMPiFi2qKzla4x2y2 zHcVj/yA/jwhLMqSsGqIKGp8= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 629BF2BE5F for ; Thu, 2 Jan 2020 17:31:27 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Thu, 2 Jan 2020 17:31:27 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 3DFA317BD0380B; Thu, 2 Jan 2020 17:31:26 +0000 (UTC) Received: from didactylos.attlocal.net (unknown [IPv6:2600:1700:57f0:ca20:763a:c795:fcf6:91ea]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: eschwartz) by orion.archlinux.org (Postfix) with ESMTPSA id AC43317BD03809; Thu, 2 Jan 2020 17:31:25 +0000 (UTC) To: arch-projects@archlinux.org Date: Thu, 2 Jan 2020 12:31:13 -0500 Message-Id: <20200102173113.409799-2-eschwartz@archlinux.org> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200102173113.409799-1-eschwartz@archlinux.org> References: <20200102173113.409799-1-eschwartz@archlinux.org> MIME-Version: 1.0 Subject: [arch-projects] [devtools] [PATCH 2/2] makechrootpkg: use the chroot database to find checkpkg packages X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Eli Schwartz via arch-projects Reply-To: Arch Linux projects development discussion Cc: Eli Schwartz Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" We don't want to check against the current version known to the host system, because that will be incorrect in a wide variety of situations, including: - the build host hasn't done a full system upgrade yet - we're building against staging, and want to see the delta between different staging versions - we're building against extra, but the host runs testing which carries changes we don't want to visualize right now - the chroot has a configured database not available to the host, and the package is only available there Essentially, it's rarely 100% correct to run checkpkg on the host, but we already have a database we *know* is correct, and that is the one we just built the package against. So let's use that. This also fixes a bug in the current logic, where in order to try downloading fresh databases, we work in a non-cached temporary working database to download the package files, but then let checkpkg default to comparing packages in the system database. Since we are explicitly trying to compare against packages that differ from the host's pacman database, we need to pass the package files as options to checkpkg, using the additional modes added in commit c14338c0fe71a74f5e56b4f3af7c548fe0928e15 Signed-off-by: Eli Schwartz --- makechrootpkg.in | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/makechrootpkg.in b/makechrootpkg.in index dfea18c..bfa69be 100644 --- a/makechrootpkg.in +++ b/makechrootpkg.in @@ -366,11 +366,7 @@ if arch-nspawn "$copydir" \ "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" \ /chrootbuild "${makepkg_args[@]}" then - pkgnames=() - for pkgfile in "$copydir"/pkgdest/*; do - pkgfile=${pkgfile##*/}; - pkgnames+=("${pkgfile%-*-*-*}"); - done + mapfile -t pkgnames < <(sudo -u "$makepkg_user" bash -c 'source PKGBUILD; printf "%s\n" "${pkgname[@]}"') move_products else (( ret += 1 )) @@ -388,29 +384,29 @@ else if (( run_checkpkg )); then msg "Running checkpkg" - # sync off-site databases for up-to-date queries - trap 'rm -rf $dbpath; cleanup' EXIT INT TERM QUIT - dbpath=$(mktemp -d --tmpdir makechrootpkg-database.XXXXXXXXXX) - mkdir -p "$dbpath" - pacman -Sy --dbpath "$dbpath" --logfile /dev/null + mapfile -t remotepkgs < <(pacman --config "$copydir"/etc/pacman.conf \ + --dbpath "$copydir"/var/lib/pacman \ + -Sddp "${pkgnames[@]}") - # query current package locations - remotepkgs=($(pacman -Sddp --dbpath "$dbpath" --logfile /dev/null "${pkgnames[@]}")) - if (( $? )); then + if ! wait $!; then warning "Skipped checkpkg due to missing repo packages" exit 0 fi # download package files if any non-local location exists for remotepkg in "${remotepkgs[@]}"; do - [[ $remotepkg == file://* ]] && continue - msg2 "Downloading current versions" - pacman --noconfirm -Swdd --dbpath "$dbpath" --logfile /dev/null "${pkgnames[@]}" - break + if [[ $remotepkg != file://* ]]; then + msg2 "Downloading current versions" + arch-nspawn "$copydir" pacman --noconfirm -Swdd "${pkgnames[@]}" + mapfile -t remotepkgs < <(pacman --config "$copydir"/etc/pacman.conf \ + --dbpath "$copydir"/var/lib/pacman \ + -Sddp "${pkgnames[@]}") + break + fi done msg2 "Checking packages" - sudo -u "$makepkg_user" checkpkg --rmdir --warn + sudo -u "$makepkg_user" checkpkg --rmdir --warn "${remotepkgs[@]/#file:\/\//}" fi true fi