From patchwork Tue Jan 15 13:34:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 952 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id A9132A955617 for ; Tue, 15 Jan 2019 13:35:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on apollo X-Spam-Level: X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00=-1, DKIMWL_WL_HIGH=-1.046,DKIM_SIGNED=0.1,DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3 autolearn=ham autolearn_force=no version=3.4.2 X-Spam-BL-Results: [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Tue, 15 Jan 2019 13:35:33 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 7913AFFEB973A; Tue, 15 Jan 2019 13:35:32 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS; Tue, 15 Jan 2019 13:35:32 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 60A072CD53; Tue, 15 Jan 2019 13:35:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1547559332; bh=3LGSTt2wMmNalAnUimgOrKAUE62Y3njk+69m93C8MTw=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc; b=nX+PAswZlid6qxodt45dJpEFLe5WWZKGLYUkO01nDoEIQU2e/rdjyYXUMnWaH5/2D kYV/403FopsALeLC2Rp6Yl8hV0wtvPlOPWITAaPGAJzlNheGjJVkzCwEQFCJz/obGF WKIM/BIxBmYXCl7jUvcqkfkeFi9jGiWK3yU5JpU0MxF9cbVRucWRjSneUaKSX5n0We SrUeI/761ja3pFdKvGJPuCCdYQwMQikbnzGCQE3z1xNl6VCcizqrfqrZv3uKuEQLVq XwewodgukXhfIGyUI7bmaX74sSZTL2EhaNkg4qzrFsjvm3JvanoHz2TrAwu5UYD3gz 8UxoZLryeDpWlN4tgaEZhb4e+o0IPHMKMJDdjzhuQy0tplNSZDU9LGzrjKQTmmwj6R mzxx7M37ihFbZqucPl5eMScKtsJxAbQ7W+oKVzInYyEitDZ04yPalsdAjjbXPyHsdJ 4msswgE19ItkFUZzsAxG0yFEDxbOlhy3xzt7/8U4IUj2s3OjyGsgm9Vr0Tl8M8hGRz gx86zGpANA3IURsZE2AtRXBO3M9zuUO8uegcekBpBPuu8LXQPQd0LZXbc31HAo8A/h X62cW9De6oeGEuDD8IS/lnytCiNZf9fP0BHLMoZbTqMC76ujNQVlmZ7cLX3/FjhGbq Pf4MpVnAPdro8FUBsoP7BYVY= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 4A5692BF97 for ; Tue, 15 Jan 2019 13:35:28 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 15 Jan 2019 13:35:28 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 16FBEFFEB9734 for ; Tue, 15 Jan 2019 13:35:27 +0000 (UTC) Received: from mx1.de-vri.es (mx1.de-vri.es [IPv6:2a01:7c8:aab4:33e::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS for ; Tue, 15 Jan 2019 13:35:27 +0000 (UTC) Received: from voyager (localhost [127.0.0.1]) by voyager (OpenSMTPD) with ESMTP id 12bf8609; Tue, 15 Jan 2019 13:35:25 +0000 (UTC) Received: from gourmetset.internal.delftrobotics.com (217.67.224.105 [217.67.224.105]) by voyager (OpenSMTPD) with ESMTPSA id 4ffc3efb (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Tue, 15 Jan 2019 13:35:25 +0000 (UTC) To: arch-projects@archlinux.org Date: Tue, 15 Jan 2019 14:34:51 +0100 Message-Id: <20190115133450.9676-1-maarten@de-vri.es> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [arch-projects] [devtools] [PATCH 1/2] arch-nspawn, mkarchroot: Allow not sharing the cache directories. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Maarten de Vries via arch-projects Reply-To: Arch Linux projects development discussion Cc: Maarten de Vries Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" These patches make it possible to build withouth a shared pacman cache using makechrootpkg. I need this myself because I'm building packages for different repositories where some of them contain packages with the same name, but different compile time configuration. It also allows building packages for different Arch based distros on the same host more easily. The first patch changes arch-nspawn and mkarchroot to accept `-c -` to disable bind mounting the host pacman cache in the container. The container will simply use whatever is configured in the container instead. The second patch allows the user set the pacman cache for makechrootpkg. Sadly -c was already taken, so here it's -C. The end result is that a user can run the following to build without a shared pacman cache: `makechrootpkg -C - -r $chroot` I considered a more generic -N ... to pass arguments to arch-nspawn, but arch-nspawn doesn't have that many options and it would get quite ugly: makechrootpkg -N -c -N - -r $chroot Signed-off-by: Maarten de Vries --- arch-nspawn.in | 12 +++++++++--- mkarchroot.in | 4 +++- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/arch-nspawn.in b/arch-nspawn.in index b9c846e..c6cc3a4 100644 --- a/arch-nspawn.in +++ b/arch-nspawn.in @@ -52,7 +52,9 @@ shift 1 [[ -z $working_dir ]] && die 'Please specify a working directory.' -if [[ -z $cache_dir ]]; then +if [[ $cache_dir = '-' ]]; then + cache_dirs=() +elif [[ -z $cache_dir ]]; then cache_dirs=($(pacman -v 2>&1 | grep '^Cache Dirs:' | sed 's/Cache Dirs:\s*//g')) else cache_dirs=("$cache_dir") @@ -72,7 +74,9 @@ build_mount_args() { mount_args+=("--bind-ro=$host_mirror_path") fi - mount_args+=("--bind=${cache_dirs[0]}") + if [[ ${#cache_dirs[@]} -ge 1 ]]; then + mount_args+=("--bind=${cache_dirs[0]}") + fi for cache_dir in "${cache_dirs[@]:1}"; do mount_args+=("--bind-ro=$cache_dir") @@ -92,7 +96,9 @@ copy_hostconf () { cp -T "$file" "$working_dir$file" done - sed -r "s|^#?\\s*CacheDir.+|CacheDir = ${cache_dirs[*]}|g" -i "$working_dir/etc/pacman.conf" + if [[ ${#cache_dirs[@]} -ge 1 ]]; then + sed -r "s|^#?\\s*CacheDir.+|CacheDir = ${cache_dirs[*]}|g" -i "$working_dir/etc/pacman.conf" + fi } # }}} diff --git a/mkarchroot.in b/mkarchroot.in index 52e363f..81bc58a 100644 --- a/mkarchroot.in +++ b/mkarchroot.in @@ -51,7 +51,9 @@ shift 1 [[ -z $working_dir ]] && die 'Please specify a working directory.' -if [[ -z $cache_dir ]]; then +if [[ $cache_dir = '-' ]]; then + cache_dirs=() +elif [[ -z $cache_dir ]]; then cache_dirs=($(pacman -v "$cache_conf" 2>&1 | grep '^Cache Dirs:' | sed 's/Cache Dirs:\s*//g')) else cache_dirs=(${cache_dir}) From patchwork Tue Jan 15 13:34:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 953 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id 3B00DA955631 for ; Tue, 15 Jan 2019 13:35:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on apollo X-Spam-Level: X-Spam-Status: No, score=-3.8 required=5.0 tests=BAYES_00=-1, DKIMWL_WL_HIGH=-1.046,DKIM_SIGNED=0.1,DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_LOW=-0.7 autolearn=ham autolearn_force=no version=3.4.2 X-Spam-BL-Results: [127.0.9.1] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Tue, 15 Jan 2019 13:35:40 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id B1682FFEB9751; Tue, 15 Jan 2019 13:35:38 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS; Tue, 15 Jan 2019 13:35:38 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 96CA22CDE7; Tue, 15 Jan 2019 13:35:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1547559338; bh=D4ilBUO3zq08/vlgtFojxiC7BTPMnlE0Cb9DxfJ1ZwU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc; b=otgfibUSBQsiQX/FVUxHPLNxMmKnZqP3gi/iSK4cLC9lyyNtvRaQEWt9A18Mhdlh3 Ep8aKV84ddjrwdJxe2uGvg4goEJiprG86XFpMlaz56lsw3lvLqQYLTQ3xatz/CLuK7 7XJ9Vicls8ui0kK3r8p3iBZnr5xJacgvAFm5VM1xiXNLuwkTOgdaUtg6QNdbS2LdSF hOMNYQzCoqRrfurO4PcIN1C8F1qz4EUUNdSe+u9910VVWnEmOodz3SIjAWQW5sfmzY It9uEpWXxx3AghU+uNLpzkyyBuyD1rFPsim9q1uHJN8+KXS3u+tvX8ZlQNHNQx+mQc eBKF3+NvbJ8EdicniOpQWrR0l4IF9gybc5GDjlNJxPbmd7DUPADdz/kWoUWf7ciA3o awnFoQLkwBMaUjdlKuX39pmKI9NCRkSnsO7ysa/hp24c/XSvLFZr6C25etBG5x14Aw j4ptti7vQY9bqC293h3zfZjLOk1fuICSzb6tXMsQzQZaTDc+/sSn4Id2gYFL814Aav TYST0QWb4pCzQNMFLuifGw0MzfiSSCwugp2JeCm+g4uph1eViHIp6KBFo3bXZ5h7oV KEOfMp4r9fUl0NmvNQpQGXeoQR/ALK0HHWmMf2X3PeeSB2m6xr0Cnh/7GjJIDwI8rG m3JfV8NAHZ/2Sml0isdcSTQY= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id CEB0C2CDE7 for ; Tue, 15 Jan 2019 13:35:36 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 15 Jan 2019 13:35:36 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 8127FFFEB974C for ; Tue, 15 Jan 2019 13:35:35 +0000 (UTC) Received: from mx1.de-vri.es (mx1.de-vri.es [IPv6:2a01:7c8:aab4:33e::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS for ; Tue, 15 Jan 2019 13:35:35 +0000 (UTC) Received: from voyager (localhost [127.0.0.1]) by voyager (OpenSMTPD) with ESMTP id 945e1418; Tue, 15 Jan 2019 13:35:34 +0000 (UTC) Received: from gourmetset.internal.delftrobotics.com (217.67.224.105 [217.67.224.105]) by voyager (OpenSMTPD) with ESMTPSA id 82264210 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Tue, 15 Jan 2019 13:35:34 +0000 (UTC) To: arch-projects@archlinux.org Date: Tue, 15 Jan 2019 14:34:53 +0100 Message-Id: <20190115133450.9676-2-maarten@de-vri.es> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190115133450.9676-1-maarten@de-vri.es> References: <20190115133450.9676-1-maarten@de-vri.es> MIME-Version: 1.0 Subject: [arch-projects] [devtools] [PATCH 2/2] makechrootpkg: Allow setting pacman cache for arch-nspawn. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Maarten de Vries via arch-projects Reply-To: Arch Linux projects development discussion Cc: Maarten de Vries Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" Signed-off-by: Maarten de Vries --- makechrootpkg.in | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/makechrootpkg.in b/makechrootpkg.in index d81be84..ad65de4 100644 --- a/makechrootpkg.in +++ b/makechrootpkg.in @@ -38,6 +38,7 @@ usage() { echo 'Flags:' echo '-h This help' echo '-c Clean the chroot before building' + echo '-C Use a folder on the host as pacman cache' echo '-d Bind directory into build chroot as read-write' echo '-D Bind directory into build chroot as read-only' echo '-u Update the working copy of the chroot before building' @@ -147,7 +148,7 @@ install_packages() { pkgnames=("${install_pkgs[@]##*/}") cp -- "${install_pkgs[@]}" "$copydir/root/" - arch-nspawn "$copydir" "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" \ + nspawn "$copydir" \ pacman -U --noconfirm -- "${pkgnames[@]/#//root/}" ret=$? rm -- "${pkgnames[@]/#/$copydir/root/}" @@ -301,6 +302,22 @@ move_products() { fi done } + +# Usage: nspawn $copydir $nspawn-args ... +# Globals: +# - pacman_cache +# - bindmounts_ro +# - bindmounts_rw +nspawn() { + local copydir=$1 + shift + + if [[ -n $pacman_cache ]]; then + arch-nspawn -c "$pacman_cache" "$copydir" "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" "$@" + else + arch-nspawn "$copydir" "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" "$@" + fi +} # }}} main() { @@ -317,6 +334,7 @@ main() { declare -a install_pkgs declare -i ret=0 + pacman_cache= bindmounts_ro=() bindmounts_rw=() @@ -325,9 +343,10 @@ main() { [[ -z "$copy" || $copy = root ]] && copy=copy src_owner=${SUDO_USER:-$USER} - while getopts 'hcur:I:l:nTD:d:U:' arg; do + while getopts 'hcur:I:l:nTC:D:d:U:' arg; do case "$arg" in c) clean_first=true ;; + C) pacman_cache="$OPTARG" ;; D) bindmounts_ro+=("--bind-ro=$OPTARG") ;; d) bindmounts_rw+=("--bind=$OPTARG") ;; u) update_first=true ;; @@ -395,8 +414,7 @@ main() { sync_chroot "$chrootdir/root" "$copydir" "$copy" fi - $update_first && arch-nspawn "$copydir" \ - "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" \ + $update_first && nspawn "$copydir" \ pacman -Syu --noconfirm if [[ -n ${install_pkgs[*]:-} ]]; then @@ -415,10 +433,9 @@ main() { prepare_chroot "$copydir" "$USER_HOME" "$keepbuilddir" "$run_namcap" - if arch-nspawn "$copydir" \ + if nspawn "$copydir" \ --bind="$PWD:/startdir" \ --bind="$SRCDEST:/srcdest" \ - "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" \ /chrootbuild "${makepkg_args[@]}" then move_products "$copydir" "$src_owner"