From patchwork Wed Apr 5 19:35:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 89 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id YJQsMThH5VieagAAtiB/HQ for ; Wed, 05 Apr 2017 21:36:24 +0200 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id EDBFE40660; Wed, 5 Apr 2017 21:36:22 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Wed, 5 Apr 2017 21:36:22 +0200 (CEST) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id AB1F72C0F1; Wed, 5 Apr 2017 19:36:22 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id D25AD2C0E6 for ; Wed, 5 Apr 2017 19:36:20 +0000 (UTC) Received: from mav.lukeshu.com (mav.lukeshu.com [104.207.138.63]) by luna.archlinux.org (Postfix) with ESMTPS for ; Wed, 5 Apr 2017 19:36:20 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 2727083EB9; Wed, 5 Apr 2017 15:36:18 -0400 (EDT) From: lukeshu@lukeshu.com To: arch-projects@lists.archlinux.org Date: Wed, 5 Apr 2017 15:35:57 -0400 Message-Id: <20170405193603.22277-2-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.12.1 In-Reply-To: <20170405193603.22277-1-lukeshu@lukeshu.com> References: <20170405193603.22277-1-lukeshu@lukeshu.com> Subject: [arch-projects] [devtools] [PATCH 1/7] Handle makepkg.conf more consistently X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 236 Status: X-Keywords: Content-Length: 1975 From: Luke Shumaker - checkpkg: Check ~/.config/pacman/makepkg.conf ; not just ~/.makepkg.conf - commitpkg: Likewise - rebuildpkgs: Check for user makepkg.conf files, not just the system one in /etc --- checkpkg.in | 6 ++++-- commitpkg.in | 6 ++++-- rebuildpkgs.in | 7 +++++++ 3 files changed, 15 insertions(+), 4 deletions(-) diff --git a/checkpkg.in b/checkpkg.in index fbd30d3..ec58ff6 100644 --- a/checkpkg.in +++ b/checkpkg.in @@ -13,8 +13,10 @@ else fi # Source user-specific makepkg.conf overrides -if [[ -r ~/.makepkg.conf ]]; then - source ~/.makepkg.conf +if [[ -r "${XDG_CONFIG_HOME:-$HOME/.config}/pacman/makepkg.conf" ]]; then + source "${XDG_CONFIG_HOME:-$HOME/.config}/pacman/makepkg.conf" +elif [[ -r "$HOME/.makepkg.conf" ]]; then + source "$HOME/.makepkg.conf" fi if [[ ! -f PKGBUILD ]]; then diff --git a/commitpkg.in b/commitpkg.in index d31f6ba..1f9492c 100644 --- a/commitpkg.in +++ b/commitpkg.in @@ -11,8 +11,10 @@ else fi # Source user-specific makepkg.conf overrides -if [[ -r ~/.makepkg.conf ]]; then - . ~/.makepkg.conf +if [[ -r "${XDG_CONFIG_HOME:-$HOME/.config}/pacman/makepkg.conf" ]]; then + source "${XDG_CONFIG_HOME:-$HOME/.config}/pacman/makepkg.conf" +elif [[ -r "$HOME/.makepkg.conf" ]]; then + source "$HOME/.makepkg.conf" fi cmd=${0##*/} diff --git a/rebuildpkgs.in b/rebuildpkgs.in index 8a141e0..9197231 100644 --- a/rebuildpkgs.in +++ b/rebuildpkgs.in @@ -26,6 +26,13 @@ else die '/etc/makepkg.conf not found!' fi +# Source user-specific makepkg.conf overrides +if [[ -r "${XDG_CONFIG_HOME:-$HOME/.config}/pacman/makepkg.conf" ]]; then + source "${XDG_CONFIG_HOME:-$HOME/.config}/pacman/makepkg.conf" +elif [[ -r "$HOME/.makepkg.conf" ]]; then + source "$HOME/.makepkg.conf" +fi + bump_pkgrel() { # Get the current pkgrel from SVN and update the working copy with it # This prevents us from incrementing out of control :) From patchwork Wed Apr 5 19:35:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 90 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id efTBFzpH5ViTagAAtiB/HQ for ; Wed, 05 Apr 2017 21:36:26 +0200 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id CF00E406FB; Wed, 5 Apr 2017 21:36:24 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Wed, 5 Apr 2017 21:36:24 +0200 (CEST) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 06C9F2C0F5; Wed, 5 Apr 2017 19:36:24 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 408FA2C0EA for ; Wed, 5 Apr 2017 19:36:22 +0000 (UTC) Received: from mav.lukeshu.com (team4272.com [IPv6:2001:19f0:5c00:8069:5400:ff:fe26:6a86]) by luna.archlinux.org (Postfix) with ESMTPS for ; Wed, 5 Apr 2017 19:36:22 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id E3F1C83EB8; Wed, 5 Apr 2017 15:36:20 -0400 (EDT) From: lukeshu@lukeshu.com To: arch-projects@lists.archlinux.org Date: Wed, 5 Apr 2017 15:35:58 -0400 Message-Id: <20170405193603.22277-3-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.12.1 In-Reply-To: <20170405193603.22277-1-lukeshu@lukeshu.com> References: <20170405193603.22277-1-lukeshu@lukeshu.com> Subject: [arch-projects] [devtools] [PATCH 2/7] makechrootpkg: Have functions be more function-y. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 237 Status: X-Keywords: Content-Length: 5527 From: Luke Shumaker Rather than them simply being named blocks of code with braces around them. That is: have them take things via arguments rather than global variables. Specific notes: - create_chroot->sync_chroot: I pulled out locking the destination chroot; getting that lock is now the caller's responsibility. It still handles locking the source chroot though. I pulled the `if [[ ! -d $copydir ]] || $clean_first;` check out; it is now the caller's responsibility to use that check when deciding if to call sync_chroot. However, when pulling that check out, I left it as `if true;`, to keep an indentation level. This patch has had to be rebased/merged many times, and changing the indentation is a sure way to make that go less smoothly; I'm not going to re-indent this block until I see the check removed in the git.archlinux.org/devtools.git repository. - install_packages: 1. Receive the list of packages as arguments, rather than a global variable. 2. Make the caller responsible for looking at PKGBUILD. From the name and arguments, one would never expect it to look at PKGBUILD. --- makechrootpkg.in | 88 +++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 75 insertions(+), 13 deletions(-) diff --git a/makechrootpkg.in b/makechrootpkg.in index 5329f76..bdcaa88 100644 --- a/makechrootpkg.in +++ b/makechrootpkg.in @@ -76,6 +76,14 @@ usage() { } # {{{ functions +# Usage: load_vars $makepkg_conf +# Globals: +# - SRCDEST +# - SRCPKGDEST +# - PKGDEST +# - LOGDEST +# - MAKEFLAGS +# - PACKAGER load_vars() { local makepkg_conf="$1" var @@ -88,11 +96,23 @@ load_vars() { return 0 } -create_chroot() { - # Lock the chroot we want to use. We'll keep this lock until we exit. - lock 9 "$copydir.lock" "Locking chroot copy [%s]" "$copy" +# Usage: sync_chroot $CHROOTDIR/$CHROOT <$CHROOTCOPY|$copydir> +sync_chroot() { + local chrootdir=$1 + local copy=$2 + local copydir='' + if [[ ${copy:0:1} = / ]]; then + copydir=$copy + else + copydir="$chrootdir/$copy" + fi + + if [[ "$chrootdir/root" -ef "$copydir" ]]; then + error 'Cannot sync copy with itself: %s' "$copydir" + return 1 + fi - if [[ ! -d $copydir ]] || $clean_first; then + if true; then # indent for rebasing/merging purposes # Get a read lock on the root chroot to make # sure we don't clone a half-updated chroot slock 8 "$chrootdir/root.lock" "Locking clean chroot" @@ -117,7 +137,11 @@ create_chroot() { touch "$copydir" } -clean_temporary() { +# Usage: delete_chroot $copydir [$copy] +delete_chroot() { + local copydir=$1 + local copy=${1:-$2} + stat_busy "Removing temporary copy [%s]" "$copy" if is_btrfs "$chrootdir" && ! mountpoint -q "$copydir"; then btrfs subvolume delete "$copydir" >/dev/null || @@ -133,7 +157,11 @@ clean_temporary() { stat_done } +# Usage: install_packages $copydir $pkgs... install_packages() { + local copydir=$1 + local install_pkgs=("${@:2}") + local -a pkgnames local ret @@ -145,11 +173,19 @@ install_packages() { ret=$? rm -- "${pkgnames[@]/#/$copydir/root/}" - # If there is no PKGBUILD we are done - [[ -f PKGBUILD ]] || exit $ret + return $ret } +# Usage: prepare_chroot $copydir $HOME $repack $run_namcap +# Globals: +# - MAKEFLAGS +# - PACKAGER prepare_chroot() { + local copydir=$1 + local USER_HOME=$2 + local repack=$3 + local run_namcap=$4 + $repack || rm -rf "$copydir/build" local builduser_uid="${SUDO_UID:-$UID}" @@ -216,7 +252,14 @@ _chrootnamcap() { done } +# Usage: download_sources $copydir $src_owner +# Globals: +# - SRCDEST +# - USER download_sources() { + local copydir=$1 + local src_owner=$2 + local builddir="$(mktemp -d)" chmod 1777 "$builddir" @@ -235,7 +278,15 @@ download_sources() { rm -rf $builddir } +# Usage: move_products $copydir $owner +# Globals: +# - PKGDEST +# - LOGDEST move_products() { + local copydir=$1 + local src_owner=$2 + + local pkgfile for pkgfile in "$copydir"/pkgdest/*; do chown "$src_owner" "$pkgfile" mv "$pkgfile" "$PKGDEST" @@ -246,6 +297,7 @@ move_products() { fi done + local l for l in "$copydir"/logdest/*; do [[ $l == */logpipe.* ]] && continue chown "$src_owner" "$l" @@ -321,17 +373,27 @@ load_vars /etc/makepkg.conf [[ -d $SRCPKGDEST ]] || SRCPKGDEST=$PWD [[ -d $LOGDEST ]] || LOGDEST=$PWD -create_chroot +# Lock the chroot we want to use. We'll keep this lock until we exit. +lock 9 "$copydir.lock" "Locking chroot copy [%s]" "$copy" + +if [[ ! -d $copydir ]] || $clean_first; then + sync_chroot "$chrootdir" "$copy" +fi $update_first && arch-nspawn "$copydir" \ "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" \ pacman -Syu --noconfirm -[[ -n ${install_pkgs[*]} ]] && install_packages +if [[ -n ${install_pkgs[*]} ]]; then + install_packages "$copydir" "${install_pkgs[@]}" + ret=$? + # If there is no PKGBUILD we have done + [[ -f PKGBUILD ]] || return $ret +fi -download_sources +download_sources "$copydir" "$src_owner" -prepare_chroot +prepare_chroot "$copydir" "$USER_HOME" "$repack" if arch-nspawn "$copydir" \ --bind="$PWD:/startdir" \ @@ -339,12 +401,12 @@ if arch-nspawn "$copydir" \ "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" \ /chrootbuild then - move_products + move_products "$copydir" "$src_owner" else (( ret += 1 )) fi -$temp_chroot && clean_temporary +$temp_chroot && delete_chroot "$copydir" "$copy" if (( ret != 0 )); then if $temp_chroot; then From patchwork Wed Apr 5 19:35:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 91 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id kQrbID1H5ViYagAAtiB/HQ for ; Wed, 05 Apr 2017 21:36:29 +0200 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id E1FC240662; Wed, 5 Apr 2017 21:36:27 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Wed, 5 Apr 2017 21:36:27 +0200 (CEST) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 416B92C102; Wed, 5 Apr 2017 19:36:27 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 2DB702C0FC for ; Wed, 5 Apr 2017 19:36:25 +0000 (UTC) Received: from mav.lukeshu.com (team4272.com [IPv6:2001:19f0:5c00:8069:5400:ff:fe26:6a86]) by luna.archlinux.org (Postfix) with ESMTPS for ; Wed, 5 Apr 2017 19:36:25 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 96C2683EBA; Wed, 5 Apr 2017 15:36:23 -0400 (EDT) From: lukeshu@lukeshu.com To: arch-projects@lists.archlinux.org Date: Wed, 5 Apr 2017 15:35:59 -0400 Message-Id: <20170405193603.22277-4-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.12.1 In-Reply-To: <20170405193603.22277-1-lukeshu@lukeshu.com> References: <20170405193603.22277-1-lukeshu@lukeshu.com> Subject: [arch-projects] [devtools] [PATCH 3/7] makechrootpkg: Adjust to have the functions work with `set -u`. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 238 Status: X-Keywords: Content-Length: 1726 From: Luke Shumaker Even though main() doesn't call `set -u`; this way the functions will continue to work if copied into an environment with `set -u`, or so that we are ready if we ever want to start using `set -u`. --- makechrootpkg.in | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/makechrootpkg.in b/makechrootpkg.in index bdcaa88..f22fe49 100644 --- a/makechrootpkg.in +++ b/makechrootpkg.in @@ -32,7 +32,7 @@ bindmounts_ro=() bindmounts_rw=() copy=$USER -[[ -n $SUDO_USER ]] && copy=$SUDO_USER +[[ -n ${SUDO_USER:-} ]] && copy=$SUDO_USER [[ -z "$copy" || $copy = root ]] && copy=copy src_owner=${SUDO_USER:-$USER} @@ -90,7 +90,7 @@ load_vars() { [[ -f $makepkg_conf ]] || return 1 for var in {SRC,SRCPKG,PKG,LOG}DEST MAKEFLAGS PACKAGER; do - [[ -z ${!var} ]] && eval $(grep "^${var}=" "$makepkg_conf") + [[ -z ${!var:-} ]] && eval $(grep "^${var}=" "$makepkg_conf") done return 0 @@ -209,7 +209,7 @@ prepare_chroot() { sed -e '/^MAKEFLAGS=/d' -e '/^PACKAGER=/d' -i "$copydir/etc/makepkg.conf" for x in BUILDDIR=/build PKGDEST=/pkgdest SRCPKGDEST=/srcpkgdest SRCDEST=/srcdest LOGDEST=/logdest \ - "MAKEFLAGS='$MAKEFLAGS'" "PACKAGER='$PACKAGER'" + "MAKEFLAGS='${MAKEFLAGS:-}'" "PACKAGER='${PACKAGER:-}'" do grep -q "^$x" "$copydir/etc/makepkg.conf" && continue echo "$x" >>"$copydir/etc/makepkg.conf" @@ -384,7 +384,7 @@ $update_first && arch-nspawn "$copydir" \ "${bindmounts_ro[@]}" "${bindmounts_rw[@]}" \ pacman -Syu --noconfirm -if [[ -n ${install_pkgs[*]} ]]; then +if [[ -n ${install_pkgs[*]:-} ]]; then install_packages "$copydir" "${install_pkgs[@]}" ret=$? # If there is no PKGBUILD we have done From patchwork Wed Apr 5 19:36:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 92 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id qRMvJEBH5ViYagAAtiB/HQ for ; Wed, 05 Apr 2017 21:36:32 +0200 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id 4F051406FA; Wed, 5 Apr 2017 21:36:31 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Wed, 5 Apr 2017 21:36:31 +0200 (CEST) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 95DD02C106; Wed, 5 Apr 2017 19:36:28 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 17E3A2C0FF for ; Wed, 5 Apr 2017 19:36:27 +0000 (UTC) Received: from mav.lukeshu.com (mav.lukeshu.com [104.207.138.63]) by luna.archlinux.org (Postfix) with ESMTPS for ; Wed, 5 Apr 2017 19:36:26 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 01F9183EBB; Wed, 5 Apr 2017 15:36:25 -0400 (EDT) From: lukeshu@lukeshu.com To: arch-projects@lists.archlinux.org Date: Wed, 5 Apr 2017 15:36:00 -0400 Message-Id: <20170405193603.22277-5-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.12.1 In-Reply-To: <20170405193603.22277-1-lukeshu@lukeshu.com> References: <20170405193603.22277-1-lukeshu@lukeshu.com> Subject: [arch-projects] [devtools] [PATCH 4/7] mkarchroot, arch-nspawn: Add an `-f` flag to add files to copy. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 239 Status: X-Keywords: Content-Length: 2869 From: Luke Shumaker This allows us to copy in files like `qemu-arm-static`, which is necessary for running an ARM chroot on an x86 box. --- arch-nspawn.in | 12 +++++++++++- mkarchroot.in | 11 ++++++++++- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/arch-nspawn.in b/arch-nspawn.in index 7481d82..dceb3ed 100644 --- a/arch-nspawn.in +++ b/arch-nspawn.in @@ -15,6 +15,8 @@ m4_include(lib/archroot.sh) working_dir='' +files=() + usage() { echo "Usage: ${0##*/} [options] working-dir [systemd-nspawn arguments]" echo "A wrapper around systemd-nspawn. Provides support for pacman." @@ -23,15 +25,17 @@ usage() { echo ' -C Location of a pacman config file' echo ' -M Location of a makepkg config file' echo ' -c Set pacman cache' + echo ' -f Copy file from the host to the chroot' echo ' -h This message' exit 1 } -while getopts 'hC:M:c:' arg; do +while getopts 'hC:M:c:f:' arg; do case "$arg" in C) pac_conf="$OPTARG" ;; M) makepkg_conf="$OPTARG" ;; c) cache_dir="$OPTARG" ;; + f) files+=("$OPTARG") ;; h|?) usage ;; *) error "invalid argument '%s'" "$arg"; usage ;; esac @@ -77,6 +81,12 @@ copy_hostconf () { [[ -n $pac_conf ]] && cp $pac_conf "$working_dir/etc/pacman.conf" [[ -n $makepkg_conf ]] && cp $makepkg_conf "$working_dir/etc/makepkg.conf" + local file + for file in "${files[@]}"; do + mkdir -p "$(dirname "$working_dir$file")" + cp -T "$file" "$working_dir$file" + done + sed -r "s|^#?\\s*CacheDir.+|CacheDir = $(echo -n ${cache_dirs[@]})|g" -i "$working_dir/etc/pacman.conf" } # }}} diff --git a/mkarchroot.in b/mkarchroot.in index 1265335..4f35c1a 100644 --- a/mkarchroot.in +++ b/mkarchroot.in @@ -15,21 +15,25 @@ m4_include(lib/archroot.sh) working_dir='' +files=() + usage() { echo "Usage: ${0##*/} [options] working-dir package-list..." echo ' options:' echo ' -C Location of a pacman config file' echo ' -M Location of a makepkg config file' echo ' -c Set pacman cache' + echo ' -f Copy file from the host to the chroot' echo ' -h This message' exit 1 } -while getopts 'hC:M:c:' arg; do +while getopts 'hC:M:c:f:' arg; do case "$arg" in C) pac_conf="$OPTARG" ;; M) makepkg_conf="$OPTARG" ;; c) cache_dir="$OPTARG" ;; + f) files+=("$OPTARG") ;; h|?) usage ;; *) error "invalid argument '%s'" "$arg"; usage ;; esac @@ -67,6 +71,11 @@ if is_btrfs "$working_dir"; then chmod 0755 "$working_dir" fi +for file in "${files[@]}"; do + mkdir -p "$(dirname "$working_dir$file")" + cp "$file" "$working_dir$file" +done + pacstrap -GMcd ${pac_conf:+-C "$pac_conf"} "$working_dir" \ "${cache_dirs[@]/#/--cachedir=}" "$@" || die 'Failed to install all packages' From patchwork Wed Apr 5 19:36:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 93 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id GQs2AUVH5VieagAAtiB/HQ for ; Wed, 05 Apr 2017 21:36:37 +0200 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id EA88340690; Wed, 5 Apr 2017 21:36:34 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Wed, 5 Apr 2017 21:36:34 +0200 (CEST) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id EBC4E2C101; Wed, 5 Apr 2017 19:36:31 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 6D9C72C0E6 for ; Wed, 5 Apr 2017 19:36:30 +0000 (UTC) Received: from mav.lukeshu.com (team4272.com [IPv6:2001:19f0:5c00:8069:5400:ff:fe26:6a86]) by luna.archlinux.org (Postfix) with ESMTPS for ; Wed, 5 Apr 2017 19:36:30 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id C408383EBC; Wed, 5 Apr 2017 15:36:27 -0400 (EDT) From: lukeshu@lukeshu.com To: arch-projects@lists.archlinux.org Date: Wed, 5 Apr 2017 15:36:01 -0400 Message-Id: <20170405193603.22277-6-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.12.1 In-Reply-To: <20170405193603.22277-1-lukeshu@lukeshu.com> References: <20170405193603.22277-1-lukeshu@lukeshu.com> Subject: [arch-projects] [devtools] [PATCH 5/7] mkarchroot, arch-nspawn: Add an `-s` flag to inhibit `setarch`. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 240 Status: X-Keywords: Content-Length: 2282 From: Luke Shumaker This allows us to run an ARM chroot on an x86 box; as the binfmt runner will set the architecture for us, and the x86 `/usr/bin/setarch` program won't know about the ARM architecture string. --- arch-nspawn.in | 6 +++++- mkarchroot.in | 5 ++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/arch-nspawn.in b/arch-nspawn.in index dceb3ed..08ed2fa 100644 --- a/arch-nspawn.in +++ b/arch-nspawn.in @@ -26,16 +26,18 @@ usage() { echo ' -M Location of a makepkg config file' echo ' -c Set pacman cache' echo ' -f Copy file from the host to the chroot' + echo ' -s Do not run setarch' echo ' -h This message' exit 1 } -while getopts 'hC:M:c:f:' arg; do +while getopts 'hC:M:c:f:s' arg; do case "$arg" in C) pac_conf="$OPTARG" ;; M) makepkg_conf="$OPTARG" ;; c) cache_dir="$OPTARG" ;; f) files+=("$OPTARG") ;; + s) nosetarch=1 ;; h|?) usage ;; *) error "invalid argument '%s'" "$arg"; usage ;; esac @@ -105,6 +107,8 @@ copy_hostconf eval $(grep '^CARCH=' "$working_dir/etc/makepkg.conf") +[[ -z $nosetarch ]] || unset CARCH + exec ${CARCH:+setarch "$CARCH"} systemd-nspawn -q \ -D "$working_dir" \ --register=no \ diff --git a/mkarchroot.in b/mkarchroot.in index 4f35c1a..9f365e8 100644 --- a/mkarchroot.in +++ b/mkarchroot.in @@ -24,16 +24,18 @@ usage() { echo ' -M Location of a makepkg config file' echo ' -c Set pacman cache' echo ' -f Copy file from the host to the chroot' + echo ' -s Do not run setarch' echo ' -h This message' exit 1 } -while getopts 'hC:M:c:f:' arg; do +while getopts 'hC:M:c:f:s' arg; do case "$arg" in C) pac_conf="$OPTARG" ;; M) makepkg_conf="$OPTARG" ;; c) cache_dir="$OPTARG" ;; f) files+=("$OPTARG") ;; + s) nosetarch=1 ;; h|?) usage ;; *) error "invalid argument '%s'" "$arg"; usage ;; esac @@ -86,6 +88,7 @@ echo "$CHROOT_VERSION" > "$working_dir/.arch-chroot" systemd-machine-id-setup --root="$working_dir" exec arch-nspawn \ + ${nosetarch:+-s} \ ${pac_conf:+-C "$pac_conf"} \ ${makepkg_conf:+-M "$makepkg_conf"} \ ${cache_dir:+-c "$cache_dir"} \ From patchwork Wed Apr 5 19:36:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 94 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id eRbbBElH5ViYagAAtiB/HQ for ; Wed, 05 Apr 2017 21:36:41 +0200 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id E4F6B406FA; Wed, 5 Apr 2017 21:36:38 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Wed, 5 Apr 2017 21:36:38 +0200 (CEST) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 674302C10D; Wed, 5 Apr 2017 19:36:32 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id AC0F82C101 for ; Wed, 5 Apr 2017 19:36:30 +0000 (UTC) Received: from mav.lukeshu.com (mav.lukeshu.com [104.207.138.63]) by luna.archlinux.org (Postfix) with ESMTPS for ; Wed, 5 Apr 2017 19:36:30 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 849E383EB9; Wed, 5 Apr 2017 15:36:29 -0400 (EDT) From: lukeshu@lukeshu.com To: arch-projects@lists.archlinux.org Date: Wed, 5 Apr 2017 15:36:02 -0400 Message-Id: <20170405193603.22277-7-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.12.1 In-Reply-To: <20170405193603.22277-1-lukeshu@lukeshu.com> References: <20170405193603.22277-1-lukeshu@lukeshu.com> Subject: [arch-projects] [devtools] [PATCH 6/7] makechrootpkg: Improve status messages. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 241 Status: X-Keywords: Content-Length: 1842 From: Luke Shumaker In sync_chroot(), this makes the messages be a bit more precise with exactly which thing they are syncing where. This is based on my users expressing confusion at what is going on (especially when something is taking a long time, and they have to blame something for blocking). With these changes, I haven't gotten such confusion in a long time (but maybe my users just got used to it). In delete_chroot(), this changes "temporary copy" to "chroot copy", since in Parabola's version of the tools, the function can get called from other places, and it isn't necessarily operating on a temporary copy. --- makechrootpkg.in | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/makechrootpkg.in b/makechrootpkg.in index f22fe49..74c41de 100644 --- a/makechrootpkg.in +++ b/makechrootpkg.in @@ -115,9 +115,10 @@ sync_chroot() { if true; then # indent for rebasing/merging purposes # Get a read lock on the root chroot to make # sure we don't clone a half-updated chroot - slock 8 "$chrootdir/root.lock" "Locking clean chroot" + slock 8 "$chrootdir/root.lock" \ + "Locking clean chroot [%s]" "$chrootdir/root" - stat_busy "Creating clean working copy [%s]" "$copy" + stat_busy "Synchronizing chroot copy [%s] -> [%s]" "$chrootdir/root" "$copydir" if is_btrfs "$chrootdir" && ! mountpoint -q "$copydir"; then subvolume_delete_recursive "$copydir" || die "Unable to delete subvolume %s" "$copydir" @@ -142,7 +143,7 @@ delete_chroot() { local copydir=$1 local copy=${1:-$2} - stat_busy "Removing temporary copy [%s]" "$copy" + stat_busy "Removing chroot copy [%s]" "$copy" if is_btrfs "$chrootdir" && ! mountpoint -q "$copydir"; then btrfs subvolume delete "$copydir" >/dev/null || die "Unable to delete subvolume %s" "$copydir" From patchwork Wed Apr 5 19:36:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 95 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id 8Qo+F0tH5VicagAAtiB/HQ for ; Wed, 05 Apr 2017 21:36:43 +0200 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id F2669406E5; Wed, 5 Apr 2017 21:36:41 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Wed, 5 Apr 2017 21:36:41 +0200 (CEST) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 0EA892C0F0; Wed, 5 Apr 2017 19:36:35 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 7866A2C0E6 for ; Wed, 5 Apr 2017 19:36:33 +0000 (UTC) Received: from mav.lukeshu.com (team4272.com [IPv6:2001:19f0:5c00:8069:5400:ff:fe26:6a86]) by luna.archlinux.org (Postfix) with ESMTPS for ; Wed, 5 Apr 2017 19:36:32 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 32C7883EBA; Wed, 5 Apr 2017 15:36:31 -0400 (EDT) From: lukeshu@lukeshu.com To: arch-projects@lists.archlinux.org Date: Wed, 5 Apr 2017 15:36:03 -0400 Message-Id: <20170405193603.22277-8-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.12.1 In-Reply-To: <20170405193603.22277-1-lukeshu@lukeshu.com> References: <20170405193603.22277-1-lukeshu@lukeshu.com> Subject: [arch-projects] [devtools] [PATCH 7/7] mkarchroot: Don't let the environment affect pacstrap (sans proxy settings). X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 242 Status: X-Keywords: Content-Length: 1306 From: Luke Shumaker A previous iteration of this change (libretools commit d7dcce53396d) simply inserted `env -i` to clear the environment. However, that lead to it ignoring proxy settings, which some users had problems with: https://labs.parabola.nu/issues/487: > To fix other bugs, the pacstrap environment is blank, which also > means that the proxy settings are blank. So (in libretools commit d17d1d82349f), I changed it to use `declare -x` to inspect the environment, and create a version of it only consisting of variables ending with "_proxy" (case-insensitive). I honestly don't remember what "other bugs" prompted me to clear the environment in the first place. --- mkarchroot.in | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mkarchroot.in b/mkarchroot.in index 9f365e8..3aff357 100644 --- a/mkarchroot.in +++ b/mkarchroot.in @@ -78,6 +78,11 @@ for file in "${files[@]}"; do cp "$file" "$working_dir$file" done +_env=() +while read -r varname; do + _env+=("$varname=${!varname}") +done < <(declare -x | sed -r 's/^declare -x ([^=]*)=.*/\1/' | grep -i '_proxy$') +env -i "${_env[@]}" \ pacstrap -GMcd ${pac_conf:+-C "$pac_conf"} "$working_dir" \ "${cache_dirs[@]/#/--cachedir=}" "$@" || die 'Failed to install all packages'