From patchwork Sat Mar 25 16:34:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 70 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id nqRNJDWc1lgFMgAAtiB/HQ for ; Sat, 25 Mar 2017 17:35:01 +0100 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id E044E40484; Sat, 25 Mar 2017 17:34:59 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Sat, 25 Mar 2017 17:34:59 +0100 (CET) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id AC8EB213CA; Sat, 25 Mar 2017 16:34:59 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id EB97D213CA for ; Sat, 25 Mar 2017 16:34:56 +0000 (UTC) Received: from mav.lukeshu.com (team4272.com [IPv6:2001:19f0:5c00:8069:5400:ff:fe26:6a86]) by luna.archlinux.org (Postfix) with ESMTPS for ; Sat, 25 Mar 2017 16:34:56 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id C13CE83EAD; Sat, 25 Mar 2017 12:34:55 -0400 (EDT) From: Luke Shumaker To: arch-projects@lists.archlinux.org Date: Sat, 25 Mar 2017 12:34:38 -0400 Message-Id: <20170325163441.28555-2-lukeshu@parabola.nu> X-Mailer: git-send-email 2.12.0 In-Reply-To: <20170325163441.28555-1-lukeshu@parabola.nu> References: <20170325163441.28555-1-lukeshu@parabola.nu> Subject: [arch-projects] [devtools] [PATCH 1/4] lib/common.sh: add 'lock_close'; use it as appropriate. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 197 Status: X-Keywords: Content-Length: 1547 `lock_close FD` is easier to remember than 'exec FD>&-`; and is especially easier if FD is a variable (though that isn't actually taken advantage of here). This uses Bash 4.1+ `exec {var}>&-`, rather than the clunkier `eval exec "$var>&-"` that was necessary in older versions of Bash. Thanks to Dave Reisner for pointing this new bit of syntax out to me the last time I submitted this (back in 2014, 4.1 had just come out). --- archbuild.in | 2 +- lib/common.sh | 8 ++++++++ makechrootpkg.in | 2 +- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/archbuild.in b/archbuild.in index a78353c..4d1b351 100644 --- a/archbuild.in +++ b/archbuild.in @@ -55,7 +55,7 @@ if ${clean_first} || [[ ! -d "${chroots}/${repo}-${arch}" ]]; then subvolume_delete_recursive "${copy}" rm -rf --one-file-system "${copy}" done - exec 9>&- + lock_close 9 rm -rf --one-file-system "${chroots}/${repo}-${arch}" mkdir -p "${chroots}/${repo}-${arch}" diff --git a/lib/common.sh b/lib/common.sh index 599be54..689772f 100644 --- a/lib/common.sh +++ b/lib/common.sh @@ -159,6 +159,14 @@ slock() { } ## +# usage : lock_close( $fd ) +## +lock_close() { + local fd=$1 + exec {fd}>&- +} + +## # usage: pkgver_equal( $pkgver1, $pkgver2 ) ## pkgver_equal() { diff --git a/makechrootpkg.in b/makechrootpkg.in index dc598f7..ad99334 100644 --- a/makechrootpkg.in +++ b/makechrootpkg.in @@ -108,7 +108,7 @@ create_chroot() { stat_done # Drop the read lock again - exec 8>&- + lock_close 8 fi # Update mtime From patchwork Sat Mar 25 16:34:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 71 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id MX4jLjic1lj/MQAAtiB/HQ for ; Sat, 25 Mar 2017 17:35:04 +0100 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id AA68940709; Sat, 25 Mar 2017 17:35:03 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Sat, 25 Mar 2017 17:35:03 +0100 (CET) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 09C1F213FB; Sat, 25 Mar 2017 16:35:02 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 7C225213F8 for ; Sat, 25 Mar 2017 16:35:01 +0000 (UTC) Received: from mav.lukeshu.com (team4272.com [IPv6:2001:19f0:5c00:8069:5400:ff:fe26:6a86]) by luna.archlinux.org (Postfix) with ESMTPS for ; Sat, 25 Mar 2017 16:35:01 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 5A98983EAC; Sat, 25 Mar 2017 12:35:00 -0400 (EDT) From: Luke Shumaker To: arch-projects@lists.archlinux.org Date: Sat, 25 Mar 2017 12:34:39 -0400 Message-Id: <20170325163441.28555-3-lukeshu@parabola.nu> X-Mailer: git-send-email 2.12.0 In-Reply-To: <20170325163441.28555-1-lukeshu@parabola.nu> References: <20170325163441.28555-1-lukeshu@parabola.nu> Subject: [arch-projects] [devtools] [PATCH 2/4] lib/common.sh: lock, slock: Allow locks to be inherited. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 199 Status: X-Keywords: Content-Length: 1642 Allow for locks to be inherited. Inheriting the lock is something that mkarchroot could do previously, but has since lost the ability to do. This allows for the programs to be more compos-able. Do this by instead of unconditionally opening $file on $fd, first check if $file is already open on $fd; and go ahead use it if it is. The naive way of doing this would be to `$(readlink /dev/fd/$fd)` and compare that to `$file`. However, if `$file` is itself a symlink; or there is a symlink somewhere in the path to `$file`, then this could easily fail. Instead, check `[[ "/dev/fd/$fd" -ef "$file" ]]`. Even though the Bash documentation (`help test`) says that `-ef` checks for if the two files are hard links to eachother, because it uses stat(3) (which resolves symlinks) to do this check, it also works with the /dev/fd/ soft links. --- lib/common.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/lib/common.sh b/lib/common.sh index 689772f..63b7795 100644 --- a/lib/common.sh +++ b/lib/common.sh @@ -138,7 +138,11 @@ get_full_version() { # usage : lock( $fd, $file, $message ) ## lock() { - eval "exec $1>"'"$2"' + # Only reopen the FD if it wasn't handed to us + if ! [[ "/dev/fd/$1" -ef "$2" ]]; then + eval "exec $1>"'"$2"' + fi + if ! flock -n $1; then stat_busy "$3" flock $1 @@ -150,7 +154,11 @@ lock() { # usage : slock( $fd, $file, $message ) ## slock() { - eval "exec $1>"'"$2"' + # Only reopen the FD if it wasn't handed to us + if ! [[ "/dev/fd/$1" -ef "$2" ]]; then + eval "exec $1>"'"$2"' + fi + if ! flock -sn $1; then stat_busy "$3" flock -s $1 From patchwork Sat Mar 25 16:34:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 72 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id AbkkIjuc1lgDMgAAtiB/HQ for ; Sat, 25 Mar 2017 17:35:07 +0100 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id BF79D406FB; Sat, 25 Mar 2017 17:35:06 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Sat, 25 Mar 2017 17:35:06 +0100 (CET) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 4936D213EB; Sat, 25 Mar 2017 16:35:06 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 487A521411 for ; Sat, 25 Mar 2017 16:35:03 +0000 (UTC) Received: from mav.lukeshu.com (team4272.com [IPv6:2001:19f0:5c00:8069:5400:ff:fe26:6a86]) by luna.archlinux.org (Postfix) with ESMTPS for ; Sat, 25 Mar 2017 16:35:03 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 28ECE83EAC; Sat, 25 Mar 2017 12:35:02 -0400 (EDT) From: Luke Shumaker To: arch-projects@lists.archlinux.org Date: Sat, 25 Mar 2017 12:34:40 -0400 Message-Id: <20170325163441.28555-4-lukeshu@parabola.nu> X-Mailer: git-send-email 2.12.0 In-Reply-To: <20170325163441.28555-1-lukeshu@parabola.nu> References: <20170325163441.28555-1-lukeshu@parabola.nu> Subject: [arch-projects] [devtools] [PATCH 3/4] lib/common.sh: lock, slock: Create directories for locks if necessary. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 200 Status: X-Keywords: Content-Length: 591 --- lib/common.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/common.sh b/lib/common.sh index 63b7795..6d873ed 100644 --- a/lib/common.sh +++ b/lib/common.sh @@ -140,6 +140,7 @@ get_full_version() { lock() { # Only reopen the FD if it wasn't handed to us if ! [[ "/dev/fd/$1" -ef "$2" ]]; then + mkdir -p -- "$(dirname -- "$2")" eval "exec $1>"'"$2"' fi @@ -156,6 +157,7 @@ lock() { slock() { # Only reopen the FD if it wasn't handed to us if ! [[ "/dev/fd/$1" -ef "$2" ]]; then + mkdir -p -- "$(dirname -- "$2")" eval "exec $1>"'"$2"' fi From patchwork Sat Mar 25 16:34:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 73 Return-Path: Delivered-To: patchwork@archlinux.org Received: from nymeria.archlinux.org by nymeria.archlinux.org (Dovecot) with LMTP id QUzbOEOc1lgFMgAAtiB/HQ for ; Sat, 25 Mar 2017 17:35:15 +0100 Received: from nymeria.archlinux.org (localhost.localdomain [127.0.0.1]) by nymeria.archlinux.org (Postfix) with ESMTP id EC23B4068D; Sat, 25 Mar 2017 17:35:13 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on nymeria.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.5 tests=BAYES_00,RCVD_IN_DNSWL_MED shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.1 Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by nymeria.archlinux.org (Postfix) with ESMTPS; Sat, 25 Mar 2017 17:35:13 +0100 (CET) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 9F267213F4; Sat, 25 Mar 2017 16:35:13 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 6B4C42140E for ; Sat, 25 Mar 2017 16:35:11 +0000 (UTC) Received: from mav.lukeshu.com (mav.lukeshu.com [104.207.138.63]) by luna.archlinux.org (Postfix) with ESMTPS for ; Sat, 25 Mar 2017 16:35:11 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:21f:e2ff:fe4d:191b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 69BAE83EAC; Sat, 25 Mar 2017 12:35:10 -0400 (EDT) From: Luke Shumaker To: arch-projects@lists.archlinux.org Date: Sat, 25 Mar 2017 12:34:41 -0400 Message-Id: <20170325163441.28555-5-lukeshu@parabola.nu> X-Mailer: git-send-email 2.12.0 In-Reply-To: <20170325163441.28555-1-lukeshu@parabola.nu> References: <20170325163441.28555-1-lukeshu@parabola.nu> Subject: [arch-projects] [devtools] [PATCH 4/4] lib/common.sh: Make setup_workdir()/cleanup() safe for programs to not use X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Cc: Luke Shumaker Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" X-UID: 201 Status: X-Keywords: Content-Length: 817 --- lib/common.sh | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/lib/common.sh b/lib/common.sh index 6d873ed..5ef9f97 100644 --- a/lib/common.sh +++ b/lib/common.sh @@ -59,12 +59,18 @@ stat_done() { printf "${BOLD}done${ALL_OFF}\n" >&2 } +_setup_workdir=false setup_workdir() { [[ -z $WORKDIR ]] && WORKDIR=$(mktemp -d --tmpdir "${0##*/}.XXXXXXXXXX") + _setup_workdir=true + trap 'trap_abort' INT QUIT TERM HUP + trap 'trap_exit' EXIT } cleanup() { - [[ -n $WORKDIR ]] && rm -rf "$WORKDIR" + if [[ -n $WORKDIR ]] && $_setup_workdir; then + rm -rf "$WORKDIR" + fi exit ${1:-0} } @@ -89,9 +95,6 @@ die() { cleanup 255 } -trap 'trap_abort' INT QUIT TERM HUP -trap 'trap_exit' EXIT - ## # usage : in_array( $needle, $haystack ) # return : 0 - found