From patchwork Fri Feb 16 03:45:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 397 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id AEC8C2192040 for ; Fri, 16 Feb 2018 03:45:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,RCVD_IN_DNSWL_MED=-2.3 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-BL-Results: [127.0.0.10] [127.0.9.2] [127.0.0.10] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Fri, 16 Feb 2018 03:45:05 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id D63008AF28C98; Fri, 16 Feb 2018 03:44:58 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS; Fri, 16 Feb 2018 03:44:57 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 37B5D2BA92; Fri, 16 Feb 2018 03:44:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1518752697; bh=7+WoQGQrzae/2OwSYqIjiG972COk8jLdKeSHVeq5sVM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc; b=LTs5MfH+m3at5Vi8CVsusPw5lMcu25fY8bvVGAtlQQxfFD7efnshg1ubNd3IcE8U8 De6iezFHAXDLDL80bLdItP34f3T9sYXAyw0AumK/TAXVCOHYWzMisvTSdMgWS/vSsH D1cgvNARK0/BE6NH42yIxY7IXLZfnXK4GCxhTA4OHgeMNXzZ43HOtGPDXouujcdyCr InTBMcmWdUV65z1XxiDyP/fXb6SFaiYvMgsZSbHoKyuIn3t/jv7huQCP1plqNHX5nT 0xEoFhEoSngskUo93pR8tZxORX+gB0P74aqq6xaGNrqxHUjFO0IYj7OsewBmf7IJnv oGa/OwuUjQp4epgoQKj+yMdlLZuUPBW01gnrxXR4raFQX7upC7Mk8+pVefnfXwuu2a UvhEus1M7yH5gu3GF6gpl9wGRfz4sUsx9YPjQEEoROvewWAGtq4pvG5cQG/NfK1BpL BjzN/k4Or4O/5bDODQpRhul8q+8U0BBVahAAQHdiUdKfKGEX14ar39+aC2PP3vceXv aGNbnMqjaF/JsGsbmkYg2hctZFwhjaYM+pdb8OZISl/N/lv25Ky+fSrk6DLu+Ck0oV /Q35Qbbawg9ZE11+KcM08ZsYUo9Sp0ysulnVCdxlISf26bYodvymr/XHLbmSNiFQvf 7+XPHckaW0/9e8tIJQyHOHZQ= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 455892B7AC for ; Fri, 16 Feb 2018 03:44:55 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Fri, 16 Feb 2018 03:44:55 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 7B9118AF28C67; Fri, 16 Feb 2018 03:44:43 +0000 (UTC) Received: from localhost.localdomain (ool-18bdf232.dyn.optonline.net [24.189.242.50]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) (Authenticated sender: eschwartz) by orion.archlinux.org (Postfix) with ESMTPSA id 03C3E8AF28C64; Fri, 16 Feb 2018 03:44:42 +0000 (UTC) To: arch-projects@archlinux.org Date: Thu, 15 Feb 2018 22:45:02 -0500 Message-Id: <20180216034504.27610-2-eschwartz@archlinux.org> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180216034504.27610-1-eschwartz@archlinux.org> References: <20180216034504.27610-1-eschwartz@archlinux.org> Subject: [arch-projects] [dbscripts] [PATCH 1/3] db-update: replace external find command with bash globbing X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Eli Schwartz via arch-projects Reply-To: Arch Linux projects development discussion Cc: Eli Schwartz Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" Don't bother emitting errors. bash doesn't show globbing errors if it cannot read a directory to try globbing there. And the former code never aborted on errors anyway, as without `set -o pipefail` the sort command swallowed the return code. Signed-off-by: Eli Schwartz --- db-functions | 4 ++++ db-update | 9 ++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/db-functions b/db-functions index e8949d7..f0f8980 100644 --- a/db-functions +++ b/db-functions @@ -2,6 +2,10 @@ . /usr/share/makepkg/util.sh +# global shell options for enhanced bash scripting +shopt -s globstar nullglob + + # Some PKGBUILDs need CARCH to be set CARCH="x86_64" diff --git a/db-update b/db-update index 45755a4..fa024b3 100755 --- a/db-update +++ b/db-update @@ -9,9 +9,12 @@ if (( $# >= 1 )); then fi # Find repos with packages to release -if ! staging_repos=($(find "${STAGING}" -mindepth 1 -type f -name "*${PKGEXT}" -printf '%h\n' | sort -u)); then - die "Could not read %s" "$STAGING" -fi +for f in "${STAGING}"/**/*${PKGEXT}; do + f="${f%/*}" + if [[ -d $f ]] && ! in_array "$f" "${staging_repos[@]}"; then + staging_repos+=("$f") + fi +done repos=() for staging_repo in ${staging_repos[@]##*/}; do From patchwork Fri Feb 16 03:45:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 399 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id 031522192056 for ; Fri, 16 Feb 2018 03:45:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,RCVD_IN_DNSWL_MED=-2.3 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-BL-Results: [127.0.9.2] [127.0.0.10] [127.0.0.10] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Fri, 16 Feb 2018 03:45:13 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 296598AF28CCD; Fri, 16 Feb 2018 03:45:07 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS; Fri, 16 Feb 2018 03:45:04 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id C60562BE41; Fri, 16 Feb 2018 03:45:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1518752704; bh=UxAhDNH2U4Hkx/RY7JS9pM5SgeqJ6VYtBakK4BbjQ9E=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc; b=VHvLVZNedBV1uNUdZ6Me9gq31jIYesWpDFr/KtNWjHClHMAoU+maFwmGPuHdSj7sx gu3IkMkZyvt0YL9pxv09x6HGIDZf2MtYpaI29cfVS2s5E7MypNh8cbWO2rvfZJTUb+ c90ndqVKzQlG2ACxfaLI/xsA8yqcf2IKyqSFUlbf5cKdEa5P6+RsIbYAeLjQawnYvB dhGt6Vd58QYG3l0CFCYfAWOjiH6AVB0kLwQ/9+5Ki4sDrLnhpOZIt3PeD6NcHbkKTW h0WCzNcuxeQ/zwE913Wj07c6iZ+6Pcs+Ab0ZrEPeGuPvvv6xj2/9832c55HQz5uFqS xe0osPG7l7QFhTafzOgvENFqiiUTfTOjLsWk5HUIqfWwQ2HvDVc6gMu614aGu9y+IG y1pB//qxY8ytYzXAW4C9cl3/eA7aN21e7/Vp+YW4DTuIDU89sd8D6Pm+doz2UjeEcV 94jFBItmyFrvvsxI+drYipAWDr0YdF3ypIAKUJ4x92sK9Kg9V58lk2bspJXn2qiYID zhxT5gAt6cvHW8KUEEYrvUYoMWwQcciq1aG6TUQ6NXjj4ZpPRmK8aRQ3GAICdtCWq/ ZjeiPOGVOIWlffjEv5XncpVrkiIwEZ5CdSsi6UN22U9rguBvraPDhxmiPhXL/Y5Eit R0yNo1lGOiJRBZccDH06mF54= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id CC9F02BE41 for ; Fri, 16 Feb 2018 03:45:02 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by luna.archlinux.org (Postfix) with ESMTPS for ; Fri, 16 Feb 2018 03:45:02 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 4658A8AF28C69; Fri, 16 Feb 2018 03:44:44 +0000 (UTC) Received: from localhost.localdomain (ool-18bdf232.dyn.optonline.net [24.189.242.50]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) (Authenticated sender: eschwartz) by orion.archlinux.org (Postfix) with ESMTPSA id B86EF8AF28C68; Fri, 16 Feb 2018 03:44:43 +0000 (UTC) To: arch-projects@archlinux.org Date: Thu, 15 Feb 2018 22:45:03 -0500 Message-Id: <20180216034504.27610-3-eschwartz@archlinux.org> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180216034504.27610-1-eschwartz@archlinux.org> References: <20180216034504.27610-1-eschwartz@archlinux.org> Subject: [arch-projects] [dbscripts] [PATCH 2/3] ftpdir-cleanup, sourceballs: replace external find command with bash globbing X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Eli Schwartz via arch-projects Reply-To: Arch Linux projects development discussion Cc: Eli Schwartz Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" This fully removes the use of find from the codebase, leads to a micro-optimization in a couple cases, and ensures that $PKGEXT is consistently treated as a shell globbing character (which is important because it is used as one). Of the eight instances in these files: - One was unnecessary as `cat` can natively consume all files passed to it and no directory traversal was in use. - Two were unnecessary as they were hardcoded to read a single file.... - Another four were only being used to strip leading directory paths, and can be replaced by globstar and ${filepath##*/} - The final two were checking the modification time of the files, and can be replaced with touch(1) and [[ -nt ]]. Although this introduces an additional temporary file, this is not such a big deal. Signed-off-by: Eli Schwartz --- cron-jobs/ftpdir-cleanup | 24 ++++++++++++++++++------ cron-jobs/sourceballs | 21 ++++++++++++++++----- 2 files changed, 34 insertions(+), 11 deletions(-) diff --git a/cron-jobs/ftpdir-cleanup b/cron-jobs/ftpdir-cleanup index 2f3d5aa..2d33047 100755 --- a/cron-jobs/ftpdir-cleanup +++ b/cron-jobs/ftpdir-cleanup @@ -38,7 +38,11 @@ for repo in ${PKGREPOS[@]}; do continue fi # get a list of actual available package files - find "${FTP_BASE}/${repo}/os/${arch}" -xtype f -name "*${PKGEXT}" -printf '%f\n' | sort > "${WORKDIR}/repo-${repo}-${arch}" + for f in "${FTP_BASE}"/${repo}/os/${arch}/*${PKGEXT}; do + if [[ -f $f ]]; then + printf '%s\n' "${f##*/}" + fi + done | sort > "${WORKDIR}/repo-${repo}-${arch}" # get a list of package files defined in the repo db bsdtar -xOf "${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT}" | awk '/^%FILENAME%/{getline;print}' | sort > "${WORKDIR}/db-${repo}-${arch}" @@ -61,10 +65,12 @@ for repo in ${PKGREPOS[@]}; do done done -# get a list of all available packages in the pacakge pool -find "$FTP_BASE/${PKGPOOL}" -name "*${PKGEXT}" -printf '%f\n' | sort > "${WORKDIR}/pool" +# get a list of all available packages in the package pool +for f in "$FTP_BASE/${PKGPOOL}"/*${PKGEXT}; do + printf '%s\n' "${f##*/}" +done | sort > "${WORKDIR}/pool" # create a list of packages in our db -find "${WORKDIR}" -maxdepth 1 -type f -name 'db-*' -exec cat {} \; | sort -u > "${WORKDIR}/db" +cat "${WORKDIR}"/db-* 2>/dev/null | sort -u > "${WORKDIR}/db" old_pkgs=($(comm -23 "${WORKDIR}/pool" "${WORKDIR}/db")) if [ ${#old_pkgs[@]} -ge 1 ]; then @@ -75,8 +81,14 @@ if [ ${#old_pkgs[@]} -ge 1 ]; then done fi -old_pkgs=($(find ${CLEANUP_DESTDIR} -type f -name "*${PKGEXT}" -mtime +${CLEANUP_KEEP} -printf '%f\n')) -if [ ${#old_pkgs[@]} -ge 1 ]; then +unset old_pkgs +touch -d "${CLEANUP_KEEP} days ago" "${WORKDIR}/cleanup_timestamp" +for f in "${CLEANUP_DESTDIR}"/**/*${PKGEXT}; do + if [[ ${WORKDIR}/cleanup_timestamp -nt $f ]]; then + old_pkgs+=("${f##*/}") + fi +done +if (( ${#old_pkgs[@]} > 1 )); then msg "Removing old packages from the cleanup directory..." for old_pkg in ${old_pkgs[@]}; do msg2 "${old_pkg}" diff --git a/cron-jobs/sourceballs b/cron-jobs/sourceballs index 9ab4e98..5844817 100755 --- a/cron-jobs/sourceballs +++ b/cron-jobs/sourceballs @@ -46,7 +46,11 @@ for repo in ${PKGREPOS[@]}; do done # Create a list of all available source package file names -find "${FTP_BASE}/${SRCPOOL}" -xtype f -name "*${SRCEXT}" -printf '%f\n' | sort -u > "${WORKDIR}/available-src-pkgs" +for f in "${FTP_BASE}"/${SRCPOOL}/*${SRCEXT}; do + if [[ -f $f ]]; then + printf '%s\n' "${f##*/}" + fi +done | sort -u > "${WORKDIR}/available-src-pkgs" # Check for all packages if we need to build a source package for repo in ${PKGREPOS[@]}; do @@ -117,8 +121,8 @@ for repo in ${PKGREPOS[@]}; do done # Cleanup old source packages -find "${WORKDIR}" -maxdepth 1 -type f -name 'expected-src-pkgs' -exec cat {} \; | sort -u > "${WORKDIR}/expected-src-pkgs.sort" -find "${WORKDIR}" -maxdepth 1 -type f -name 'available-src-pkgs' -exec cat {} \; | sort -u > "${WORKDIR}/available-src-pkgs.sort" +sort -u "${WORKDIR}/expected-src-pkgs" > "${WORKDIR}/expected-src-pkgs.sort" +sort -u "${WORKDIR}/available-src-pkgs" > "${WORKDIR}/available-src-pkgs.sort" old_pkgs=($(comm -23 "${WORKDIR}/available-src-pkgs.sort" "${WORKDIR}/expected-src-pkgs.sort")) if [ ${#old_pkgs[@]} -ge 1 ]; then @@ -133,8 +137,15 @@ if [ ${#old_pkgs[@]} -ge 1 ]; then done fi -old_pkgs=($(find ${SOURCE_CLEANUP_DESTDIR} -type f -name "*${SRCEXT}" -mtime +${SOURCE_CLEANUP_KEEP} -printf '%f\n')) -if [ ${#old_pkgs[@]} -ge 1 ]; then +unset old_pkgs +touch -d "${SOURCE_CLEANUP_KEEP} days ago" "${WORKDIR}/cleanup_timestamp" +for f in "${SOURCE_CLEANUP_DESTDIR}"/*${SRCEXT}; do + if [[ ${WORKDIR}/cleanup_timestamp -nt $f ]]; then + old_pkgs+=("${f##*/}") + fi +done + +if (( ${#old_pkgs[@]} > 1 )); then msg "Removing old source packages from the cleanup directory..." for old_pkg in ${old_pkgs[@]}; do msg2 "${old_pkg}" From patchwork Fri Feb 16 03:45:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 398 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id 4B6092192047 for ; Fri, 16 Feb 2018 03:45:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,RCVD_IN_DNSWL_MED=-2.3 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-BL-Results: [127.0.0.10] [127.0.9.2] [127.0.0.10] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Fri, 16 Feb 2018 03:45:09 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id E982F8AF28CAA; Fri, 16 Feb 2018 03:45:02 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS; Fri, 16 Feb 2018 03:45:02 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 738DF2BE1D; Fri, 16 Feb 2018 03:45:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1518752701; bh=1546IonCRVR1S3/+45GZDGnLXodeekExXfghqhMp3s8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc; b=UBF/hn7dg+ZmB4dXvB8t8aBypD6Oq2c0lX+18dZhWQgmyWuNKlhrIxC1MIM2+dpYj h0A32jSOJldrnwqMcmcNvdtPYDjU2SEOAFp/yM3ktOo7yVvxbgARyRPrX7qfrZn4T7 XGvE0BHoQLjV727I5JGfxBg35QUsbxW60orhxZF1b+HcBMDYkcekezOAvTrzF3/pq8 TgM4V/8CzKwpnRkyFrInwJNFjgn2pRyUO/k4tY/v4axP2qWY2ccCPuGkoYsjqOhiTy O1Uh1h9XC63pm6wgTnHTAVncHP6npGuaUTM/XoUcdq3zPD2roVYOmUNZF9xTl3TAsL mF4ev+w1/s3pst1PbNaO7pwH0nB2bV/6qSsQvzul3jSkiAiVGm+yO80OJxG+2i1ye5 uqsJpykrZc308k880HrXqPOaOBHPBGde56jqOSxv3zsmfgBCA7E+tJdE1VvZ2CODfT 4SeLcnQn1rpAefPo+PLE69XiU2x0whYFcZ5/LwznBpTdVeDv1gzqRNHVlxs5tsZM0r NtgZm2gBDz77ZLuhkr3gDHl6HiX9jHorg7xQZ8ZhGfL+7CShkn9HsSCMZGhAGwp47x GR7JakshBDQf2fjE9dzFLR0IxCQfBUGDCWU/jSJQnIW8YCoLOREwt+0djE3hYa3nVl 6Rh+o3LgKLjhVlFIOvhauAWY= Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id E24FA2BE1D for ; Fri, 16 Feb 2018 03:44:58 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Fri, 16 Feb 2018 03:44:58 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 079358AF28C6B; Fri, 16 Feb 2018 03:44:45 +0000 (UTC) Received: from localhost.localdomain (ool-18bdf232.dyn.optonline.net [24.189.242.50]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) (Authenticated sender: eschwartz) by orion.archlinux.org (Postfix) with ESMTPSA id 7A6CD8AF28C6A; Fri, 16 Feb 2018 03:44:44 +0000 (UTC) To: arch-projects@archlinux.org Date: Thu, 15 Feb 2018 22:45:04 -0500 Message-Id: <20180216034504.27610-4-eschwartz@archlinux.org> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180216034504.27610-1-eschwartz@archlinux.org> References: <20180216034504.27610-1-eschwartz@archlinux.org> Subject: [arch-projects] [dbscripts] [PATCH 3/3] Globally set $PKGEXT to a bash extended glob representing valid choices. X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Eli Schwartz via arch-projects Reply-To: Arch Linux projects development discussion Cc: Eli Schwartz Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" This can be anything makepkg.conf accepts, therefore it needs to be able to match all that. Document the fact that this has *always* been some sort of glob, and update the two cases where this was (not!) being evaluated by bash [[ ... ]], to use a proxy function is_globfile() Signed-off-by: Eli Schwartz --- config | 3 ++- db-functions | 11 ++++++++--- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/config b/config index d2c1942..7a90fc6 100644 --- a/config +++ b/config @@ -25,7 +25,8 @@ TMPDIR="/var/tmp" ARCHES=(x86_64) DBEXT=".db.tar.gz" FILESEXT=".files.tar.gz" -PKGEXT=".pkg.tar.?z" +# bash glob listing allowed extensions. Note that db-functions turns on extglob. +PKGEXT=".pkg.tar.@(gz|bz2|xz|lzo|lrz|Z)" SRCEXT=".src.tar.gz" # Allowed licenses: get sourceballs only for licenses in this array diff --git a/db-functions b/db-functions index f0f8980..7cf8444 100644 --- a/db-functions +++ b/db-functions @@ -3,7 +3,7 @@ . /usr/share/makepkg/util.sh # global shell options for enhanced bash scripting -shopt -s globstar nullglob +shopt -s extglob globstar nullglob # Some PKGBUILDs need CARCH to be set @@ -20,6 +20,11 @@ restore_umask () { umask $UMASK >/dev/null } +# Check if a file exists, even if the file uses wildcards +is_globfile() { + [[ -f $1 ]] +} + # just like mv -f, but we touch the file and then copy the content so # default ACLs in the target dir will be applied mv_acl() { @@ -378,8 +383,8 @@ check_pkgrepos() { local pkgver="$(getpkgver ${pkgfile})" || return 1 local pkgarch="$(getpkgarch ${pkgfile})" || return 1 - [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT} ]] && return 1 - [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT}.sig ]] && return 1 + is_globfile "${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXT} && return 1 + is_globfile "${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXT}.sig && return 1 [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/} ]] && return 1 [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/}.sig ]] && return 1