From patchwork Sun Feb 18 17:17:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 412 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id F33FB21DE36C for ; Sun, 18 Feb 2018 17:18:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED=-2.3 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-BL-Results: [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Sun, 18 Feb 2018 17:18:31 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 309668C7A6C1E; Sun, 18 Feb 2018 17:18:25 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS; Sun, 18 Feb 2018 17:18:12 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id EDD772B050; Sun, 18 Feb 2018 17:18:02 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 8A8C02078A for ; Sun, 18 Feb 2018 17:18:01 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by luna.archlinux.org (Postfix) with ESMTPS for ; Sun, 18 Feb 2018 17:18:01 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 84E398C7A6B93 for ; Sun, 18 Feb 2018 17:17:52 +0000 (UTC) Received: from mav.lukeshu.com (mav.lukeshu.com [IPv6:2001:19f0:5c00:8069:5400:ff:fe26:6a86]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS for ; Sun, 18 Feb 2018 17:17:47 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:da50:e6ff:fe00:4a5b]) by mav.lukeshu.com (Postfix) with ESMTPSA id 68A0A80509 for ; Sun, 18 Feb 2018 12:17:40 -0500 (EST) From: Luke Shumaker To: arch-projects@archlinux.org Date: Sun, 18 Feb 2018 12:17:35 -0500 Message-Id: <20180218171736.4473-8-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180218171736.4473-1-lukeshu@lukeshu.com> References: <20180218171736.4473-1-lukeshu@lukeshu.com> Subject: [arch-projects] [dbscripts] [PATCH v2 7/8] Replace all instances of `find` command with bash globbing X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" From: Luke Shumaker This fully removes the use of find from the codebase, and leads to a micro-optimization in a couple cases. Of the 5 instances in these files: - (ftpdir-cleanup) One was unnecessary as `cat` can natively consume all files passed to it (from a glob) and no directory traversal was in use. We don't have to worry about the glob being empty, unless config:ARCHES or config:PKGREPOS is mis/non-configured and is empty. - (sourceballs) One was only being used to strip leading directory paths, and can be replaced by ${filepath##*/} in a for loop. - (sourceballs) Two were unnecessary as they were hardcoded to read a single file. - (sourceballs) One was checking the modification time of the files, and can be replaced with touch(1) and [[ -nt ]]. Although this introduces an additional temporary file, this is not such a big deal. This is based on patches by Eli Schwartz --- cron-jobs/ftpdir-cleanup | 2 +- cron-jobs/sourceballs | 18 ++++++++++++++---- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/cron-jobs/ftpdir-cleanup b/cron-jobs/ftpdir-cleanup index 64d50ef..6a6c82f 100755 --- a/cron-jobs/ftpdir-cleanup +++ b/cron-jobs/ftpdir-cleanup @@ -70,7 +70,7 @@ for f in "$FTP_BASE/${PKGPOOL}"/*${PKGEXT_glob}; do printf '%s\n' "${f##*/}" done | sort > "${WORKDIR}/pool" # create a list of packages in our db -find "${WORKDIR}" -maxdepth 1 -type f -name 'db-*' -exec cat {} \; | sort -u > "${WORKDIR}/db" +cat "${WORKDIR}"/db-* 2>/dev/null | sort -u > "${WORKDIR}/db" old_pkgs=($(comm -23 "${WORKDIR}/pool" "${WORKDIR}/db")) if [ ${#old_pkgs[@]} -ge 1 ]; then diff --git a/cron-jobs/sourceballs b/cron-jobs/sourceballs index 9ab4e98..c47d5ad 100755 --- a/cron-jobs/sourceballs +++ b/cron-jobs/sourceballs @@ -46,7 +46,11 @@ for repo in ${PKGREPOS[@]}; do done # Create a list of all available source package file names -find "${FTP_BASE}/${SRCPOOL}" -xtype f -name "*${SRCEXT}" -printf '%f\n' | sort -u > "${WORKDIR}/available-src-pkgs" +for f in "${FTP_BASE}"/${SRCPOOL}/*${SRCEXT}; do + if [[ -f $f ]]; then + printf '%s\n' "${f##*/}" + fi +done | sort -u > "${WORKDIR}/available-src-pkgs" # Check for all packages if we need to build a source package for repo in ${PKGREPOS[@]}; do @@ -117,8 +121,8 @@ for repo in ${PKGREPOS[@]}; do done # Cleanup old source packages -find "${WORKDIR}" -maxdepth 1 -type f -name 'expected-src-pkgs' -exec cat {} \; | sort -u > "${WORKDIR}/expected-src-pkgs.sort" -find "${WORKDIR}" -maxdepth 1 -type f -name 'available-src-pkgs' -exec cat {} \; | sort -u > "${WORKDIR}/available-src-pkgs.sort" +sort -u "${WORKDIR}/expected-src-pkgs" > "${WORKDIR}/expected-src-pkgs.sort" +sort -u "${WORKDIR}/available-src-pkgs" > "${WORKDIR}/available-src-pkgs.sort" old_pkgs=($(comm -23 "${WORKDIR}/available-src-pkgs.sort" "${WORKDIR}/expected-src-pkgs.sort")) if [ ${#old_pkgs[@]} -ge 1 ]; then @@ -133,7 +137,13 @@ if [ ${#old_pkgs[@]} -ge 1 ]; then done fi -old_pkgs=($(find ${SOURCE_CLEANUP_DESTDIR} -type f -name "*${SRCEXT}" -mtime +${SOURCE_CLEANUP_KEEP} -printf '%f\n')) +old_pkgs=() +touch -d "${SOURCE_CLEANUP_KEEP} days ago" "${WORKDIR}/cleanup_timestamp" +for f in "${SOURCE_CLEANUP_DESTDIR}"/*${SRCEXT}; do + if [[ ${WORKDIR}/cleanup_timestamp -nt $f ]]; then + old_pkgs+=("${f##*/}") + fi +done if [ ${#old_pkgs[@]} -ge 1 ]; then msg "Removing old source packages from the cleanup directory..." for old_pkg in ${old_pkgs[@]}; do