From patchwork Sun Feb 18 17:17:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Shumaker X-Patchwork-Id: 407 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id CA59021DE34E for ; Sun, 18 Feb 2018 17:18:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED=-2.3 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-BL-Results: [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Sun, 18 Feb 2018 17:18:09 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 8081C8C7A6BCB; Sun, 18 Feb 2018 17:18:04 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS; Sun, 18 Feb 2018 17:17:56 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 211292B141; Sun, 18 Feb 2018 17:17:51 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=none Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 953DC2B04B for ; Sun, 18 Feb 2018 17:17:48 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by luna.archlinux.org (Postfix) with ESMTPS for ; Sun, 18 Feb 2018 17:17:48 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id B9DDC8C7A6B7C for ; Sun, 18 Feb 2018 17:17:43 +0000 (UTC) Received: from mav.lukeshu.com (mav.lukeshu.com [104.207.138.63]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS for ; Sun, 18 Feb 2018 17:17:39 +0000 (UTC) Received: from build64-par (unknown [IPv6:2601:803:202:9275:da50:e6ff:fe00:4a5b]) by mav.lukeshu.com (Postfix) with ESMTPSA id E073280505 for ; Sun, 18 Feb 2018 12:17:38 -0500 (EST) From: Luke Shumaker To: arch-projects@archlinux.org Date: Sun, 18 Feb 2018 12:17:31 -0500 Message-Id: <20180218171736.4473-4-lukeshu@lukeshu.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180218171736.4473-1-lukeshu@lukeshu.com> References: <20180218171736.4473-1-lukeshu@lukeshu.com> Subject: [arch-projects] [dbscripts] [PATCH v2 3/8] config: Rename PKGEXT to PKGEXT_glob X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Arch Linux projects development discussion Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" From: Luke Shumaker Unlike the other *EXT variables, which are prescriptive, PKGEXT is descriptive, and is a glob (this has always been the case). This is confusing because of the other variables, and because it is used prescriptively in makepkg.conf. Simply put, the configuration variable name PKGEXT is overloaded. Let's rename the glob version, to make things clearer. Now, in test/lib/common.bash, there *are* 2 places where it is used prescriptively. How does that work!? The value has a glob character in it! Well, because of sloppy quoting, it just kind of works out. So, in those places, *don't* rename it to PKGEXT_glob, but introduce a local PKGEXT variable with a prescriptive value. In db-update.bats, there's a place where it was set to .pkg.tar.gz both descriptively and prescriptively; so set both PKGEXT and PKGEXT_glob in that case. --- config | 4 +++- cron-jobs/ftpdir-cleanup | 6 +++--- db-functions | 4 ++-- db-move | 4 ++-- db-update | 8 ++++---- test/cases/db-repo-add.bats | 6 +++--- test/cases/db-update.bats | 4 ++-- test/cases/ftpdir-cleanup.bats | 4 ++-- test/lib/common.bash | 8 +++++--- 9 files changed, 26 insertions(+), 22 deletions(-) diff --git a/config b/config index d2c1942..13fe202 100644 --- a/config +++ b/config @@ -23,10 +23,12 @@ LOCK_TIMEOUT=300 STAGING="$HOME/staging" TMPDIR="/var/tmp" ARCHES=(x86_64) +# prescriptive DBEXT=".db.tar.gz" FILESEXT=".files.tar.gz" -PKGEXT=".pkg.tar.?z" SRCEXT=".src.tar.gz" +# descriptive; bash glob listing allowed extensions. +PKGEXT_glob=".pkg.tar.?z" # Allowed licenses: get sourceballs only for licenses in this array ALLOWED_LICENSES=('GPL' 'GPL1' 'GPL2' 'GPL3' 'LGPL' 'LGPL1' 'LGPL2' 'LGPL2.1' 'LGPL3') diff --git a/cron-jobs/ftpdir-cleanup b/cron-jobs/ftpdir-cleanup index 2f3d5aa..4dc02a0 100755 --- a/cron-jobs/ftpdir-cleanup +++ b/cron-jobs/ftpdir-cleanup @@ -38,7 +38,7 @@ for repo in ${PKGREPOS[@]}; do continue fi # get a list of actual available package files - find "${FTP_BASE}/${repo}/os/${arch}" -xtype f -name "*${PKGEXT}" -printf '%f\n' | sort > "${WORKDIR}/repo-${repo}-${arch}" + find "${FTP_BASE}/${repo}/os/${arch}" -xtype f -name "*${PKGEXT_glob}" -printf '%f\n' | sort > "${WORKDIR}/repo-${repo}-${arch}" # get a list of package files defined in the repo db bsdtar -xOf "${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT}" | awk '/^%FILENAME%/{getline;print}' | sort > "${WORKDIR}/db-${repo}-${arch}" @@ -62,7 +62,7 @@ for repo in ${PKGREPOS[@]}; do done # get a list of all available packages in the pacakge pool -find "$FTP_BASE/${PKGPOOL}" -name "*${PKGEXT}" -printf '%f\n' | sort > "${WORKDIR}/pool" +find "$FTP_BASE/${PKGPOOL}" -name "*${PKGEXT_glob}" -printf '%f\n' | sort > "${WORKDIR}/pool" # create a list of packages in our db find "${WORKDIR}" -maxdepth 1 -type f -name 'db-*' -exec cat {} \; | sort -u > "${WORKDIR}/db" @@ -75,7 +75,7 @@ if [ ${#old_pkgs[@]} -ge 1 ]; then done fi -old_pkgs=($(find ${CLEANUP_DESTDIR} -type f -name "*${PKGEXT}" -mtime +${CLEANUP_KEEP} -printf '%f\n')) +old_pkgs=($(find ${CLEANUP_DESTDIR} -type f -name "*${PKGEXT_glob}" -mtime +${CLEANUP_KEEP} -printf '%f\n')) if [ ${#old_pkgs[@]} -ge 1 ]; then msg "Removing old packages from the cleanup directory..." for old_pkg in ${old_pkgs[@]}; do diff --git a/db-functions b/db-functions index e8949d7..84be241 100644 --- a/db-functions +++ b/db-functions @@ -374,8 +374,8 @@ check_pkgrepos() { local pkgver="$(getpkgver ${pkgfile})" || return 1 local pkgarch="$(getpkgarch ${pkgfile})" || return 1 - [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT} ]] && return 1 - [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT}.sig ]] && return 1 + [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT_glob} ]] && return 1 + [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT_glob}.sig ]] && return 1 [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/} ]] && return 1 [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/}.sig ]] && return 1 diff --git a/db-move b/db-move index 37a9884..e3bc16e 100755 --- a/db-move +++ b/db-move @@ -49,7 +49,7 @@ for pkgbase in ${args[@]:2}; do for pkgname in ${pkgnames[@]}; do for tarch in ${tarches[@]}; do - getpkgfile "${ftppath_from}/${tarch}/"${pkgname}-${pkgver}-${pkgarch}${PKGEXT} >/dev/null + getpkgfile "${ftppath_from}/${tarch}/"${pkgname}-${pkgver}-${pkgarch}${PKGEXT_glob} >/dev/null done done continue 2 @@ -95,7 +95,7 @@ for pkgbase in ${args[@]:2}; do for pkgname in ${pkgnames[@]}; do for tarch in ${tarches[@]}; do - pkgpath=$(getpkgfile "${ftppath_from}/${tarch}/"${pkgname}-${pkgver}-${pkgarch}${PKGEXT}) + pkgpath=$(getpkgfile "${ftppath_from}/${tarch}/"${pkgname}-${pkgver}-${pkgarch}${PKGEXT_glob}) pkgfile="${pkgpath##*/}" ln -s "../../../${PKGPOOL}/${pkgfile}" ${ftppath_to}/${tarch}/ diff --git a/db-update b/db-update index 45755a4..4afeb6e 100755 --- a/db-update +++ b/db-update @@ -9,7 +9,7 @@ if (( $# >= 1 )); then fi # Find repos with packages to release -if ! staging_repos=($(find "${STAGING}" -mindepth 1 -type f -name "*${PKGEXT}" -printf '%h\n' | sort -u)); then +if ! staging_repos=($(find "${STAGING}" -mindepth 1 -type f -name "*${PKGEXT_glob}" -printf '%h\n' | sort -u)); then die "Could not read %s" "$STAGING" fi @@ -32,7 +32,7 @@ for repo in ${repos[@]}; do if ! check_repo_permission "${repo}"; then die "You don't have permission to update packages in %s" "$repo" fi - pkgs=($(getpkgfiles "${STAGING}/${repo}/"*${PKGEXT})) + pkgs=($(getpkgfiles "${STAGING}/${repo}/"*${PKGEXT_glob})) if (( $? == 0 )); then for pkg in ${pkgs[@]}; do if [[ -h ${pkg} ]]; then @@ -70,10 +70,10 @@ done for repo in ${repos[@]}; do msg "Updating [%s]..." "$repo" - any_pkgs=($(getpkgfiles "${STAGING}/${repo}/"*-any${PKGEXT} 2>/dev/null)) + any_pkgs=($(getpkgfiles "${STAGING}/${repo}/"*-any${PKGEXT_glob} 2>/dev/null)) for pkgarch in ${ARCHES[@]}; do add_pkgs=() - arch_pkgs=($(getpkgfiles "${STAGING}/${repo}/"*-${pkgarch}${PKGEXT} 2>/dev/null)) + arch_pkgs=($(getpkgfiles "${STAGING}/${repo}/"*-${pkgarch}${PKGEXT_glob} 2>/dev/null)) for pkg in ${arch_pkgs[@]} ${any_pkgs[@]}; do pkgfile="${pkg##*/}" msg2 "${pkgfile} (${pkgarch})" diff --git a/test/cases/db-repo-add.bats b/test/cases/db-repo-add.bats index 9fde381..ace7452 100644 --- a/test/cases/db-repo-add.bats +++ b/test/cases/db-repo-add.bats @@ -14,10 +14,10 @@ __movePackageToRepo() { fi # FIXME: pkgbase might not be part of the package filename - mv -v "${STAGING}"/${repo}/${pkgbase}-*-*-${arch}${PKGEXT}{,.sig} "${FTP_BASE}/${PKGPOOL}/" + mv -v "${STAGING}"/${repo}/${pkgbase}-*-*-${arch}${PKGEXT_glob}{,.sig} "${FTP_BASE}/${PKGPOOL}/" for tarch in ${tarches[@]}; do - ln -sv ${FTP_BASE}/${PKGPOOL}/${pkgbase}-*-*-${arch}${PKGEXT} "${FTP_BASE}/${repo}/os/${tarch}/" - ln -sv ${FTP_BASE}/${PKGPOOL}/${pkgbase}-*-*-${arch}${PKGEXT}.sig "${FTP_BASE}/${repo}/os/${tarch}/" + ln -sv ${FTP_BASE}/${PKGPOOL}/${pkgbase}-*-*-${arch}${PKGEXT_glob} "${FTP_BASE}/${repo}/os/${tarch}/" + ln -sv ${FTP_BASE}/${PKGPOOL}/${pkgbase}-*-*-${arch}${PKGEXT_glob}.sig "${FTP_BASE}/${repo}/os/${tarch}/" done } diff --git a/test/cases/db-update.bats b/test/cases/db-update.bats index 36511c9..c1b8eb4 100644 --- a/test/cases/db-update.bats +++ b/test/cases/db-update.bats @@ -92,7 +92,7 @@ load ../lib/common db-update checkPackage extra pkg-any-a - PKGEXT=.pkg.tar.gz releasePackage extra pkg-any-a + PKGEXT=.pkg.tar.gz PKGEXT_glob=.pkg.tar.gz releasePackage extra pkg-any-a run db-update [[ -z $BUILDDIR ]] || rm -f "${BUILDDIR}/$(__getCheckSum "${TMP}/svn-packages-copy/pkg-any-a/trunk/PKGBUILD")"/*.pkg.tar.gz{,.sig} [ "$status" -ne 0 ] @@ -151,7 +151,7 @@ load ../lib/common @test "add invalid signed package fails" { local p releasePackage extra 'pkg-any-a' - for p in "${STAGING}"/extra/*${PKGEXT}; do + for p in "${STAGING}"/extra/*${PKGEXT_glob}; do unxz $p xz -0 ${p%%.xz} done diff --git a/test/cases/ftpdir-cleanup.bats b/test/cases/ftpdir-cleanup.bats index 6280ce0..7dfad4a 100644 --- a/test/cases/ftpdir-cleanup.bats +++ b/test/cases/ftpdir-cleanup.bats @@ -13,8 +13,8 @@ __checkRepoRemovedPackage() { local pkgname for pkgname in $(__getPackageNamesFromPackageBase ${pkgbase}); do - [[ ! -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-*${PKGEXT} ]] - [[ ! -f ${FTP_BASE}/${repo}/os/${repoarch}/${pkgname}-*${PKGEXT} ]] + [[ ! -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-*${PKGEXT_glob} ]] + [[ ! -f ${FTP_BASE}/${repo}/os/${repoarch}/${pkgname}-*${PKGEXT_glob} ]] done } diff --git a/test/lib/common.bash b/test/lib/common.bash index d34af8a..94fedfe 100644 --- a/test/lib/common.bash +++ b/test/lib/common.bash @@ -26,11 +26,12 @@ __buildPackage() { local pkgarches local tarch local pkgnames + local PKGEXT="${PKGEXT:-.pkg.tar.xz}" if [[ -n ${BUILDDIR} ]]; then cache=${BUILDDIR}/$(__getCheckSum PKGBUILD) - if __isGlobfile "${cache}"/*${PKGEXT}; then - cp -Lv ${cache}/*${PKGEXT}{,.sig} ${pkgdest} + if __isGlobfile "${cache}"/*${PKGEXT_glob}; then + cp -Lv ${cache}/*${PKGEXT_glob}{,.sig} ${pkgdest} return 0 else mkdir -p ${cache} @@ -174,6 +175,7 @@ checkPackageDB() { local repoarches local pkgfile local pkgname + local PKGEXT="${PKGEXT:-.pkg.tar.xz}" # FIXME: We guess the location of the PKGBUILD used for this repo # We cannot read from trunk as __updatePKGBUILD() might have bumped the version @@ -221,7 +223,7 @@ checkPackageDB() { for db in ${DBEXT} ${FILESEXT}; do [ -r "${FTP_BASE}/${repo}/os/${repoarch}/${repo}${db%.tar.*}" ] - bsdtar -xf "${FTP_BASE}/${repo}/os/${repoarch}/${repo}${db%.tar.*}" -O | grep -q "${pkgfile%${PKGEXT}}" + bsdtar -xf "${FTP_BASE}/${repo}/os/${repoarch}/${repo}${db%.tar.*}" -O | grep -q "${pkgfile%${PKGEXT_glob}}" done done done