From patchwork Fri Dec 11 19:25:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Emil Velikov via arch-projects X-Patchwork-Id: 1835 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id AE2461CAA3704 for ; Fri, 11 Dec 2020 19:25:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1, MAILING_LIST_MULTI=-1,RCVD_IN_MSPIKE_H4=0.001,RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001,T_DMARC_POLICY_NONE=0.01 autolearn=ham autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.19] Received: from mail.archlinux.org (mail.archlinux.org [95.216.189.61]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Fri, 11 Dec 2020 19:25:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=archlinux.org; s=luna2; t=1607714749; bh=ELURsO+ShtJAalKuS640i+3QNptpa0IJFHLA1PP9+y0=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc; b=Roiy6DReuGg2MQf/Rb+0TAWxdE1A4OmpkePgFsqybpc+k4x+CQyVpt2UVtzCYLFHG A5GNZKbb/mSMo25VAHUonAjVe57lkv3+k7MM5v6KQLzTgeIOzoBv8caUqw0Cx9fdlF zUe28fiyPUr/ZlheWOmkr5q0FqxkfNe8zUtIM0BzkmtI0UHcfFgDO3nGlYZLWak6jh tpbgDirTgpxOBQ6qHgbu/vi7VyJNULG4WmJXQvlWhfuWwzmAW/3854+c7xLO5kyRko LEEhzd9m57GpVWaFEogx68+QTjiGNcVMzA8Gq3I3AcC1baEZWXD0OHyEeyjnz7I/A0 BEMoz4ao4uyjfR0honrOMI6POJ6El8OlEGDaAtEGG8W/oUE80SE0shhym0U0ANAlaU KJxyTjcCGiia0iQWAE+TbDOOjK83T98kVWI0XwDmtU0lnArhbiUC6gDy6gZ0UhRp7F Ys7+wKF5sfXW0lsDHBK8MMBBy7vNIZDCSBzxlDWQMs1D0CI1Hkv6hURHKO0jO4cwR2 1KcgyuOMAhwYyKzUJCFU2ZwqGF7DT3gJ9F2jlsWCkYwqVP5e1lD4lWSAPmhqUiwRJ2 pbmctiy9Ejm+5c3A0nQKFcFlH+GSKjqIHiV/NjWKw3B7Li7kxvNQfkMZyNCaajh4Xg jvcrLjXQBTgWdzP9ZH/bfLJk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=archlinux.org; s=dkim-rsa; t=1607714749; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ELURsO+ShtJAalKuS640i+3QNptpa0IJFHLA1PP9+y0=; b=WwyneTTrFYPF0f1vx5wTMw+C7o4qktPQJ8PzkoI2y9FYgvZ7s0mkHs5QRtOdB7MboeZEWv QWQV4t4uYOH89XATr70nbStXoz/JewEBFJLNKBTAn4JV0aL2p0hPD9Vjapj8xyGco+/Jqe weF2KOxGZMMAOIkm/OtkfAHzLrD3/FQ9v1lmyioUmzGM5EeKYeq/fZ+EzyTsWuHFe/ki+I MxCzBTuqVrACYSfbAPHoRdfx9+LyWCPy8GSWy45xLcqJGyRfjU9li9E8F6MFvqpevohmBj LN5nSZwwnqwSPibCjy7sKJRyI8btr0fYLAXoIJyg5zlgemYVFyE+x5TY1+XkuTH7TEw4m+ JfwNRTTMZt5vDo/DhhrfdNj6/mM4oejEEsko/t84UrOSycXQNVFfw9PRObXZqm+Nm70Xcb ez3AirXZe58O6KsesXcahNULSqKKBi8/X8Lm60Z+6kVqRATtoQ6ADT3RJRTSPl002qu3je I7Dpif6VX2MyI/n0BGRpr8wVczAtFJfdQqTZ+AbC6MgaDj4eXJEbuA5SgGBUynTUhl6Tzj a2UBQTo1E9IVci9mXZy6gAp17laPvqUlQeLiVKQSMjxx8r0x7R/uXrBsmY4TARcUBisIXd Lun5SE3UJwn8WtCH3ySfbn1TZXcp04I4+Y+kHswDg8M4RyXiFyUmo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=archlinux.org; s=dkim-ed25519; t=1607714749; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ELURsO+ShtJAalKuS640i+3QNptpa0IJFHLA1PP9+y0=; b=hbFFhCSyMOeuLg0Gy4bYp4Nl5WRYNTNIXve6cTltliMIaBSUKWiZQjFxNLJciyzb+oNucf f84VkQQTIa7jXKBA== Date: Fri, 11 Dec 2020 19:25:37 +0000 To: "arch-projects@archlinux.org" Message-ID: MIME-Version: 1.0 X-Rspamd-Queue-Id: 77B3B2C64AB X-Spamd-Result: default: False [-1.00 / 15.00]; HAS_REPLYTO(0.00)[meganomic@pm.me]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[pm.me:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[pm.me:+]; DMARC_POLICY_ALLOW(-0.50)[pm.me,quarantine]; NEURAL_HAM(-0.00)[-1.000]; TO_DN_EQ_ADDR_ALL(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.134:from] X-Rspamd-Server: mail.archlinux.org Subject: [arch-projects] [namcap] [PATCH] Add handling for compression into namcap.py X-BeenThere: arch-projects@archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Arch Linux projects development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: meganomic via arch-projects Reply-To: meganomic , Arch Linux projects development discussion Cc: meganomic Errors-To: arch-projects-bounces@archlinux.org Sender: "arch-projects" Authentication-Results: mail.archlinux.org; auth=pass smtp.auth=luna smtp.mailfrom=arch-projects-bounces@archlinux.org Adds handling of the compression and temporary storage into namcap.py so it can be removed from the bash script. https://bugs.archlinux.org/task/59844 mentions "use setuptools entry points." instead of the bash script but I don't know how to do that. I just removed it from the bash script for now. diff --git a/namcap b/namcap index ea0bc94..019b077 100755 --- a/namcap +++ b/namcap @@ -1,39 +1,2 @@  #!/bin/bash - -args='' -tmp=$(mktemp -d --tmpdir namcap.XXXXXXXXXX) -cleanup() { -    rm -rf "${tmp}" -} -trap 'cleanup' 0 - -for arg in "${@}"; do -    if echo "${arg}" | grep -q -E "^.+\.pkg\.tar\..+$" && [ -f "${arg}" ]; then - -        extra_opts='' -        case "${arg##*.}" in -            gz|z|Z) cmd='gzip' ;; -            bz2|bz) cmd='bzip2' ;; -            xz)     cmd='xz' ;; -            lzo)    cmd='lzop' ;; -            lrz)    cmd='lrzip' -                extra_opts="-q -o -" ;; -            lz4)    cmd='lz4' -                extra_opts="-q" ;; -            lz)     cmd='lzip' -                extra_opts="-q" ;; -            zst)    cmd='zstd' -                extra_opts="-q" ;; -            *)      echo 'Unsupported compression'; exit 1;; -        esac - -        tar="${tmp}/$(basename "${arg%.*}")" -        $cmd -dcf $extra_opts "${arg}" > "${tar}" - -        args="${args} ${tar}" -    else -        args="${args} ${arg}" -    fi -done - -/usr/bin/env python3 -m namcap ${args} +/usr/bin/env python3 -m namcap ${@} diff --git a/namcap.py b/namcap.py index a7f532a..c5b54ca 100755 --- a/namcap.py +++ b/namcap.py @@ -23,6 +23,9 @@ import getopt  import os  import sys  import tarfile +import subprocess +import tempfile +import pathlib    import Namcap.depends  import Namcap.tags @@ -237,14 +240,51 @@ if len(active_modules) == 0:    # Go through each package, get the info, and apply the rules  for package in packages: -    if not os.access(package, os.R_OK): +    pkgpath = pathlib.Path(package) + +    if not pkgpath.is_file():          print("Error: Problem reading %s" % package) -        usage() +        parser.print_usage() +        continue # Skip to next package if any + +    if pkgpath.with_suffix('').suffix == '.tar': +        # What compression is used? +        extra_opts = '' +        if pkgpath.suffix == '.gz' or pkgpath.suffix == '.z' or pkgpath.suffix == '.Z': +            cmd = 'gzip' +        elif pkgpath.suffix == '.bz2' or pkgpath.suffix == '.bz': +            cmd = 'bzip2' +        elif pkgpath.suffix == '.xz': +            cmd = 'xz' +        elif pkgpath.suffix == '.lzo': +            cmd = 'lzop' +        elif pkgpath.suffix == '.lrz': +            cmd = 'lrzip' +            extra_opts="-q -o -" +        elif pkgpath.suffix == '.lz4': +            cmd = 'lz4' +            extra_opts="-q" +        elif pkgpath.suffix == '.lz': +            cmd = 'lzip' +            extra_opts="-q" +        elif pkgpath.suffix == '.zst': +            cmd = 'zstd' +            extra_opts="-q" +        else: +            print("Unsupported compression!") +            continue # Skip to next package if any + +        # Run decompression and put the .tar file in a temporary directory +        tmpdir = tempfile.TemporaryDirectory(prefix='namcap.') +        tmpfilepath = pathlib.Path(tmpdir.name).joinpath(pkgpath.with_suffix('').name) +        subprocess.run(cmd + ' -dcf ' + extra_opts + pkgpath.as_posix() + ' > ' + tmpfilepath.as_posix(), shell=True) + +        if tmpfilepath.is_file() and tarfile.is_tarfile(tmpfilepath): +            process_realpackage(package, active_modules)   -    if os.path.isfile(package) and tarfile.is_tarfile(package): -        process_realpackage(package, active_modules)      elif 'PKGBUILD' in package:          process_pkgbuild(package, active_modules) +      else:          print("Error: %s not package or PKGBUILD" % package)