From patchwork Tue Jan 19 18:30:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Gregory X-Patchwork-Id: 1858 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.15) for (single-drop); Tue, 19 Jan 2021 18:30:45 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id MMIpOVQlB2DongYAK+/4rw (envelope-from ) for ; Tue, 19 Jan 2021 18:30:44 +0000 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id 1620E3812A3; Tue, 19 Jan 2021 18:30:44 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id BDF392B76F; Tue, 19 Jan 2021 18:30:43 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id D79A82B76E for ; Tue, 19 Jan 2021 18:30:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on luna.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AC_FROM_MANY_DOTS=1.378, DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FROM=0.5, RCVD_IN_DNSWL_MED=-2.3,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=failed autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.10] [127.0.0.10] [127.0.9.2] Received: from mail.archlinux.org (mail.archlinux.org [IPv6:2a01:4f9:c010:3052::1]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 19 Jan 2021 18:30:40 +0000 (UTC) Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id 11B5B38129C for ; Tue, 19 Jan 2021 18:30:40 +0000 (UTC) Received: by mail-pg1-x52e.google.com with SMTP id 15so13484639pgx.7 for ; Tue, 19 Jan 2021 10:30:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=uw/5vi3DXa35uf6Kj0Hc7wZ13Z/ZeXDRvKLNdFZaRQU=; b=Vv/UwNfXdSS7C4r+QmnHqHMsU9lmTNdHArh+BJSMi61HjWptJY0XVa5ZquFtBmpfQC 93WgYFkTKKzV0k9YlmOQjVvaMSemOj0YVERaiaRRzYs2UX6OQk/Ohtl7wL5tjNmCRNB9 dbn6b6KNjy/O+32OnLEVbYpPlPMQHaVZN5Yj01vBc7ZIAWln8WCGvHGMdkLrUPckG82X 0CRr7UNfx7vNiDEpNR98dOFyy1kLQ8p2xIfEADcXSKF5Td8MaZ5nhmkWKDg1p4ycw7zs 1oe04/LUC7g40XQJ8/WYS4yAxjKYXcQy9K+auOTx6vc4BztSGBZD8LSw4OonRohLHeXu XGBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=uw/5vi3DXa35uf6Kj0Hc7wZ13Z/ZeXDRvKLNdFZaRQU=; b=XOw7tWsyrtvoAe0JATyDjyjezD1vOAOuY0d6M22LbCdFp/YuPV7mKT02lICj7fu9BJ RUhEtzYWNskPh8/Ibb54tNvUXvuQszd3pfWmmuccvc4IMx2PVdH/75a7D/ps83rYBy5C ZP/JEOuegcbcDwjgj46xsW2yngGN4ll6oHRH7wGNWNGiBkzx3GIX048W4dWEyKbNq5YT Ch4HoW1IUwOYpmuXpR1dGPbhwPRNzbdRGBzZ4gS7H1BCb8AVcgS6zyVQ3+DrP8OLjrLY L+HKMOufcofDafPgGEuhf0wQOFwQ5DRAx4KqxOE05qv4+KlvqodRz1o3gnOq4KEGT6Cb hhJw== X-Gm-Message-State: AOAM531RB69Cx0NbFSm5rBllRC+Icgaa4r+pRWWxT1GrDtmWEKyVD1La xGOqrzatH/w6NIcmbfP4paL3Vn4jVyA= X-Google-Smtp-Source: ABdhPJyGmnoYiL/3VQIlhrn0iXJ+xbks7Y5OEY7vUgvsj81K3wH+NTR/LA2Wt1y4ot+sHhdo90SiSQ== X-Received: by 2002:a63:4c4b:: with SMTP id m11mr5659553pgl.20.1611081037646; Tue, 19 Jan 2021 10:30:37 -0800 (PST) Received: from localhost.localdomain (c-98-207-137-206.hsd1.ca.comcast.net. [98.207.137.206]) by smtp.gmail.com with ESMTPSA id t8sm3647685pjd.51.2021.01.19.10.30.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 10:30:36 -0800 (PST) From: Andrew Gregory To: pacman-dev@archlinux.org Cc: Andrew Gregory Date: Tue, 19 Jan 2021 10:30:04 -0800 Message-Id: <20210119183010.12831-1-andrew.gregory.8@gmail.com> X-Mailer: git-send-email 2.30.0 MIME-Version: 1.0 Subject: [pacman-dev] [PATCH 1/7] pacman: add -w to -U X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=Vv/UwNfX; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 5.9.250.164 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org X-Rspamd-Queue-Id: 1620E3812A3 X-Spamd-Result: default: False [4.59 / 15.00]; HAS_REPLYTO(0.00)[pacman-dev@lists.archlinux.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,meta]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.9.250.164]; R_DKIM_REJECT(1.00)[gmail.com:s=20161025]; FREEMAIL_FROM(0.00)[gmail.com]; BROKEN_CONTENT_TYPE(1.50)[]; R_MISSING_CHARSET(2.50)[]; NEURAL_SPAM(0.00)[0.087]; DKIM_TRACE(0.00)[gmail.com:-]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; RECEIVED_SPAMHAUS_PBL(0.00)[98.207.137.206:received]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[andrewgregory8@gmail.com,pacman-dev-bounces@lists.archlinux.org]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_SEVEN(0.00)[7]; FORGED_SENDER_MAILLIST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : SPF not aligned (relaxed),none] X-Rspamd-Server: mail.archlinux.org Mostly for testing. Signed-off-by: Andrew Gregory --- doc/pacman.8.asciidoc | 6 +++--- src/pacman/pacman.c | 14 +++++++------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/doc/pacman.8.asciidoc b/doc/pacman.8.asciidoc index 476c16f3..bb205627 100644 --- a/doc/pacman.8.asciidoc +++ b/doc/pacman.8.asciidoc @@ -241,6 +241,9 @@ Transaction Options (apply to '-S', '-R' and '-U') Upgrade Options (apply to '-S' and '-U')[[UO]] ---------------------------------------------- +*-w, \--downloadonly*:: + Retrieve all packages from the server, but do not install/upgrade anything. + *\--asdeps*:: Install packages non-explicitly; in other words, fake their install reason to be installed as a dependency. This is useful for makepkg and other @@ -445,9 +448,6 @@ can be useful when the user switches from a testing repository to a stable one. Additional targets can also be specified manually, so that '-Su foo' will do a system upgrade and install/upgrade the "foo" package in the same operation. -*-w, \--downloadonly*:: - Retrieve all packages from the server, but do not install/upgrade anything. - *-y, \--refresh*:: Download a fresh copy of the master package database from the server(s) defined in linkman:pacman.conf[5]. This should typically be used each time diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c index 3045a58e..cc280d41 100644 --- a/src/pacman/pacman.c +++ b/src/pacman/pacman.c @@ -161,7 +161,6 @@ static void usage(int op, const char * const myname) addlist(_(" -q, --quiet show less information for query and search\n")); addlist(_(" -s, --search search remote repositories for matching strings\n")); addlist(_(" -u, --sysupgrade upgrade installed packages (-uu enables downgrades)\n")); - addlist(_(" -w, --downloadonly download packages but do not install/upgrade anything\n")); addlist(_(" -y, --refresh download fresh package databases from the server\n" " (-yy to force a refresh even if up to date)\n")); addlist(_(" --needed do not reinstall up to date packages\n")); @@ -189,6 +188,7 @@ static void usage(int op, const char * const myname) switch(op) { case PM_OP_SYNC: case PM_OP_UPGRADE: + addlist(_(" -w, --downloadonly download packages but do not install/upgrade anything\n")); addlist(_(" --overwrite \n" " overwrite conflicting files (can be used more than once)\n")); addlist(_(" --asdeps install packages as non-explicitly installed\n")); @@ -735,6 +735,12 @@ static int parsearg_upgrade(int opt) case OP_IGNOREGROUP: parsearg_util_addlist(&(config->ignoregrp)); break; + case OP_DOWNLOADONLY: + case 'w': + config->op_s_downloadonly = 1; + config->flags |= ALPM_TRANS_FLAG_DOWNLOADONLY; + config->flags |= ALPM_TRANS_FLAG_NOCONFLICTS; + break; default: return 1; } return 0; @@ -820,12 +826,6 @@ static int parsearg_sync(int opt) case 'u': (config->op_s_upgrade)++; break; - case OP_DOWNLOADONLY: - case 'w': - config->op_s_downloadonly = 1; - config->flags |= ALPM_TRANS_FLAG_DOWNLOADONLY; - config->flags |= ALPM_TRANS_FLAG_NOCONFLICTS; - break; case OP_REFRESH: case 'y': (config->op_s_sync)++; From patchwork Tue Jan 19 18:30:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Gregory X-Patchwork-Id: 1864 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.15) for (single-drop); Tue, 19 Jan 2021 18:40:19 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id maU9C5MnB2AtoAYAK+/4rw (envelope-from ) for ; Tue, 19 Jan 2021 18:40:19 +0000 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id AB77A38136C; Tue, 19 Jan 2021 18:40:18 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id EEB132C4FD; Tue, 19 Jan 2021 18:30:44 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 796D62B76F for ; Tue, 19 Jan 2021 18:30:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on luna.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FROM=0.5, RCVD_IN_DNSWL_MED=-2.3,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=failed autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.10] [127.0.0.10] [127.0.9.2] Received: from mail.archlinux.org (mail.archlinux.org [95.216.189.61]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 19 Jan 2021 18:30:41 +0000 (UTC) Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id BA1F538129E for ; Tue, 19 Jan 2021 18:30:40 +0000 (UTC) Received: by mail-pf1-x431.google.com with SMTP id q131so2430236pfq.10 for ; Tue, 19 Jan 2021 10:30:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JFHbhsTtLd/R/eRtBaRTFRFMwPkI0ijhef69bpJynFo=; b=UWvr/eEu1DRuZXjF61zN9bSlCAgJTIoMQFiEH07IS7xoojT9uZWXsYmrp7t8+Hg84/ opk0rQwqNpy/J/mY/hxTySZlqzDjbNOZPrrJ7PaFv22FEwRdtijh3rA3GXy0YKLsHLs2 5guLWGizxrc+X8x+0ZUe1tlUICgcjLXcNPO4wDMY+22lV6NNhkTKd8NRr5+dlA3lS7oi UM+7tqjXGaN6VLXz+jy9XF1eXLyu76GvHGXTiKSc4W1TJVlEZqaAZUo41ARy3O/Y73d6 9bpXyUU9IIqxX8+yySiON2TIRVf2UszklPCMr37ua++YdRUMoGV9LYFgmZWUpxrvi3B7 yTqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JFHbhsTtLd/R/eRtBaRTFRFMwPkI0ijhef69bpJynFo=; b=uJX7krExfk/i1siOfvz6mbE0f4WkBseGIODWR/a5O7RWhM8pTx2qbV5jrpIX1hH1nV U2R01qh5VOTCVRWn5gDMgwAPdz/4fnS2xvNrFCPu/AhzIBuzVl9Y62N4KQF/8whcXn8f KwkdCFqPo0YAGsoyx+4upYNN0DHfH9uw4Wvxyy3ZxLQzbV+NsAIORNWujJ83r48fxe2R DZWwGz/DN3TZXmoxCP0kK77Wog4GJTlftVOBwIPjr3Rz/3tRRrvq0cc7oh+v2vJ286Yi cTICuoa1/juGNRAodSOf5GEecgl2TiEn56fggVyQANq970wUT+dmrVynJWooTMw9phyL pZ+A== X-Gm-Message-State: AOAM530H2pGXr7hwNVtu6LE2+Tn3VfqXtvlNbPIfozCMk5/WSfkU9JEp kL4CtIpF+V9ITwFE9u0pqkENC27bOUg= X-Google-Smtp-Source: ABdhPJy9VSapliQYLjGc7elnlzzV+7DH/eQHy5DA2dCDGBrOF9vRcsElKZqngp4nOKSgZIN3VHc/Tw== X-Received: by 2002:a63:f608:: with SMTP id m8mr5478778pgh.11.1611081038656; Tue, 19 Jan 2021 10:30:38 -0800 (PST) Received: from localhost.localdomain (c-98-207-137-206.hsd1.ca.comcast.net. [98.207.137.206]) by smtp.gmail.com with ESMTPSA id t8sm3647685pjd.51.2021.01.19.10.30.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 10:30:38 -0800 (PST) From: Andrew Gregory To: pacman-dev@archlinux.org Cc: Andrew Gregory Date: Tue, 19 Jan 2021 10:30:05 -0800 Message-Id: <20210119183010.12831-2-andrew.gregory.8@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210119183010.12831-1-andrew.gregory.8@gmail.com> References: <20210119183010.12831-1-andrew.gregory.8@gmail.com> MIME-Version: 1.0 Subject: [pacman-dev] [PATCH 2/7] pactest.py: read options from PACTEST_OPTS X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=UWvr/eEu; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 5.9.250.164 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org X-Rspamd-Queue-Id: AB77A38136C X-Spamd-Result: default: False [4.59 / 15.00]; HAS_REPLYTO(0.00)[pacman-dev@lists.archlinux.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,body]; TO_DN_SOME(0.00)[]; R_DKIM_REJECT(1.00)[gmail.com:s=20161025]; R_SPF_ALLOW(-0.20)[+ip4:5.9.250.164:c]; FREEMAIL_FROM(0.00)[gmail.com]; BROKEN_CONTENT_TYPE(1.50)[]; R_MISSING_CHARSET(2.50)[]; NEURAL_SPAM(0.00)[0.082]; DKIM_TRACE(0.00)[gmail.com:-]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; RECEIVED_SPAMHAUS_PBL(0.00)[98.207.137.206:received]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[andrewgregory8@gmail.com,pacman-dev-bounces@lists.archlinux.org]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_SEVEN(0.00)[7]; FORGED_SENDER_MAILLIST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : SPF not aligned (relaxed),none] X-Rspamd-Server: mail.archlinux.org Makes it easier to pass options when not running pactest directly. Signed-off-by: Andrew Gregory --- test/pacman/pactest.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/test/pacman/pactest.py b/test/pacman/pactest.py index 20af41dc..c295bfc0 100755 --- a/test/pacman/pactest.py +++ b/test/pacman/pactest.py @@ -125,7 +125,8 @@ def create_parser(): # parse options opt_parser = create_parser() - (opts, args) = opt_parser.parse_args() + (opts, args) = opt_parser.parse_args(args=os.getenv('PACTEST_OPTS', '').split()) + (opts, args) = opt_parser.parse_args(values=opts) if args is None or len(args) == 0: tap.bail("no tests defined, nothing to do") From patchwork Tue Jan 19 18:30:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Gregory X-Patchwork-Id: 1859 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.15) for (single-drop); Tue, 19 Jan 2021 18:30:51 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id KHBPCVslB2DongYAK+/4rw (envelope-from ) for ; Tue, 19 Jan 2021 18:30:51 +0000 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id BC51F3812B7; Tue, 19 Jan 2021 18:30:50 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 2648A2C502; Tue, 19 Jan 2021 18:30:46 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 8FD262B779 for ; Tue, 19 Jan 2021 18:30:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on luna.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FROM=0.5, RCVD_IN_DNSWL_MED=-2.3,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=failed autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.9.2] [127.0.0.10] [127.0.0.10] Received: from mail.archlinux.org (mail.archlinux.org [95.216.189.61]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 19 Jan 2021 18:30:42 +0000 (UTC) Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id BF45E38129F for ; Tue, 19 Jan 2021 18:30:41 +0000 (UTC) Received: by mail-pg1-x530.google.com with SMTP id 15so13484711pgx.7 for ; Tue, 19 Jan 2021 10:30:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=A2TyLZ6rhi6zGm9O3Dt/e7kYENJahSuBcu5oPJ1hlPg=; b=l175fnPaghzEegGDzh/3WZPrxIiSkXEiV/+Y/j/Xqgoj5b3vBW8CnjcgqZ4qHiC/No dX4vnDxjVIA2kdT5rG3ChRWvgq4Ht7ruNe0AV/9P/7DC0VbP/ctcSaSb1WdiOxEbMHqJ OiVRvT7ckEcTtFxJUu0WLbeYQlTX19b1CUiA2xrjmgujNkMaegM00q9o05n+WHI/ojXx r9erBoirYcTxhxT1F8jiWpETm8bqqfXKaqEq+qoTgxV9BoAb1Zf53eu3ZfcDNDz+aQCm 9SglcinDJXdzT8FMsnSiT2XCkiI6UBwhBI6h2KjCxgAgQt4EyutoLKyNF6YohZNNwrFQ GyPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=A2TyLZ6rhi6zGm9O3Dt/e7kYENJahSuBcu5oPJ1hlPg=; b=nQJI3Yxr5jlnRHEodtUuXFWkUD4zIwK+ZT9E/+Tg3RuV/NnUktwdRz2Yq/+Obdxecs 5foC1VFE2xAnPxtecsXbGGfZTR7qvnpo9yM9K96uVF7DuLtEknBh/c6LS9KQuq8o472g Px/oOPeJyWpNgJfvfN9A4ApS3MVu/BgAvs+wTYoMJP19WpDzuCeV+eS28SzqzQ1aJw0R DeGds7IrS1BVmxwM22zI7sncSDL9XDUSwny1bPxRFgZBkhsUFaC/xK9esElmqm326FTB SLsKj46OYLVOnEyAFz9iQUBHXo7HHcm7z2I6LGAYHHUne/4rYiJGqZIjOZXEd/ByJFw0 jaVQ== X-Gm-Message-State: AOAM531UWjVPu/HP0x6hXQUJl1CMW6V8DEBMpg+J54rmt13/kxj5s0I1 56AO5eeB4zH1MqzthGAVAZNg5VsyhCY= X-Google-Smtp-Source: ABdhPJx0Pl4vh4CcVSL2IkdVYVxaUYMbYfiq3ORYM5z6/hUrvjOLWV12Lm9Ow4vTtrVvkxWzuB5o+g== X-Received: by 2002:a63:535a:: with SMTP id t26mr5562776pgl.433.1611081039743; Tue, 19 Jan 2021 10:30:39 -0800 (PST) Received: from localhost.localdomain (c-98-207-137-206.hsd1.ca.comcast.net. [98.207.137.206]) by smtp.gmail.com with ESMTPSA id t8sm3647685pjd.51.2021.01.19.10.30.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 10:30:39 -0800 (PST) From: Andrew Gregory To: pacman-dev@archlinux.org Cc: Andrew Gregory Date: Tue, 19 Jan 2021 10:30:06 -0800 Message-Id: <20210119183010.12831-3-andrew.gregory.8@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210119183010.12831-1-andrew.gregory.8@gmail.com> References: <20210119183010.12831-1-andrew.gregory.8@gmail.com> MIME-Version: 1.0 Subject: [pacman-dev] [PATCH 3/7] gitlab-ci: print output and logs for failed tests X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=l175fnPa; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 5.9.250.164 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org X-Rspamd-Queue-Id: BC51F3812B7 X-Spamd-Result: default: False [4.59 / 15.00]; HAS_REPLYTO(0.00)[pacman-dev@lists.archlinux.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,meta]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.9.250.164:c]; R_DKIM_REJECT(1.00)[gmail.com:s=20161025]; FREEMAIL_FROM(0.00)[gmail.com]; BROKEN_CONTENT_TYPE(1.50)[]; R_MISSING_CHARSET(2.50)[]; DKIM_TRACE(0.00)[gmail.com:-]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; RECEIVED_SPAMHAUS_PBL(0.00)[98.207.137.206:received]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[andrewgregory8@gmail.com,pacman-dev-bounces@lists.archlinux.org]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; NEURAL_HAM(-0.00)[-0.800]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_SEVEN(0.00)[7]; FORGED_SENDER_MAILLIST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : SPF not aligned (relaxed),none] X-Rspamd-Server: mail.archlinux.org --- .gitlab-ci.yml | 5 +++++ build-aux/cat-test-file | 11 +++++++++++ build-aux/print-failed-test-output | 17 +++++++++++++++++ 3 files changed, 33 insertions(+) create mode 100755 build-aux/cat-test-file create mode 100755 build-aux/print-failed-test-output diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index e889c498..6ca35941 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -2,6 +2,7 @@ variables: MAKEFLAGS: "-j10" VERBOSE: 1 PACMAN_OPTS: --needed --noconfirm --cachedir .pkg-cache + PACTEST_OPTS: --review --editor=../build-aux/cat-test-file cache: key: pkgs-v1 @@ -9,6 +10,10 @@ cache: # For some reason Gitlab CI only supports storing cache/artifacts in a path relative to the build directory - .pkg-cache +default: + after_script: + - build-aux/print-failed-test-output build/meson-logs/testlog.json + .arch-test: image: archlinux/base before_script: diff --git a/build-aux/cat-test-file b/build-aux/cat-test-file new file mode 100755 index 00000000..1bcc1113 --- /dev/null +++ b/build-aux/cat-test-file @@ -0,0 +1,11 @@ +#!/usr/bin/python3 + +import sys + +for path in sys.argv[1:]: + print('# -----------------------------------') + print('# ' + path + ':') + print('# -----------------------------------') + with open(path, 'r') as f: + for line in f: + print('# ' + line, end='') diff --git a/build-aux/print-failed-test-output b/build-aux/print-failed-test-output new file mode 100755 index 00000000..1f1ab76d --- /dev/null +++ b/build-aux/print-failed-test-output @@ -0,0 +1,17 @@ +#!/usr/bin/python + +import json +import sys + +def print_result(result): + print('==================================================================') + print(result['name']) + print(' '.join(result['command'])) + print('==================================================================') + print(result['stdout']) + +with open(sys.argv[1], 'r') as f: + for line in f: + result = json.loads(line) + if result['result'] == 'FAIL': + print_result(result) From patchwork Tue Jan 19 18:30:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Gregory X-Patchwork-Id: 1860 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.15) for (single-drop); Tue, 19 Jan 2021 18:30:56 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id YKDnIV8lB2BjngYAK+/4rw (envelope-from ) for ; Tue, 19 Jan 2021 18:30:55 +0000 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id 3E0083812C4; Tue, 19 Jan 2021 18:30:55 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 52F532C506; Tue, 19 Jan 2021 18:30:46 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 2A2D12B77C for ; Tue, 19 Jan 2021 18:30:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on luna.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FROM=0.5, RCVD_IN_DNSWL_MED=-2.3,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=failed autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.10] [127.0.9.2] [127.0.0.10] Received: from mail.archlinux.org (mail.archlinux.org [95.216.189.61]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 19 Jan 2021 18:30:44 +0000 (UTC) Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id 7723F3812A1 for ; Tue, 19 Jan 2021 18:30:43 +0000 (UTC) Received: by mail-pl1-x62d.google.com with SMTP id t6so11012102plq.1 for ; Tue, 19 Jan 2021 10:30:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gzc+QOwHSxSk21l2YGaKytbKui0DGAIEknR/LSni40g=; b=HT1SNouFBLRf1lw9AFOjiyGH3dC4538cHdA4mZLXxy9zCrsEQ+U5G45FAR0zxLLHyD sx7IhrXxQMXdnhOu/wQG7JrrChT2VzjXx1OcpyajYHERJdWUcvU8cHNDolkyyq08zOcj bULkvEBmf1/wuv0Tbd1Nr4zcdPXg+U31WpPCiz3NijW136YLEseT+00Uk3IqHw/v6EbS eUwupaXwRG3XmB93j4q8BCtlSsN/yGqyoSW3+pxK2hMiJQ4zvOdNU/1Fn1Dhm4UYEilY LWy+UXXhK5pJ8aRaOaNzl2HXjHCnLjYfxFVZZrLz5/zKk4eyCH4cdLvLXBAitsTJy6eq KSAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gzc+QOwHSxSk21l2YGaKytbKui0DGAIEknR/LSni40g=; b=s8Ym0v+b4IF+HIItmlitaMFBopdm8e9eRQpwJatxNorResOE26HsDhplhet9V26lw+ V//GAOA5ghlkuH+WLSP9/0qlMAEW5Ohlza9e5nvIHAE924OooubeooVKJVGpqRe7sUr3 GC5Ivut1Z7h/3scKXZ4NKYDWHh/bGDaUP+8PTJXi8HrNhZIrdfRwQRAPIW2wLPmGKnMI R8OV7aikENYRojF6/x24EaJj3Ix8er4mjzAll0TSlkLSZC7AaRxgVS8FT7P5q0/U/A8r cY0tQLdsw7SHpiikIDwspVOSSulkHTQ7ihCl/ANbulkxhyygDVH7FbhL/yAsaCLW0Aoa AmKQ== X-Gm-Message-State: AOAM532v8LLcZ5PpHcpVPLZ9AJ2Bbr2JadNJxLOLeCpoT4L7XWU/51/Q dPbGiuPYsz4uyY5jpO+5DZths6NFLp8= X-Google-Smtp-Source: ABdhPJyg/BwqT4Xzl0CdUL4rEYjPHiO+f+6Jm5I8aeMF0UHFP+RpUqWpx+RWZcBo0gpKKVLvEJ1wtQ== X-Received: by 2002:a17:902:e80c:b029:de:a20b:7a9c with SMTP id u12-20020a170902e80cb02900dea20b7a9cmr6218959plg.12.1611081041055; Tue, 19 Jan 2021 10:30:41 -0800 (PST) Received: from localhost.localdomain (c-98-207-137-206.hsd1.ca.comcast.net. [98.207.137.206]) by smtp.gmail.com with ESMTPSA id t8sm3647685pjd.51.2021.01.19.10.30.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 10:30:40 -0800 (PST) From: Andrew Gregory To: pacman-dev@archlinux.org Cc: Andrew Gregory Date: Tue, 19 Jan 2021 10:30:07 -0800 Message-Id: <20210119183010.12831-4-andrew.gregory.8@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210119183010.12831-1-andrew.gregory.8@gmail.com> References: <20210119183010.12831-1-andrew.gregory.8@gmail.com> MIME-Version: 1.0 Subject: [pacman-dev] [PATCH 4/7] add CACHE_FEXISTS and CACHE_FCONTENTS test rules X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=HT1SNouF; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 5.9.250.164 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org X-Rspamd-Queue-Id: 3E0083812C4 X-Spamd-Result: default: False [4.59 / 15.00]; HAS_REPLYTO(0.00)[pacman-dev@lists.archlinux.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,meta]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.9.250.164:c]; R_DKIM_REJECT(1.00)[gmail.com:s=20161025]; FREEMAIL_FROM(0.00)[gmail.com]; BROKEN_CONTENT_TYPE(1.50)[]; R_MISSING_CHARSET(2.50)[]; NEURAL_SPAM(0.00)[0.153]; DKIM_TRACE(0.00)[gmail.com:-]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; RECEIVED_SPAMHAUS_PBL(0.00)[98.207.137.206:received]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[andrewgregory8@gmail.com,pacman-dev-bounces@lists.archlinux.org]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_SEVEN(0.00)[7]; FORGED_SENDER_MAILLIST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : SPF not aligned (relaxed),none] X-Rspamd-Server: mail.archlinux.org The existing CACHE_EXISTS rule takes a package, which is not suitable for -U tests that need to be able to check for specific files. Signed-off-by: Andrew Gregory --- test/pacman/pmrule.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/test/pacman/pmrule.py b/test/pacman/pmrule.py index 59293266..aef73cdb 100644 --- a/test/pacman/pmrule.py +++ b/test/pacman/pmrule.py @@ -181,6 +181,16 @@ def check(self, test): if not pkg or not os.path.isfile( os.path.join(cachedir, pkg.filename())): success = 0 + elif case == "FEXISTS": + if not os.path.isfile(os.path.join(cachedir, key)): + success = 0 + elif case == "FCONTENTS": + filename = os.path.join(cachedir, key) + try: + with open(filename, 'r') as f: + success = f.read() == value + except: + success = 0 else: tap.diag("Rule kind '%s' not found" % kind) success = -1 From patchwork Tue Jan 19 18:30:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Gregory X-Patchwork-Id: 1861 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.15) for (single-drop); Tue, 19 Jan 2021 18:31:00 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id CH6rBWQlB2BjngYAK+/4rw (envelope-from ) for ; Tue, 19 Jan 2021 18:31:00 +0000 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id B28D23812CF; Tue, 19 Jan 2021 18:30:59 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 7E9A52C50A; Tue, 19 Jan 2021 18:30:47 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 6A1722C500 for ; Tue, 19 Jan 2021 18:30:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on luna.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FROM=0.5, RCVD_IN_DNSWL_MED=-2.3,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=failed autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.10] [127.0.9.2] [127.0.0.10] Received: from mail.archlinux.org (mail.archlinux.org [IPv6:2a01:4f9:c010:3052::1]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 19 Jan 2021 18:30:45 +0000 (UTC) Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id A27AE3812A9 for ; Tue, 19 Jan 2021 18:30:44 +0000 (UTC) Received: by mail-pl1-x62e.google.com with SMTP id t6so11012139plq.1 for ; Tue, 19 Jan 2021 10:30:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kax+dwN4Aqvf1gwWKxwGGXXNPIplGiB5p+CNTwYoS90=; b=rULKWUktBErm95lcWsqjKTBR/6auTQ9b9tT6rBSVRhwfQ5buxAy97O9fF6QOfp2bpI rszUUS3J1UblayAOdxHIw4zTIlggqstsdVlfO3fbPJZswwBjneQAJ0ZlHekDerSoLySB 5BTG/rMa/UwKEeLsV8ODEucRm0Q09B5WmmmbsbYXzTvZ9No87uFeGPOweR/LYsvZjowa 44/A7mI05W277jPVuyda6sobTMaW6gNMFo5gEYL6AdLCiAQ5pF+cDiefNBt/caZnXWu1 oKlVhuupMjHYgI3AeQktnON5LJsvlKQ6ndWRWStKwnhnuASRWzL3i9Kz5hcSss+2W8lD nO9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kax+dwN4Aqvf1gwWKxwGGXXNPIplGiB5p+CNTwYoS90=; b=oUH4Emnt65mi800HoTtAAGz9imBKnudUqco7iZvLmVf3IyHQUAxDfAeCG/mci40FqG 8bKiqLl3wO1RCVOtLkZksIiVZz+z5ARrhIMLwrwkv6d5GRkLOydUpgPfkMr4yCEksajZ +DY9UMZC51Zh4IjM+qdttJ2dySHtMiFrmtZpMmy1vsPYg5rYAu98U7YUnQRy3f58Nf+6 xoD83GxpAsNq5G1m5dePEfPI8RgGa/zISYI/B3ugGTvg261fR6CreXiquiKNRrSEE2+G 5+x4bZ2GG375OVWcXSU05lTgW/Lk3emPH2wY8hKuntYJSiOESmebFBflbf7vPOchj1Cy rOqg== X-Gm-Message-State: AOAM533Gv4SiP6PezcGT5Z8Sbmh9vqpPjv4YXWcoVgwdcd3gAF5slVpZ vN0kHMBavHIbR58zl2mS7wWbI5hTGlA= X-Google-Smtp-Source: ABdhPJwbgJWXYLx/RIj+sgCLHJh1L6uCqM4PwTfanOdbvG1jltQI399SLUuI+M7neZ6E70ySt2ZJMQ== X-Received: by 2002:a17:902:694c:b029:db:d939:1061 with SMTP id k12-20020a170902694cb02900dbd9391061mr5992768plt.80.1611081042672; Tue, 19 Jan 2021 10:30:42 -0800 (PST) Received: from localhost.localdomain (c-98-207-137-206.hsd1.ca.comcast.net. [98.207.137.206]) by smtp.gmail.com with ESMTPSA id t8sm3647685pjd.51.2021.01.19.10.30.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 10:30:42 -0800 (PST) From: Andrew Gregory To: pacman-dev@archlinux.org Cc: Andrew Gregory Date: Tue, 19 Jan 2021 10:30:08 -0800 Message-Id: <20210119183010.12831-5-andrew.gregory.8@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210119183010.12831-1-andrew.gregory.8@gmail.com> References: <20210119183010.12831-1-andrew.gregory.8@gmail.com> MIME-Version: 1.0 Subject: [pacman-dev] [PATCH 5/7] pmdb: add option to skip populating file:// server X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=rULKWUkt; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 5.9.250.164 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org X-Rspamd-Queue-Id: B28D23812CF X-Spamd-Result: default: False [4.59 / 15.00]; HAS_REPLYTO(0.00)[pacman-dev@lists.archlinux.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,meta]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.9.250.164:c]; R_DKIM_REJECT(1.00)[gmail.com:s=20161025]; FREEMAIL_FROM(0.00)[gmail.com]; BROKEN_CONTENT_TYPE(1.50)[]; R_MISSING_CHARSET(2.50)[]; NEURAL_SPAM(0.00)[0.140]; DKIM_TRACE(0.00)[gmail.com:-]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; RECEIVED_SPAMHAUS_PBL(0.00)[98.207.137.206:received]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[andrewgregory8@gmail.com,pacman-dev-bounces@lists.archlinux.org]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_SEVEN(0.00)[7]; FORGED_SENDER_MAILLIST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : SPF not aligned (relaxed),none] X-Rspamd-Server: mail.archlinux.org Populating a file:// Server prevents any manually registered HTTP servers from ever being used. Signed-off-by: Andrew Gregory --- test/pacman/pmdb.py | 1 + test/pacman/pmtest.py | 7 ++++--- test/pacman/util.py | 9 ++++----- 3 files changed, 9 insertions(+), 8 deletions(-) diff --git a/test/pacman/pmdb.py b/test/pacman/pmdb.py index 971e28eb..cae51b4e 100644 --- a/test/pacman/pmdb.py +++ b/test/pacman/pmdb.py @@ -54,6 +54,7 @@ def __init__(self, treename, root): self.root = root self.pkgs = [] self.option = {} + self.syncdir = True if self.treename == "local": self.dbdir = os.path.join(root, util.PM_DBPATH, treename) self.dbfile = None diff --git a/test/pacman/pmtest.py b/test/pacman/pmtest.py index ed6eab6a..8669f31d 100644 --- a/test/pacman/pmtest.py +++ b/test/pacman/pmtest.py @@ -167,10 +167,11 @@ def generate(self, pacman): vprint("\t%s" % os.path.join(util.PM_CACHEDIR, pkg.filename())) if self.cachepkgs: pkg.makepkg(cachedir) - else: + elif value.syncdir: pkg.makepkg(os.path.join(syncdir, value.treename)) - pkg.md5sum = util.getmd5sum(pkg.path) - pkg.csize = os.stat(pkg.path)[stat.ST_SIZE] + if pkg.path: + pkg.md5sum = util.getmd5sum(pkg.path) + pkg.csize = os.stat(pkg.path)[stat.ST_SIZE] # Creating sync database archives vprint(" Creating databases") diff --git a/test/pacman/util.py b/test/pacman/util.py index 1e0eb385..2957a020 100644 --- a/test/pacman/util.py +++ b/test/pacman/util.py @@ -122,11 +122,10 @@ def mkcfgfile(filename, root, option, db): for key in sorted(db.keys()): if key != "local": value = db[key] - data.append("[%s]\n" \ - "SigLevel = %s\n" \ - "Server = file://%s" \ - % (value.treename, value.getverify(), \ - os.path.join(root, SYNCREPO, value.treename))) + data.append("[%s]\n" % (value.treename)) + data.append("SigLevel = %s\n" % (value.getverify())) + if value.syncdir: + data.append("Server = file://%s" % (os.path.join(root, SYNCREPO, value.treename))) for optkey, optval in value.option.items(): data.extend(["%s = %s" % (optkey, j) for j in optval]) From patchwork Tue Jan 19 18:30:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Gregory X-Patchwork-Id: 1862 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.15) for (single-drop); Tue, 19 Jan 2021 18:31:05 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id GLKGMWglB2DsngYAK+/4rw (envelope-from ) for ; Tue, 19 Jan 2021 18:31:04 +0000 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id 5AC1C3812DB; Tue, 19 Jan 2021 18:31:04 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id CB01A2C512; Tue, 19 Jan 2021 18:30:51 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id C590C2C509 for ; Tue, 19 Jan 2021 18:30:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on luna.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FROM=0.5, LOCAL_FAKEBUSINESS=0.5,RCVD_IN_DNSWL_MED=-2.3,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=failed autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.10] [127.0.0.10] [127.0.9.2] Received: from mail.archlinux.org (mail.archlinux.org [IPv6:2a01:4f9:c010:3052::1]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 19 Jan 2021 18:30:46 +0000 (UTC) Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id 0B73F3812B1 for ; Tue, 19 Jan 2021 18:30:46 +0000 (UTC) Received: by mail-pj1-x102f.google.com with SMTP id cq1so448028pjb.4 for ; Tue, 19 Jan 2021 10:30:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t2Tg0NWcJBUz01vK7FVVnQbgcy/MIC5XncIOw1wFmYI=; b=EsBSMyzTtVftYl2AIK5KEcQcxwBj4WXHG9Q9QRK2bjns57M18PZpZqBqlgU/DRMR1W O3O2X3Wpn9eJwlzAhdj79NqY5bbqZkg93/P6tPnOc+RMTyJWgp+0mbR8w12vnyWOcEvA +DVuMcsEjVg+0Ao92O6r1/Fd5WUxwJHubykfobEaPVI98jsYN82PVemfzjPXSvdAPK2B XGtY6lMy8BL1YEv3Gi0+VeX7BqLhKWj1aBmvduC2M3vvE07hBn9LmVrgmssxq1vCAmm0 xmMEiH+2rX54HN7/L0oWoTHrKa+yzy+IpvqClws+0Qji5YPpXNSMkISHz6eQh1Y8su9n Q/7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t2Tg0NWcJBUz01vK7FVVnQbgcy/MIC5XncIOw1wFmYI=; b=l0cw3KsZNgiu/wX3qIktPdpAailF65tkepzLo2F+V/KiII0Tah3ICumTP0ILiY0oAs xVsgTo3zVzkUqBiSGtyzyhG5vfKz2MQc9m7DOsu6ExA3lgJRUrUsYq/jP1WVp+XRY3uo uSkArF+VWkRfZAnFtrsf8rbJlAqqL602oeqs1d7A7pVOvhz3oIDJJGAIz76yjGRMZWCY VsG/pmSZvOytXU1ielO6ZZkXjMJtL9FWHoPVK1JCmK9vipaTMAUxO9qi7KcWA4XjvMsk dZH7OF30gSMRVMnAiJe0EWRdVbN+Fa2AEtVPvb6psSOd2mqFbTU+IPYSuYrTtvJ4/9hk awGA== X-Gm-Message-State: AOAM5339jbERULhue7xULvmbbHe2oo7CAMX9IOev3iMLiPgQDOSBuNx5 LEP1kVRLlg2iHWHHnAhrxLCYaV1xj7w= X-Google-Smtp-Source: ABdhPJy38zAyVemrU/GMrHV0VMuc27fkgH4OcEphGxvP/VE9lL0DQdSi4tQjQw0Tyihv1ygdXidrmQ== X-Received: by 2002:a17:903:1cc:b029:de:98bb:d46d with SMTP id e12-20020a17090301ccb02900de98bbd46dmr6051736plh.54.1611081043984; Tue, 19 Jan 2021 10:30:43 -0800 (PST) Received: from localhost.localdomain (c-98-207-137-206.hsd1.ca.comcast.net. [98.207.137.206]) by smtp.gmail.com with ESMTPSA id t8sm3647685pjd.51.2021.01.19.10.30.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 10:30:43 -0800 (PST) From: Andrew Gregory To: pacman-dev@archlinux.org Cc: Andrew Gregory Date: Tue, 19 Jan 2021 10:30:09 -0800 Message-Id: <20210119183010.12831-6-andrew.gregory.8@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210119183010.12831-1-andrew.gregory.8@gmail.com> References: <20210119183010.12831-1-andrew.gregory.8@gmail.com> MIME-Version: 1.0 Subject: [pacman-dev] [PATCH 6/7] pmtest: add ability to spawn simple http servers X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=EsBSMyzT; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 5.9.250.164 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org X-Rspamd-Queue-Id: 5AC1C3812DB X-Spamd-Result: default: False [4.59 / 15.00]; HAS_REPLYTO(0.00)[pacman-dev@lists.archlinux.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,meta]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.9.250.164:c]; R_DKIM_REJECT(1.00)[gmail.com:s=20161025]; FREEMAIL_FROM(0.00)[gmail.com]; BROKEN_CONTENT_TYPE(1.50)[]; R_MISSING_CHARSET(2.50)[]; DKIM_TRACE(0.00)[gmail.com:-]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; RECEIVED_SPAMHAUS_PBL(0.00)[98.207.137.206:received]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[andrewgregory8@gmail.com,pacman-dev-bounces@lists.archlinux.org]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; NEURAL_HAM(-0.00)[-0.662]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_SEVEN(0.00)[7]; FORGED_SENDER_MAILLIST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : SPF not aligned (relaxed),none] X-Rspamd-Server: mail.archlinux.org Signed-off-by: Andrew Gregory --- test/pacman/README | 23 ++++++++++++ test/pacman/pmserve.py | 84 ++++++++++++++++++++++++++++++++++++++++++ test/pacman/pmtest.py | 28 ++++++++++++++ 3 files changed, 135 insertions(+) create mode 100644 test/pacman/pmserve.py diff --git a/test/pacman/README b/test/pacman/README index 224acd84..61a67d05 100644 --- a/test/pacman/README +++ b/test/pacman/README @@ -317,3 +317,26 @@ Example: pactest will ensure the file /etc/test.conf exists in the filesystem. +Serving Files +============= + +Tests can run a simple http server using the `add_simple_http_server` method, +which takes a dict with request paths for keys and their responses as values +and returns a url for the server. Responses may either be a simple string or +a dict with the following keys: `code`, `headers`, and `body`. If a value is +provided for the empty path it will be used as a fallback response for any +requests that do not match a path. + + url = self.add_simple_http_server({ + "/": "simple response", + "/custom": { + "headers": { "Content-Disposition": "attachment; filename=foo" }, + "body": ("Custom response. Code and any necessary headers " + "will by automatically set if not provided"), + } + "": { + "code": 404, + "headers": { "Content-Length": "14" }, + "body": "Page Not Found", + } + }) diff --git a/test/pacman/pmserve.py b/test/pacman/pmserve.py new file mode 100644 index 00000000..0ca34093 --- /dev/null +++ b/test/pacman/pmserve.py @@ -0,0 +1,84 @@ +# Copyright (c) 2020 Pacman Development Team +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import http +import http.server +import sys +import re + +class pmHTTPServer(http.server.ThreadingHTTPServer): + pass + +class pmHTTPRequestHandler(http.server.BaseHTTPRequestHandler): + """BaseHTTPRequestHandler subclass with helper methods and common setup""" + + logfile = sys.stderr + + def respond(self, response, headers={}, code=200): + self.protocol_version = "HTTP/1.1" + self.send_response(code) + for header, value in headers.items(): + self.send_header(header, value) + self.end_headers() + self.wfile.write(response) + + def parse_range_bytes(self, text): + parser = re.compile(r'^bytes=(\d+)-(\d+)?$') + if m := parser.match(text): + return map(lambda d: None if d is None else int(d), m.groups()) + else: + raise ValueError("Unrecognized Range value") + + def respond_bytes(self, response, headers={}, code=200): + headers = headers.copy() + if code == 200 and self.headers['Range']: + (start, end) = self.parse_range_bytes(self.headers['Range']) + code = 206 + response = response[start:end] + headers.setdefault('Content-Range', 'bytes */%s' % (len(response))) + headers.setdefault('Content-Type', "application/octet-stream") + headers.setdefault('Content-Length', str(len(response))) + self.respond(response, headers, code) + + def respond_string(self, response, headers={}, code=200): + headers = headers.copy() + headers.setdefault('Content-Type', 'text/plain; charset=utf-8') + self.respond_bytes(response.encode('UTF-8'), headers, code) + + def log_message(self, format, *args): + if callable(self.logfile): + self.logfile = self.logfile() + self.logfile.write("%s - - [%s] %s\n" % + (self.address_string(), + self.log_date_time_string(), + format%args)) + +class pmStringHTTPRequestHandler(pmHTTPRequestHandler): + """pmHTTPRequestHandler subclass to respond with simple string messages""" + + responses = dict() + + def do_GET(self): + response = self.responses.get(self.path, self.responses.get('')) + if response is not None: + if isinstance(response, dict): + self.respond_string( + response.get('body', ''), + headers=response.get('headers', {}), + code=response.get('code', 200)) + else: + self.respond_string(response) + else: + self.send_error(http.HTTPStatus.NOT_FOUND); diff --git a/test/pacman/pmtest.py b/test/pacman/pmtest.py index 8669f31d..7e62fc92 100644 --- a/test/pacman/pmtest.py +++ b/test/pacman/pmtest.py @@ -20,9 +20,11 @@ import shutil import stat import subprocess +import threading import time import pmrule +import pmserve import pmdb import pmfile import tap @@ -47,6 +49,8 @@ def __init__(self, name, root, config): "--hookdir", self.hookdir(), "--cachedir", self.cachedir()] + self.http_servers = [] + def __str__(self): return "name = %s\n" \ "testname = %s\n" \ @@ -285,6 +289,8 @@ def run(self, pacman): output = None vprint("\trunning: %s" % " ".join(cmd)) + self.start_http_servers() + # Change to the tmp dir before running pacman, so that local package # archives are made available more easily. time_start = time.time() @@ -293,6 +299,8 @@ def run(self, pacman): time_end = time.time() vprint("\ttime elapsed: %.2fs" % (time_end - time_start)) + self.stop_http_servers() + if output: output.close() @@ -330,3 +338,23 @@ def cachedir(self): def hookdir(self): return os.path.join(self.root, util.PM_HOOKDIR) + + def add_simple_http_server(self, responses): + logfile = lambda h: open(os.path.join(self.root, 'var/log/httpd.log'), 'a') + handler = type(self.name + 'HTTPServer', + (pmserve.pmStringHTTPRequestHandler,), + {'responses': responses, 'logfile': logfile}) + server = pmserve.pmHTTPServer(('127.0.0.1', 0), handler) + self.http_servers.append(server) + host, port = server.server_address[:2] + return 'http://%s:%d' % (host, port) + + def start_http_servers(self): + for srv in self.http_servers: + thread = threading.Thread(target=srv.serve_forever) + thread.daemon = True + thread.start() + + def stop_http_servers(self): + for srv in self.http_servers: + srv.shutdown() From patchwork Tue Jan 19 18:30:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Gregory X-Patchwork-Id: 1863 Return-Path: Delivered-To: patchwork@archlinux.org Received: from mail.archlinux.org [95.216.189.61] by patchwork.archlinux.org with IMAP (fetchmail-6.4.15) for (single-drop); Tue, 19 Jan 2021 18:31:08 +0000 (UTC) Received: from mail.archlinux.org by mail.archlinux.org with LMTP id iHLJCWwlB2D+ngYAK+/4rw (envelope-from ) for ; Tue, 19 Jan 2021 18:31:08 +0000 Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id C03B63812E8; Tue, 19 Jan 2021 18:31:07 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 001772C51A; Tue, 19 Jan 2021 18:30:51 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 160C62C512 for ; Tue, 19 Jan 2021 18:30:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on luna.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=DKIM_SIGNED=0.1, DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FREEMAIL_FROM=0.5, RCVD_IN_DNSWL_MED=-2.3,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=failed autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.10] [127.0.9.2] [127.0.0.10] Received: from mail.archlinux.org (mail.archlinux.org [95.216.189.61]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 19 Jan 2021 18:30:48 +0000 (UTC) Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.archlinux.org (Postfix) with ESMTPS id 428D13812B3 for ; Tue, 19 Jan 2021 18:30:47 +0000 (UTC) Received: by mail-pj1-x102d.google.com with SMTP id u4so438127pjn.4 for ; Tue, 19 Jan 2021 10:30:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CiXnlUjUeGKvPY0QZSRMrHc0Uu71FzTeBnfL9iXLNS8=; b=I2In1GGpAEOyjF2FCqflSRVUHvejoCnzDs7F+LZHT3b/LdamkfipUb7nsB867r3R9a M3y13cQqusC4KvxJoDwi6px7JzyooWErQ0p7RsRJNIq6Z9+clV3n0TMSIdE9/966izF5 04bwIPZ/tGcogsqthZGDlIvnEiIUVKqc0SfQYf3s/M4gqk8OtJqQGhBYqWL+4lUiYV+M hdrAFUWWqacl/QHHTke0L9bNbs1as84yECfG4XSR57hIBBvZmp8lPaoJwfF8U7bVvpRl BmzzqnLzuzJ7MEOs8j+X27v9kQMzJj6Hkugfa/2LxdlhM6deLYo41DLuQ29iE7JXwLkc F3zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CiXnlUjUeGKvPY0QZSRMrHc0Uu71FzTeBnfL9iXLNS8=; b=eBOxe4aho39J/+MB3jeAFqWh1U70JhVeHVX6jBruaQrQeEZ7tt9yjvnJuUJaZGkuj8 C+XN/vIkbsRzClKSB81mvpVZSOyr7Pvd0IIfFf8pRyv5ExyeYHXhrpfszd4q+HkK26FB w+MtBMTinOwUsI3G4bNlFC44v3rcj6P2kBam4DSjJF6sVpBf4ruGDf7RF2brDPDqlWua /vkBhw7mJH1QGaJ6Iyqrtmc89lVKQa56Wv2G7hYKD/2cX8XYumsalE4aHM7Ec7esyr/Q CuHbRD6pYxRe9AEJtqFG+klck/9aAWxPKO5gsEx8lIdxWbi+4f0wwN+VMTXX3SGTX6Z7 dYYA== X-Gm-Message-State: AOAM533wyDjZoBDTO4Yh/Dopi9P9C3ACXPBfgCRWUqjapIdun5ih1Deo M+2DfbvytPOtdh5xpz1jIIsDudjerUg= X-Google-Smtp-Source: ABdhPJxI4N7vz0zimNzQrLScRAlrtfE/43NOMqLcv3q4lgCEONos67H/xHQCrcs5NoLhibE8uKLQvw== X-Received: by 2002:a17:902:9f89:b029:dc:3032:e439 with SMTP id g9-20020a1709029f89b02900dc3032e439mr6022068plq.35.1611081045276; Tue, 19 Jan 2021 10:30:45 -0800 (PST) Received: from localhost.localdomain (c-98-207-137-206.hsd1.ca.comcast.net. [98.207.137.206]) by smtp.gmail.com with ESMTPSA id t8sm3647685pjd.51.2021.01.19.10.30.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jan 2021 10:30:44 -0800 (PST) From: Andrew Gregory To: pacman-dev@archlinux.org Cc: Andrew Gregory Date: Tue, 19 Jan 2021 10:30:10 -0800 Message-Id: <20210119183010.12831-7-andrew.gregory.8@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210119183010.12831-1-andrew.gregory.8@gmail.com> References: <20210119183010.12831-1-andrew.gregory.8@gmail.com> MIME-Version: 1.0 Subject: [pacman-dev] [PATCH 7/7] add --upgrade download tests X-BeenThere: pacman-dev@lists.archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@lists.archlinux.org Sender: "pacman-dev" Authentication-Results: mail.archlinux.org; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=I2In1GGp; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (mail.archlinux.org: domain of pacman-dev-bounces@lists.archlinux.org designates 5.9.250.164 as permitted sender) smtp.mailfrom=pacman-dev-bounces@lists.archlinux.org X-Rspamd-Queue-Id: C03B63812E8 X-Spamd-Result: default: False [4.59 / 15.00]; HAS_REPLYTO(0.00)[pacman-dev@lists.archlinux.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,meta]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.9.250.164:c]; R_DKIM_REJECT(1.00)[gmail.com:s=20161025]; FREEMAIL_FROM(0.00)[gmail.com]; BROKEN_CONTENT_TYPE(1.50)[]; R_MISSING_CHARSET(2.50)[]; NEURAL_SPAM(0.00)[0.087]; DKIM_TRACE(0.00)[gmail.com:-]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; RECEIVED_SPAMHAUS_PBL(0.00)[98.207.137.206:received]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:5.9.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[andrewgregory8@gmail.com,pacman-dev-bounces@lists.archlinux.org]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_SEVEN(0.00)[7]; FORGED_SENDER_MAILLIST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : SPF not aligned (relaxed),none] X-Rspamd-Server: mail.archlinux.org Signed-off-by: Andrew Gregory --- test/pacman/meson.build | 3 + test/pacman/tests/upgrade-download-404.py | 10 +++ ...rade-download-pkg-and-sig-with-filename.py | 61 ++++++++++++++++++ ...e-download-pkg-and-sig-without-filename.py | 63 +++++++++++++++++++ 4 files changed, 137 insertions(+) create mode 100644 test/pacman/tests/upgrade-download-404.py create mode 100644 test/pacman/tests/upgrade-download-pkg-and-sig-with-filename.py create mode 100644 test/pacman/tests/upgrade-download-pkg-and-sig-without-filename.py diff --git a/test/pacman/meson.build b/test/pacman/meson.build index fd0dbf90..52ff9b9a 100644 --- a/test/pacman/meson.build +++ b/test/pacman/meson.build @@ -328,6 +328,9 @@ pacman_tests = [ 'tests/upgrade090.py', 'tests/upgrade100.py', 'tests/xfercommand001.py', + 'tests/upgrade-download-404.py', + 'tests/upgrade-download-pkg-and-sig-with-filename.py', + 'tests/upgrade-download-pkg-and-sig-without-filename.py', ] xfail_tests = { diff --git a/test/pacman/tests/upgrade-download-404.py b/test/pacman/tests/upgrade-download-404.py new file mode 100644 index 00000000..ed5ad2b4 --- /dev/null +++ b/test/pacman/tests/upgrade-download-404.py @@ -0,0 +1,10 @@ +self.description = 'download a remote package with -U' +self.require_capability("curl") + +url = self.add_simple_http_server({}) + +self.args = '-Uw {url}/foo.pkg'.format(url=url) + +self.addrule('!PACMAN_RETCODE=0') +self.addrule('!CACHE_FEXISTS=foo.pkg') +self.addrule('!CACHE_FEXISTS=foo.pkg.sig') diff --git a/test/pacman/tests/upgrade-download-pkg-and-sig-with-filename.py b/test/pacman/tests/upgrade-download-pkg-and-sig-with-filename.py new file mode 100644 index 00000000..ac5072c9 --- /dev/null +++ b/test/pacman/tests/upgrade-download-pkg-and-sig-with-filename.py @@ -0,0 +1,61 @@ +self.description = 'download remote packages with -U with a URL filename' +self.require_capability("gpg") +self.require_capability("curl") + +url = self.add_simple_http_server({ + # simple + '/simple.pkg': 'simple', + '/simple.pkg.sig': { + 'headers': { 'Content-Disposition': 'attachment; filename="simple.sig-alt' }, + 'body': 'simple.sig', + }, + + # content-disposition filename + '/cd.pkg': { + 'headers': { 'Content-Disposition': 'attachment; filename="cd-alt.pkg"' }, + 'body': 'cd' + }, + '/cd.pkg.sig': 'cd.sig', + + # redirect + '/redir.pkg': { 'code': 303, 'headers': { 'Location': '/redir-dest.pkg' } }, + '/redir-dest.pkg': 'redir-dest', + '/redir-dest.pkg.sig': 'redir-dest.sig', + + # content-disposition and redirect + '/cd-redir.pkg': { 'code': 303, 'headers': { 'Location': '/cd-redir-dest.pkg' } }, + '/cd-redir-dest.pkg': { + 'headers': { 'Content-Disposition': 'attachment; filename="cd-redir-dest-alt.pkg"' }, + 'body': 'cd-redir-dest' + }, + '/cd-redir-dest.pkg.sig': 'cd-redir-dest.sig', + + # TODO: absolutely terrible hack to prevent pacman from attempting to + # validate packages, which causes failure under --valgrind thanks to + # a memory leak in gpgme that is too general for inclusion in valgrind.supp + '/404': { 'code': 404 }, + + '': 'fallback', +}) + +self.args = '-Uw {url}/simple.pkg {url}/cd.pkg {url}/redir.pkg {url}/cd-redir.pkg {url}/404'.format(url=url) + +# packages/sigs are not valid, error is expected +self.addrule('!PACMAN_RETCODE=0') + +self.addrule('CACHE_FCONTENTS=simple.pkg|simple') +self.addrule('CACHE_FCONTENTS=simple.pkg.sig|simple.sig') + +self.addrule('!CACHE_FEXISTS=cd.pkg') +self.addrule('!CACHE_FEXISTS=cd.pkg.sig') +self.addrule('CACHE_FCONTENTS=cd-alt.pkg|cd') +self.addrule('CACHE_FCONTENTS=cd-alt.pkg.sig|cd.sig') + +self.addrule('!CACHE_FEXISTS=redir.pkg') +self.addrule('CACHE_FCONTENTS=redir-dest.pkg|redir-dest') +self.addrule('CACHE_FCONTENTS=redir-dest.pkg.sig|redir-dest.sig') + +self.addrule('!CACHE_FEXISTS=cd-redir.pkg') +self.addrule('!CACHE_FEXISTS=cd-redir-dest.pkg') +self.addrule('CACHE_FCONTENTS=cd-redir-dest-alt.pkg|cd-redir-dest') +self.addrule('CACHE_FCONTENTS=cd-redir-dest-alt.pkg.sig|cd-redir-dest.sig') diff --git a/test/pacman/tests/upgrade-download-pkg-and-sig-without-filename.py b/test/pacman/tests/upgrade-download-pkg-and-sig-without-filename.py new file mode 100644 index 00000000..651fb3f5 --- /dev/null +++ b/test/pacman/tests/upgrade-download-pkg-and-sig-without-filename.py @@ -0,0 +1,63 @@ +self.description = 'download remote packages with -U without a URL filename' +self.require_capability("gpg") +self.require_capability("curl") + +url = self.add_simple_http_server({ + # simple + '/simple.pkg/': 'simple', + '/simple.pkg/.sig': 'simple.sig', + + # content-disposition filename + '/cd.pkg/': { + 'headers': { 'Content-Disposition': 'attachment; filename="cd-alt.pkg"' }, + 'body': 'cd' + }, + '/cd.pkg/.sig': { + 'headers': { 'Content-Disposition': 'attachment; filename="cd-alt-bad.pkg.sig"' }, + 'body': 'cd.sig' + }, + + # redirect + '/redir.pkg/': { 'code': 303, 'headers': { 'Location': '/redir-dest.pkg' } }, + '/redir-dest.pkg': 'redir-dest', + '/redir-dest.pkg.sig': 'redir-dest.sig', + + # content-disposition and redirect + '/cd-redir.pkg/': { 'code': 303, 'headers': { 'Location': '/cd-redir-dest.pkg' } }, + '/cd-redir-dest.pkg': { + 'headers': { 'Content-Disposition': 'attachment; filename="cd-redir-dest-alt.pkg"' }, + 'body': 'cd-redir-dest' + }, + '/cd-redir-dest.pkg.sig': 'cd-redir-dest.sig', + + # TODO: absolutely terrible hack to prevent pacman from attempting to + # validate packages, which causes failure under --valgrind thanks to + # a memory leak in gpgme that is too general for inclusion in valgrind.supp + '/404': { 'code': 404 }, + + '': 'fallback', +}) + +self.args = '-Uw {url}/simple.pkg/ {url}/cd.pkg/ {url}/redir.pkg/ {url}/cd-redir.pkg/ {url}/404'.format(url=url) + +# packages/sigs are not valid, error is expected +self.addrule('!PACMAN_RETCODE=0') + +# TODO: use a predictable file name +#self.addrule('CACHE_FCONTENTS=simple.pkg|simple') +#self.addrule('CACHE_FCONTENTS=simple.pkg.sig|simple.sig') + +self.addrule('!CACHE_FEXISTS=cd.pkg') +self.addrule('CACHE_FCONTENTS=cd-alt.pkg|cd') +self.addrule('CACHE_FCONTENTS=cd-alt.pkg.sig|cd.sig') + +self.addrule('!CACHE_FEXISTS=redir.pkg') +self.addrule('CACHE_FCONTENTS=redir-dest.pkg|redir-dest') +self.addrule('CACHE_FCONTENTS=redir-dest.pkg.sig|redir-dest.sig') + +self.addrule('!CACHE_FEXISTS=cd-redir.pkg') +self.addrule('!CACHE_FEXISTS=cd-redir-dest.pkg') +self.addrule('CACHE_FCONTENTS=cd-redir-dest-alt.pkg|cd-redir-dest') +self.addrule('CACHE_FCONTENTS=cd-redir-dest-alt.pkg.sig|cd-redir-dest.sig') + +self.addrule('!CACHE_FEXISTS=.sig')