From patchwork Fri Jul 3 03:10:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Morris X-Patchwork-Id: 1702 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id 32F0119B3EB0B for ; Fri, 3 Jul 2020 03:11:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=DKIM_ADSP_CUSTOM_MED=0.001, DKIM_INVALID=1,DKIM_SIGNED=0.1,FREEMAIL_FROM=0.5,MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_MSPIKE_H4=0.001,RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001,T_DMARC_POLICY_NONE=0.01,T_DMARC_SIMPLE_DKIM=0.01 autolearn=ham autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.19] [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Fri, 3 Jul 2020 03:11:13 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 196FB1D357B27F; Fri, 3 Jul 2020 03:11:08 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: luna) by orion.archlinux.org (Postfix) with ESMTPSA id 1DE2C1D357B274; Fri, 3 Jul 2020 03:11:07 +0000 (UTC) Authentication-Results: orion.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lhqSyGqB Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 0D9D429CBE; Fri, 3 Jul 2020 03:11:07 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lhqSyGqB Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id C4FAE29CBA for ; Fri, 3 Jul 2020 03:11:02 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by luna.archlinux.org (Postfix) with ESMTPS for ; Fri, 3 Jul 2020 03:11:02 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id F2CBD1D357B272 for ; Fri, 3 Jul 2020 03:10:58 +0000 (UTC) Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS for ; Fri, 3 Jul 2020 03:10:58 +0000 (UTC) Received: by mail-pg1-x534.google.com with SMTP id e8so14451365pgc.5 for ; Thu, 02 Jul 2020 20:10:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fRml5VcEwL6HYJjIiKv4uf0eoiDjSGG5XGu5y+nsrxw=; b=lhqSyGqBNL1xIe5eSG5pTZuGU/7xQ71jqJqKrpDsABo0Z2E6r0eNtJ0aZBI3GFN1RE JqjEWjqu8AIFG0CPhbDwcAbyPH8Q9GE3czc4q7nkgftUNwgKIbboZeoqk53HDBmpLhKT czUvBv4B+LoFGGNP6165dbg0jqpKxRgvoSZnK9DKTTALIAhviAhtlUWSWeSsXeXRHvlK OCoEUu0SzWh6YSVnf0I77c6g0uCCL27pOv9MreQNSFniO4BPHkhsWIcAHZ3PzM+nN4Qc S4GxlUFFp8ONSfSu8VN+nUs7tImIVd8ErM8PN0NdUCkxxHngRmfKnzdJk9F2cGm74v9i ZXow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fRml5VcEwL6HYJjIiKv4uf0eoiDjSGG5XGu5y+nsrxw=; b=r9twFZPtYDAS05tB6PZv1+GAOWBZ7UdaVq0diAFasbRsDl9cFbPocrHKmpzWkvayUz dxdaadE5aMSJivyG0Ya0F2cSxJN7TOqvlnNmXm19J5d2TpHzwRyJivG3Iz/atpuz3z45 SpGiHxnMPslneLpWe4LU9Np1nR582AeTOj6dYfb6A/p7qcR4hh1wlAKQK2NMytNoHHbY HozsqeB7/z5AFJpWxIjK2cse/9C0A9nTgwO2sA0VFXyRCj4OIa2gcrUWrdELD/UV3ao2 v7HE0FTGb2fqBgc3mYJ2q1hNerEBfViJ4Ko3LVW8otd+EAksbjFfuOS4z2Kruc8BRu5y 5W3A== X-Gm-Message-State: AOAM533Ft1LYJn6Af+b3QBUsvM2+f7GkrL1+uAtQF0YDGs4HQVvjb1cK Mb9AygMNWMW402eYI8CYI8N5m7DDUBs= X-Google-Smtp-Source: ABdhPJxwaLFckiY+Zrno14OrAVVsA5w4rHIiY0ea/hAXi2tvLWvXdIL1xw65vCMo9qvg8bywoclgjw== X-Received: by 2002:a62:2c54:: with SMTP id s81mr26183603pfs.252.1593745855843; Thu, 02 Jul 2020 20:10:55 -0700 (PDT) Received: from localhost.localdomain ([2600:1:9a0e:ad89:9544:bd74:2d8:3df1]) by smtp.gmail.com with ESMTPSA id c2sm9764263pgk.77.2020.07.02.20.10.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jul 2020 20:10:55 -0700 (PDT) From: Kevin Morris To: aur-dev@archlinux.org Subject: [PATCH] Bump RPC API Version 6 Date: Thu, 2 Jul 2020 20:10:40 -0700 Message-Id: <20200703031040.27579-1-kevr.gtalk@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200630233601.32314-1-kevr.gtalk@gmail.com> References: <20200630233601.32314-1-kevr.gtalk@gmail.com> MIME-Version: 1.0 X-BeenThere: aur-dev@archlinux.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Arch User Repository \(AUR\) Development" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: aur-dev-bounces@archlinux.org Sender: "aur-dev" API Version 6 modifies `type=search` functionality: Split `arg` into keywords separated by spaces (quoted keywords are preserved). Search for packages containing the literal keyword `blah blah` AND `haha`: https://aur.archlinux.org/rpc/?v=6&type=search&arg="blah blah"%20haha Search for packages containing the literal keyword `abc 123`: https://aur.archlinux.org/rpc/?v=6&type=search&arg="abc 123" The following example searches for packages that contain `blah` AND `abc`: https://aur.archlinux.org/rpc/?v=6&type=search&arg=blah%20abc The legacy method still searches for packages that contain 'blah abc': https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc API Version 6 is only considered during a `search` of `name` or `name-desc`. Note: This change was written as a solution to https://bugs.archlinux.org/task/49133. PS: + Some spacing issues fixed in comments. Signed-off-by: Kevin Morris --- doc/rpc.txt | 4 +++ web/lib/aurjson.class.php | 66 ++++++++++++++++++++++++++++++++++----- 2 files changed, 62 insertions(+), 8 deletions(-) diff --git a/doc/rpc.txt b/doc/rpc.txt index 3148ebea..b0f5c4e1 100644 --- a/doc/rpc.txt +++ b/doc/rpc.txt @@ -39,6 +39,10 @@ Examples `/rpc/?v=5&type=search&by=makedepends&arg=boost` `search` with callback:: `/rpc/?v=5&type=search&arg=foobar&callback=jsonp1192244621103` +`search` with API Version 6 for packages containing `cookie` AND `milk`:: + `/rpc/?v=6&type=search&arg=cookie%20milk` +`search` with API Version 6 for packages containing `cookie milk`:: + `/rpc/?v=6&type=search&arg="cookie milk"` `info`:: `/rpc/?v=5&type=info&arg[]=foobar` `info` with multiple packages:: diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 0ac586fe..a3224e57 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -80,7 +80,7 @@ class AurJSON { if (isset($http_data['v'])) { $this->version = intval($http_data['v']); } - if ($this->version < 1 || $this->version > 5) { + if ($this->version < 1 || $this->version > 6) { return $this->json_error('Invalid version specified.'); } @@ -140,7 +140,7 @@ class AurJSON { } /* - * Check if an IP needs to be rate limited. + * Check if an IP needs to be rate limited. * * @param $ip IP of the current request * @@ -192,7 +192,7 @@ class AurJSON { $value = get_cache_value('ratelimit-ws:' . $ip, $status); if (!$status || ($status && $value < $deletion_time)) { if (set_cache_value('ratelimit-ws:' . $ip, $time, $window_length) && - set_cache_value('ratelimit:' . $ip, 1, $window_length)) { + set_cache_value('ratelimit:' . $ip, 1, $window_length)) { return; } } else { @@ -370,7 +370,7 @@ class AurJSON { } elseif ($this->version >= 2) { if ($this->version == 2 || $this->version == 3) { $fields = implode(',', self::$fields_v2); - } else if ($this->version == 4 || $this->version == 5) { + } else if ($this->version >= 4 && $this->version <= 6) { $fields = implode(',', self::$fields_v4); } $query = "SELECT {$fields} " . @@ -472,6 +472,33 @@ class AurJSON { return array('ids' => $id_args, 'names' => $name_args); } + /* + * Prepare a WHERE statement for each keyword: append $func($keyword) + * separated by $delim. Each keyword is sanitized as wildcards before + * it's passed to $func. + * + * @param $delim Delimiter to use in the middle of two keywords. + * @param $keywords Array of keywords to prepare. + * @param $func A function that returns a string. This value is concatenated. + * + * @return A WHERE condition statement of keywords separated by $delim. + */ + private function join_where($delim, $keywords, $func) { + // Applied to each item to concatenate our entire statement. + $reduce_func = function($carry, $item) use(&$func) { + array_push($carry, $func($item)); + return $carry; + }; + + // Manual array_reduce with a local lambda. + $acc = array(); // Initial + foreach ($keywords as &$keyword) { + $acc += $reduce_func($acc, $keyword); + } + + return join(" $delim ", $acc); + } + /* * Performs a fulltext mysql search of the package database. * @@ -492,13 +519,36 @@ class AurJSON { if (strlen($keyword_string) < 2) { return $this->json_error('Query arg too small.'); } - $keyword_string = $this->dbh->quote("%" . addcslashes($keyword_string, '%_') . "%"); if ($search_by === 'name') { - $where_condition = "(Packages.Name LIKE $keyword_string)"; + if ($this->version >= 6) { + $keywords = preg_split('/"([^"]*)"|\h+/', $keyword_string, -1, + PREG_SPLIT_NO_EMPTY|PREG_SPLIT_DELIM_CAPTURE); + $func = function($keyword) { + $str = $this->dbh->quote("%" . addcslashes($keyword, '%_') . "%"); + return "(Packages.Name LIKE $str)"; + }; + $where_condition = $this->join_where("AND", $keywords, $func); + } else { + $keyword_string = $this->dbh->quote( + "%" . addcslashes($keyword_string, '%_') . "%"); + $where_condition = "(Packages.Name LIKE $keyword_string)"; + } } else if ($search_by === 'name-desc') { - $where_condition = "(Packages.Name LIKE $keyword_string OR "; - $where_condition .= "Description LIKE $keyword_string)"; + if ($this->version >= 6) { + $keywords = preg_split('/"([^"]*)"|\h+/', $keyword_string, -1, + PREG_SPLIT_NO_EMPTY|PREG_SPLIT_DELIM_CAPTURE); + $func = function($keyword) { + $str = $this->dbh->quote("%" . addcslashes($keyword, '%_') . "%"); + return "(Packages.Name LIKE $str OR Description LIKE $str)"; + }; + $where_condition = $this->join_where("AND", $keywords, $func); + } else { + $keyword_string = $this->dbh->quote( + "%" . addcslashes($keyword_string, '%_') . "%"); + $where_condition = "(Packages.Name LIKE $keyword_string "; + $where_condition .= "OR Description LIKE $keyword_string)"; + } } } else if ($search_by === 'maintainer') { if (empty($keyword_string)) {