From patchwork Tue Jun 30 23:36:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Morris X-Patchwork-Id: 1701 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id EEE2319AA3253 for ; Tue, 30 Jun 2020 23:36:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=DKIM_ADSP_CUSTOM_MED=0.001, DKIM_INVALID=1,DKIM_SIGNED=0.1,FREEMAIL_FROM=0.5,LOCAL_CHARITY=1.5, MAILING_LIST_MULTI=-1,RCVD_IN_DNSWL_MED=-2.3,SPF_HELO_NONE=0.001, T_DMARC_POLICY_NONE=0.01,T_DMARC_SIMPLE_DKIM=0.01 autolearn=ham autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Tue, 30 Jun 2020 23:36:31 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id A61811D356B313; Tue, 30 Jun 2020 23:36:23 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [5.9.250.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: luna) by orion.archlinux.org (Postfix) with ESMTPSA id 64ADC1D356B30E; Tue, 30 Jun 2020 23:36:23 +0000 (UTC) Authentication-Results: orion.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UASV6y5n Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 328B529CB8; Tue, 30 Jun 2020 23:36:23 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UASV6y5n Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id E865729CB4 for ; Tue, 30 Jun 2020 23:36:18 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Tue, 30 Jun 2020 23:36:18 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 407F01D356B309 for ; Tue, 30 Jun 2020 23:36:15 +0000 (UTC) Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS for ; Tue, 30 Jun 2020 23:36:15 +0000 (UTC) Received: by mail-pl1-x62a.google.com with SMTP id f2so9129898plr.8 for ; Tue, 30 Jun 2020 16:36:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=p2nlDIyATmge4guCk0Shg93quacy9awLEE89w2EIcjs=; b=UASV6y5nXpYgSe3rikAfyp8oFYFvO4/6oCllU+tjmPH+CXG2w9C5LYi3CYX9uLsFxf cZE8lgEYFwR9lo/lAFAsTyJi/C0UeNn2ge3RVx4idtykFMu0p6UjyH0Vog97wkWH/HRW 15oIxhyYcaBS+V38j7ZEeqpx/TXikGTARD06gSuE4nnsqI+TZsfO2BXbs/jeP0oCE9d3 vwPc7Kr9SEYbZVehiZYIovIo4nIqSzcWhlS1B6TPd2K5LY35MSFuK3j4etVxnmVZyHhM PDpWSi886azq+MK6r9GTDNEG1WzUis2HVfUiWwflknlNGHzn+sYIoE8n6U5ELsbsEfcv nWnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=p2nlDIyATmge4guCk0Shg93quacy9awLEE89w2EIcjs=; b=T4yYFA8wUtIkvH4KvEiaNULd8kim/PeiWW+UvnRWatMZgHuEOTjOJG+wtMhma3hTm/ 5mfL46nTqHz6A/m6plOrb+dVbM9Zn2D0vcIHzhVvNmjIGrl7JwetsZO+w/Bkf9cO9ki6 IvYyKEfI1o0VrJGghabPe5iZHste9EH2Boybmrj3dobQbkW81cYhoDaXgfnE+UuvyiEi 0RSZkyTkuKoB8OwKA7knuOIra2JCRHheJa0uQ1NrCniHWVBKc+TRXI04OirxyK2SqjEe fNzvM5rRRJtbyK7Ij+PPCYmNGMqdbxsjJ3g9BChUtgbLLQLFTU6n6VVC2/QIVNEYlZve sY/A== X-Gm-Message-State: AOAM533vgrBmBE4XoH7967jUbxB12Gcegig4lXwno2Tul5VaWTJxZ7S5 TSNOmhgF0ojyDGNG9UDoFhAVi7Ksc8U= X-Google-Smtp-Source: ABdhPJxN6McTfoO3znGJejaJmNmyiyxLntLLW5Vjvk3T7mdlU171Ni0W4pp+Bp42w8aKZ4motDrTNA== X-Received: by 2002:a17:902:728c:: with SMTP id d12mr20022513pll.155.1593560171975; Tue, 30 Jun 2020 16:36:11 -0700 (PDT) Received: from localhost.localdomain ([2600:1:9a6e:23cc:d17c:f7c2:8967:889e]) by smtp.gmail.com with ESMTPSA id g4sm3558046pfi.68.2020.06.30.16.36.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 16:36:11 -0700 (PDT) From: Kevin Morris To: aur-dev@archlinux.org Subject: [PATCH] Add search_type url parameter Date: Tue, 30 Jun 2020 16:36:01 -0700 Message-Id: <20200630233601.32314-1-kevr.gtalk@gmail.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-BeenThere: aur-dev@archlinux.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Arch User Repository \(AUR\) Development" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: aur-dev-bounces@archlinux.org Sender: "aur-dev" This commit introduces new functionality: When `search_type` is `web`, search behavior matches aurweb's HTML search page. New parameters: `search_type` Valid search_type values: `rpc` (default), `web` The `rpc` search type uses the existing legacy search method. The `web` search type clones the web-based aurweb search page's behavior as closely as possible (see note at the bottom of this message). The following example searches for packages that contain `blah` AND `abc`: https://aur.archlinux.org/rpc/?v=5&type=search&search_type=web&arg=blah%20abc The legacy method still searches for packages that contain 'blah abc': https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc https://aur.archlinux.org/rpc/?v=5&type=search&search_type=rpc&arg=blah%20abc An invalid `search_type` returns a json error message about it being unsupported. Additionally, `search_type` is only parsed and considered during a search of type `name` or `name-desc`. Note: This change was written as a solution to https://bugs.archlinux.org/task/49133. PS: + Some spacing issues fixed in comments. Signed-off-by: Kevin Morris --- doc/rpc.txt | 7 ++++ web/lib/aurjson.class.php | 75 +++++++++++++++++++++++++++++++++++---- 2 files changed, 76 insertions(+), 6 deletions(-) diff --git a/doc/rpc.txt b/doc/rpc.txt index 3148ebea..eb97547d 100644 --- a/doc/rpc.txt +++ b/doc/rpc.txt @@ -21,6 +21,11 @@ The _by_ parameter can be skipped and defaults to `name-desc`. If a maintainer search is performed and the search argument is left empty, a list of orphan packages is returned. +The _search_type_ parameter can be skipped and defaults to `rpc`. + +_search_type_ is only ever used during a `name` or `name-desc` search and +supports the following values: `rpc`, `web`. + Package Details --------------- @@ -39,6 +44,8 @@ Examples `/rpc/?v=5&type=search&by=makedepends&arg=boost` `search` with callback:: `/rpc/?v=5&type=search&arg=foobar&callback=jsonp1192244621103` +`search` with `web` _search_type_ for packages containing `cookie` AND `milk`:: + `/rpc/?v=5&type=search&search_type=web&arg=cookie%20milk` `info`:: `/rpc/?v=5&type=info&arg[]=foobar` `info` with multiple packages:: diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 0ac586fe..0e94b578 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -23,6 +23,9 @@ class AurJSON { private static $exposed_depfields = array( 'depends', 'makedepends', 'checkdepends', 'optdepends' ); + private static $exposed_search_types = array( + 'rpc', 'web' + ); private static $fields_v1 = array( 'Packages.ID', 'Packages.Name', 'PackageBases.ID AS PackageBaseID', @@ -140,7 +143,7 @@ class AurJSON { } /* - * Check if an IP needs to be rate limited. + * Check if an IP needs to be rate limited. * * @param $ip IP of the current request * @@ -192,7 +195,7 @@ class AurJSON { $value = get_cache_value('ratelimit-ws:' . $ip, $status); if (!$status || ($status && $value < $deletion_time)) { if (set_cache_value('ratelimit-ws:' . $ip, $time, $window_length) && - set_cache_value('ratelimit:' . $ip, 1, $window_length)) { + set_cache_value('ratelimit:' . $ip, 1, $window_length)) { return; } } else { @@ -472,6 +475,33 @@ class AurJSON { return array('ids' => $id_args, 'names' => $name_args); } + /* + * Prepare a WHERE statement for each keyword: append $func($keyword) + * separated by $delim. Each keyword is sanitized as wildcards before + * it's passed to $func. + * + * @param $delim Delimiter to use in the middle of two keywords. + * @param $keywords Array of keywords to prepare. + * @param $func A function that returns a string. This value is concatenated. + * + * @return A WHERE condition statement of keywords separated by $delim. + */ + private function join_where($delim, $keywords, $func) { + // Applied to each item to concatenate our entire statement. + $reduce_func = function($carry, $item) use(&$func) { + array_push($carry, $func($item)); + return $carry; + }; + + // Manual array_reduce with a local lambda. + $acc = array(); // Initial + foreach ($keywords as &$keyword) { + $acc += $reduce_func($acc, $keyword); + } + + return join(" $delim ", $acc); + } + /* * Performs a fulltext mysql search of the package database. * @@ -480,6 +510,7 @@ class AurJSON { * @return mixed Returns an array of package matches. */ private function search($http_data) { + $keyword_string = $http_data['arg']; if (isset($http_data['by'])) { @@ -488,17 +519,49 @@ class AurJSON { $search_by = 'name-desc'; } + if (isset($http_data['search_type'])) { + $search_type = $http_data['search_type']; + } else { + // Default search_type: rpc + $search_type = 'rpc'; + } + + if (!in_array($search_type, self::$exposed_search_types)) { + return $this->json_error('Unsupported search type.'); + } + if ($search_by === 'name' || $search_by === 'name-desc') { if (strlen($keyword_string) < 2) { return $this->json_error('Query arg too small.'); } - $keyword_string = $this->dbh->quote("%" . addcslashes($keyword_string, '%_') . "%"); if ($search_by === 'name') { - $where_condition = "(Packages.Name LIKE $keyword_string)"; + if ($search_type === 'rpc') { + $keyword_string = $this->dbh->quote( + "%" . addcslashes($keyword_string, '%_') . "%"); + $where_condition = "(Packages.Name LIKE $keyword_string)"; + } else { + $keywords = explode(' ', $keyword_string); + $func = function($keyword) { + $str = $this->dbh->quote("%" . addcslashes($keyword, '%_') . "%"); + return "(Packages.Name LIKE $str)"; + }; + $where_condition = $this->join_where("AND", $keywords, $func); + } } else if ($search_by === 'name-desc') { - $where_condition = "(Packages.Name LIKE $keyword_string OR "; - $where_condition .= "Description LIKE $keyword_string)"; + if ($search_type === 'rpc') { + $keyword_string = $this->dbh->quote( + "%" . addcslashes($keyword_string, '%_') . "%"); + $where_condition = "(Packages.Name LIKE $keyword_string "; + $where_condition .= "OR Description LIKE $keyword_string)"; + } else { + $keywords = explode(' ', $keyword_string); + $func = function($keyword) { + $str = $this->dbh->quote("%" . addcslashes($keyword, '%_') . "%"); + return "(Packages.Name LIKE $str OR Description LIKE $str)"; + }; + $where_condition = $this->join_where("AND", $keywords, $func); + } } } else if ($search_by === 'maintainer') { if (empty($keyword_string)) {