From patchwork Sun Jul 5 05:10:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Morris X-Patchwork-Id: 1708 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id D1B9619BDB38B for ; Sun, 5 Jul 2020 05:10:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=DKIM_ADSP_CUSTOM_MED=0.001, DKIM_INVALID=1,DKIM_SIGNED=0.1,FREEMAIL_FROM=0.5,MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_MED=-2.3,SPF_HELO_NONE=0.001,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=ham autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [IPv6:2a01:4f8:160:6087::1]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Sun, 5 Jul 2020 05:10:57 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id C2CE61D3585597; Sun, 5 Jul 2020 05:10:48 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: luna) by orion.archlinux.org (Postfix) with ESMTPSA id 4BB7B1D3585591; Sun, 5 Jul 2020 05:10:48 +0000 (UTC) Authentication-Results: orion.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=o887zUbD Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id 371C8210F6; Sun, 5 Jul 2020 05:10:48 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=o887zUbD Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id E02F320F85 for ; Sun, 5 Jul 2020 05:10:44 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Sun, 5 Jul 2020 05:10:44 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 3597F1D358558F for ; Sun, 5 Jul 2020 05:10:38 +0000 (UTC) Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS for ; Sun, 5 Jul 2020 05:10:38 +0000 (UTC) Received: by mail-pg1-x544.google.com with SMTP id e8so16906263pgc.5 for ; Sat, 04 Jul 2020 22:10:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ODFiqK8cmRuoYWgmbWzC5zA9vFCYe67JbZDnN+R9rvM=; b=o887zUbDe/ns9E4af8Qb/H9xTCa5cLQC1c4kAONubtKQOQdM3T0KtN+scT9nP+93mh 5Zl34DHF1OH9K9nYCbT85/Ubao0EPZ4bkcb7dS3MmqqsQJ0esuq53u9E1ib15d66FkFw PyJAQRWOU8qw0CUYrD0lYu/ULgVUPx7tel17KWuB0QJ/0nKForb5GEt1lEBJXzRHGtAO /K4CyWxo8/BiRTHfebIAw+Yg5QgidZt3i67Aumyzh9FZQMbj8dYEl3nsqcRW9XBusLwa 5lYNceFbjQFBKUf18XjPS8z+mq72iLY8vng/lTt3eRXCrLnPhVnMh6TEpc75brjtHT6R EqGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ODFiqK8cmRuoYWgmbWzC5zA9vFCYe67JbZDnN+R9rvM=; b=NSMH1r8Wj44XJHbWGZ8nV0Yw8i5auqhiNN8YDai9gDI0Y5r0s/qZA+JbqZsbm1UitT cwD3cxpt+4wY5VvMqdbjmf0BTh2AvCfRSDpAg7ASyZnAW3bRTaZosdgQQC61vrZY/Mdr GUPTy0xHvujC8oe4jxvEdIkSNb5giLcKILfJ9n40rjeNJ87MmX60Ukn21m0GefvUH3uO ULCqfwEfisUa+KOpH85btAWA8mR+X9VN+loYt+I2Vk2TeLioHoZdK90Fs3c7Z7keo8aU crCIhzW6IiaxD4RVBbBqe9MmFpLWCLCz1CZ4sQGG5mGF38uanY+JUCifRlNVSgtGjp87 hehQ== X-Gm-Message-State: AOAM531cxE+MXBi33ApgCrS5WcmpnHAD4K77PMxFfzoxD5Q+l252dt3R 7e3cOPJUkG3Fp62DGKviFYu7YSQ1vF4= X-Google-Smtp-Source: ABdhPJzBa0k1/Pvo1RKC16Rkazkp9bTnXnGMVhnyngUsTtRtKDrdWKXjTGKs6T9+/1Xoz6qFskX44w== X-Received: by 2002:a63:7357:: with SMTP id d23mr35333292pgn.393.1593925835780; Sat, 04 Jul 2020 22:10:35 -0700 (PDT) Received: from localhost.localdomain ([2600:1:9a62:7913:78d3:8a2f:514b:d2b7]) by smtp.gmail.com with ESMTPSA id c23sm15041058pfo.32.2020.07.04.22.10.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 04 Jul 2020 22:10:34 -0700 (PDT) From: Kevin Morris To: aur-dev@archlinux.org Subject: [PATCH] Support conjunctive keyword search in RPC interface Date: Sat, 4 Jul 2020 22:10:07 -0700 Message-Id: <20200705051007.30145-1-kevr.gtalk@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <159391176976.775568.9840552971139958718@typhoon> References: <159391176976.775568.9840552971139958718@typhoon> MIME-Version: 1.0 X-BeenThere: aur-dev@archlinux.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Arch User Repository \(AUR\) Development" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: aur-dev-bounces@archlinux.org Sender: "aur-dev" Newly supported API Version 6 modifies `type=search` functionality; it now behaves the same as `name` or `name-desc` search through the https://aur.archlinux.org/packages/ search page. Search for packages containing the literal keyword `blah blah` AND `haha`: https://aur.archlinux.org/rpc/?v=6&type=search&arg="blah blah"%20haha Search for packages containing the literal keyword `abc 123`: https://aur.archlinux.org/rpc/?v=6&type=search&arg="abc 123" The following example searches for packages that contain `blah` AND `abc`: https://aur.archlinux.org/rpc/?v=6&type=search&arg=blah%20abc The legacy method still searches for packages that contain `blah abc`: https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc API Version 6 is currently only considered during a `search` of `name` or `name-desc`. Note: This change was written as a solution to https://bugs.archlinux.org/task/49133. PS: + Some spacing issues fixed in comments. Signed-off-by: Kevin Morris --- doc/rpc.txt | 4 ++++ web/lib/aurjson.class.php | 29 +++++++++++++++++++++-------- web/lib/pkgfuncs.inc.php | 10 +++++----- 3 files changed, 30 insertions(+), 13 deletions(-) diff --git a/doc/rpc.txt b/doc/rpc.txt index 3148ebea..b0f5c4e1 100644 --- a/doc/rpc.txt +++ b/doc/rpc.txt @@ -39,6 +39,10 @@ Examples `/rpc/?v=5&type=search&by=makedepends&arg=boost` `search` with callback:: `/rpc/?v=5&type=search&arg=foobar&callback=jsonp1192244621103` +`search` with API Version 6 for packages containing `cookie` AND `milk`:: + `/rpc/?v=6&type=search&arg=cookie%20milk` +`search` with API Version 6 for packages containing `cookie milk`:: + `/rpc/?v=6&type=search&arg="cookie milk"` `info`:: `/rpc/?v=5&type=info&arg[]=foobar` `info` with multiple packages:: diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 0ac586fe..83ce502a 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -80,7 +80,7 @@ class AurJSON { if (isset($http_data['v'])) { $this->version = intval($http_data['v']); } - if ($this->version < 1 || $this->version > 5) { + if ($this->version < 1 || $this->version > 6) { return $this->json_error('Invalid version specified.'); } @@ -140,7 +140,7 @@ class AurJSON { } /* - * Check if an IP needs to be rate limited. + * Check if an IP needs to be rate limited. * * @param $ip IP of the current request * @@ -192,7 +192,7 @@ class AurJSON { $value = get_cache_value('ratelimit-ws:' . $ip, $status); if (!$status || ($status && $value < $deletion_time)) { if (set_cache_value('ratelimit-ws:' . $ip, $time, $window_length) && - set_cache_value('ratelimit:' . $ip, 1, $window_length)) { + set_cache_value('ratelimit:' . $ip, 1, $window_length)) { return; } } else { @@ -370,7 +370,7 @@ class AurJSON { } elseif ($this->version >= 2) { if ($this->version == 2 || $this->version == 3) { $fields = implode(',', self::$fields_v2); - } else if ($this->version == 4 || $this->version == 5) { + } else if ($this->version >= 4 && $this->version <= 6) { $fields = implode(',', self::$fields_v4); } $query = "SELECT {$fields} " . @@ -492,13 +492,26 @@ class AurJSON { if (strlen($keyword_string) < 2) { return $this->json_error('Query arg too small.'); } - $keyword_string = $this->dbh->quote("%" . addcslashes($keyword_string, '%_') . "%"); if ($search_by === 'name') { - $where_condition = "(Packages.Name LIKE $keyword_string)"; + if ($this->version >= 6) { + $where_condition = construct_keyword_search($this->dbh, + $keyword_string, false); + } else { + $keyword_string = $this->dbh->quote( + "%" . addcslashes($keyword_string, '%_') . "%"); + $where_condition = "(Packages.Name LIKE $keyword_string)"; + } } else if ($search_by === 'name-desc') { - $where_condition = "(Packages.Name LIKE $keyword_string OR "; - $where_condition .= "Description LIKE $keyword_string)"; + if ($this->version >= 6) { + $where_condition = construct_keyword_search($this->dbh, + $keyword_string, true); + } else { + $keyword_string = $this->dbh->quote( + "%" . addcslashes($keyword_string, '%_') . "%"); + $where_condition = "(Packages.Name LIKE $keyword_string "; + $where_condition .= "OR Description LIKE $keyword_string)"; + } } } else if ($search_by === 'maintainer') { if (empty($keyword_string)) { diff --git a/web/lib/pkgfuncs.inc.php b/web/lib/pkgfuncs.inc.php index 8c915711..f6108e5a 100644 --- a/web/lib/pkgfuncs.inc.php +++ b/web/lib/pkgfuncs.inc.php @@ -697,7 +697,9 @@ function pkg_search_page($params, $show_headers=true, $SID="") { } elseif (isset($params["SeB"]) && $params["SeB"] == "k") { /* Search by keywords. */ + $q_where .= " AND ( "; $q_where .= construct_keyword_search($dbh, $params['K'], false); + $q_where .= " )"; } elseif (isset($params["SeB"]) && $params["SeB"] == "N") { /* Search by name (exact match). */ @@ -709,7 +711,9 @@ function pkg_search_page($params, $show_headers=true, $SID="") { } else { /* Keyword search (default). */ + $q_where .= " AND ( "; $q_where .= construct_keyword_search($dbh, $params['K'], true); + $q_where .= " )"; } } @@ -876,11 +880,7 @@ function construct_keyword_search($dbh, $keywords, $namedesc) { $op = "AND "; } - if (!empty($q_keywords)) { - $where_part = "AND (" . $q_keywords . ") "; - } - - return $where_part; + return $q_keywords; } /**