[v2] Support conjunctive keyword search in RPC interface

Message ID 20200706005726.24915-1-kevr.gtalk@gmail.com
State New
Headers show
Series
  • [v2] Support conjunctive keyword search in RPC interface
Related show

Commit Message

Kevin Morris July 6, 2020, 12:57 a.m. UTC
Newly supported API Version 6 modifies `type=search` functionality; it
now behaves the same as `name` or `name-desc` search through the
https://aur.archlinux.org/packages/ search page.

Search for packages containing the literal keyword `blah blah` AND `haha`:
https://aur.archlinux.org/rpc/?v=6&type=search&arg="blah blah"%20haha

Search for packages containing the literal keyword `abc 123`:
https://aur.archlinux.org/rpc/?v=6&type=search&arg="abc 123"

The following example searches for packages that contain `blah` AND `abc`:
https://aur.archlinux.org/rpc/?v=6&type=search&arg=blah%20abc

The legacy method still searches for packages that contain `blah abc`:
https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc
https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc

API Version 6 is currently only considered during a `search` of `name` or
`name-desc`.

Additionally, this change adds support for conjuctive search when searching by
`name` in https://aur.archlinux.org/packages/.

Note: This change was written as a solution to
https://bugs.archlinux.org/task/49133.

PS: + Some spacing issues fixed in comments.

Signed-off-by: Kevin Morris <kevr.gtalk@gmail.com>
---
 doc/rpc.txt               |  4 ++++
 web/lib/aurjson.class.php | 34 ++++++++++++++++++++++----------
 web/lib/pkgfuncs.inc.php  | 41 +++++++++++++++++++++++++--------------
 3 files changed, 54 insertions(+), 25 deletions(-)

Comments

Kevin Morris July 6, 2020, 1:03 a.m. UTC | #1
Uh, unsure if adding conjunctive to `name` search in the search page is
what you want? I figured that made sense for both name and name-desc, like
it does on rpc currently...

Thoughts?

On Sun, Jul 5, 2020 at 5:57 PM Kevin Morris <kevr.gtalk@gmail.com> wrote:

> Newly supported API Version 6 modifies `type=search` functionality; it
> now behaves the same as `name` or `name-desc` search through the
> https://aur.archlinux.org/packages/ search page.
>
> Search for packages containing the literal keyword `blah blah` AND `haha`:
> https://aur.archlinux.org/rpc/?v=6&type=search&arg="blah blah"%20haha
>
> Search for packages containing the literal keyword `abc 123`:
> https://aur.archlinux.org/rpc/?v=6&type=search&arg="abc 123"
>
> The following example searches for packages that contain `blah` AND `abc`:
> https://aur.archlinux.org/rpc/?v=6&type=search&arg=blah%20abc
>
> The legacy method still searches for packages that contain `blah abc`:
> https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc
> https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc
>
> API Version 6 is currently only considered during a `search` of `name` or
> `name-desc`.
>
> Additionally, this change adds support for conjuctive search when
> searching by
> `name` in https://aur.archlinux.org/packages/.
>
> Note: This change was written as a solution to
> https://bugs.archlinux.org/task/49133.
>
> PS: + Some spacing issues fixed in comments.
>
> Signed-off-by: Kevin Morris <kevr.gtalk@gmail.com>
> ---
>  doc/rpc.txt               |  4 ++++
>  web/lib/aurjson.class.php | 34 ++++++++++++++++++++++----------
>  web/lib/pkgfuncs.inc.php  | 41 +++++++++++++++++++++++++--------------
>  3 files changed, 54 insertions(+), 25 deletions(-)
>
> diff --git a/doc/rpc.txt b/doc/rpc.txt
> index 3148ebea..b0f5c4e1 100644
> --- a/doc/rpc.txt
> +++ b/doc/rpc.txt
> @@ -39,6 +39,10 @@ Examples
>    `/rpc/?v=5&type=search&by=makedepends&arg=boost`
>  `search` with callback::
>    `/rpc/?v=5&type=search&arg=foobar&callback=jsonp1192244621103`
> +`search` with API Version 6 for packages containing `cookie` AND `milk`::
> +  `/rpc/?v=6&type=search&arg=cookie%20milk`
> +`search` with API Version 6 for packages containing `cookie milk`::
> +  `/rpc/?v=6&type=search&arg="cookie milk"`
>  `info`::
>    `/rpc/?v=5&type=info&arg[]=foobar`
>  `info` with multiple packages::
> diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php
> index 0ac586fe..da1af9be 100644
> --- a/web/lib/aurjson.class.php
> +++ b/web/lib/aurjson.class.php
> @@ -80,7 +80,7 @@ class AurJSON {
>                 if (isset($http_data['v'])) {
>                         $this->version = intval($http_data['v']);
>                 }
> -               if ($this->version < 1 || $this->version > 5) {
> +               if ($this->version < 1 || $this->version > 6) {
>                         return $this->json_error('Invalid version
> specified.');
>                 }
>
> @@ -140,7 +140,7 @@ class AurJSON {
>         }
>
>         /*
> -        * Check if an IP needs to be  rate limited.
> +        * Check if an IP needs to be rate limited.
>          *
>          * @param $ip IP of the current request
>          *
> @@ -192,7 +192,7 @@ class AurJSON {
>                 $value = get_cache_value('ratelimit-ws:' . $ip, $status);
>                 if (!$status || ($status && $value < $deletion_time)) {
>                         if (set_cache_value('ratelimit-ws:' . $ip, $time,
> $window_length) &&
> -                           set_cache_value('ratelimit:' . $ip, 1,
> $window_length)) {
> +                               set_cache_value('ratelimit:' . $ip, 1,
> $window_length)) {
>                                 return;
>                         }
>                 } else {
> @@ -370,7 +370,7 @@ class AurJSON {
>                 } elseif ($this->version >= 2) {
>                         if ($this->version == 2 || $this->version == 3) {
>                                 $fields = implode(',', self::$fields_v2);
> -                       } else if ($this->version == 4 || $this->version
> == 5) {
> +                       } else if ($this->version >= 4 && $this->version
> <= 6) {
>                                 $fields = implode(',', self::$fields_v4);
>                         }
>                         $query = "SELECT {$fields} " .
> @@ -492,13 +492,27 @@ class AurJSON {
>                         if (strlen($keyword_string) < 2) {
>                                 return $this->json_error('Query arg too
> small.');
>                         }
> -                       $keyword_string = $this->dbh->quote("%" .
> addcslashes($keyword_string, '%_') . "%");
>
> -                       if ($search_by === 'name') {
> -                               $where_condition = "(Packages.Name LIKE
> $keyword_string)";
> -                       } else if ($search_by === 'name-desc') {
> -                               $where_condition = "(Packages.Name LIKE
> $keyword_string OR ";
> -                               $where_condition .= "Description LIKE
> $keyword_string)";
> +                       if ($this->version >= 6) {
> +
> +                               // API Version 6.
> +                               $namedesc = $search_by === 'name-desc';
> +                               $where_condition =
> construct_keyword_search($this->dbh,
> +                                       $keyword_string, $namedesc, false);
> +
> +                       } else {
> +
> +                               // API Version 5 and below.
> +                               $keyword_string = $this->dbh->quote(
> +                                       "%" . addcslashes($keyword_string,
> '%_') . "%");
> +
> +                               if ($search_by === 'name') {
> +                                       $where_condition = "(Packages.Name
> LIKE $keyword_string)";
> +                               } else if ($search_by === 'name-desc') {
> +                                       $where_condition = "(Packages.Name
> LIKE $keyword_string ";
> +                                       $where_condition .= "OR
> Description LIKE $keyword_string)";
> +                               }
> +
>                         }
>                 } else if ($search_by === 'maintainer') {
>                         if (empty($keyword_string)) {
> diff --git a/web/lib/pkgfuncs.inc.php b/web/lib/pkgfuncs.inc.php
> index 8c915711..e56fef77 100644
> --- a/web/lib/pkgfuncs.inc.php
> +++ b/web/lib/pkgfuncs.inc.php
> @@ -687,8 +687,9 @@ function pkg_search_page($params, $show_headers=true,
> $SID="") {
>                 }
>                 elseif (isset($params["SeB"]) && $params["SeB"] == "n") {
>                         /* Search by name. */
> -                       $K = "%" . addcslashes($params['K'], '%_') . "%";
> -                       $q_where .= "AND (Packages.Name LIKE " .
> $dbh->quote($K) . ") ";
> +                       $q_where .= "AND (";
> +                       $q_where .= construct_keyword_search($dbh,
> $params['K'], false, false);
> +                       $q_where .= ") ";
>                 }
>                 elseif (isset($params["SeB"]) && $params["SeB"] == "b") {
>                         /* Search by package base name. */
> @@ -696,8 +697,10 @@ function pkg_search_page($params, $show_headers=true,
> $SID="") {
>                         $q_where .= "AND (PackageBases.Name LIKE " .
> $dbh->quote($K) . ") ";
>                 }
>                 elseif (isset($params["SeB"]) && $params["SeB"] == "k") {
> -                       /* Search by keywords. */
> -                       $q_where .= construct_keyword_search($dbh,
> $params['K'], false);
> +                       /* Search by name. */
> +                       $q_where .= "AND (";
> +                       $q_where .= construct_keyword_search($dbh,
> $params['K'], false, true);
> +                       $q_where .= ") ";
>                 }
>                 elseif (isset($params["SeB"]) && $params["SeB"] == "N") {
>                         /* Search by name (exact match). */
> @@ -709,7 +712,9 @@ function pkg_search_page($params, $show_headers=true,
> $SID="") {
>                 }
>                 else {
>                         /* Keyword search (default). */
> -                       $q_where .= construct_keyword_search($dbh,
> $params['K'], true);
> +                       $q_where .= "AND (";
> +                       $q_where .= construct_keyword_search($dbh,
> $params['K'], true, true);
> +                       $q_where .= ") ";
>                 }
>         }
>
> @@ -833,10 +838,11 @@ function pkg_search_page($params,
> $show_headers=true, $SID="") {
>   * @param handle $dbh Database handle
>   * @param string $keywords The search term
>   * @param bool $namedesc Search name and description fields
> + * @param bool $keyword Search packages with a matching
> PackageBases.Keyword
>   *
>   * @return string WHERE part of the SQL clause
>   */
> -function construct_keyword_search($dbh, $keywords, $namedesc) {
> +function construct_keyword_search($dbh, $keywords, $namedesc,
> $keyword=false) {
>         $count = 0;
>         $where_part = "";
>         $q_keywords = "";
> @@ -863,11 +869,20 @@ function construct_keyword_search($dbh, $keywords,
> $namedesc) {
>                 $q_keywords .= $op . " (";
>                 if ($namedesc) {
>                         $q_keywords .= "Packages.Name LIKE " .
> $dbh->quote($term) . " OR ";
> -                       $q_keywords .= "Description LIKE " .
> $dbh->quote($term) . " OR ";
> +                       $q_keywords .= "Description LIKE " .
> $dbh->quote($term) . " ";
> +               }
> +               else {
> +                       $q_keywords .= "Packages.Name LIKE " .
> $dbh->quote($term) . " ";
> +               }
> +
> +               if ($keyword) {
> +                       $q_keywords .= "OR EXISTS (SELECT * FROM
> PackageKeywords WHERE ";
> +                       $q_keywords .= "PackageKeywords.PackageBaseID =
> Packages.PackageBaseID AND ";
> +                       $q_keywords .= "PackageKeywords.Keyword LIKE " .
> $dbh->quote($term) . ")) ";
> +               }
> +               else {
> +                       $q_keywords .= ") ";
>                 }
> -               $q_keywords .= "EXISTS (SELECT * FROM PackageKeywords
> WHERE ";
> -               $q_keywords .= "PackageKeywords.PackageBaseID =
> Packages.PackageBaseID AND ";
> -               $q_keywords .= "PackageKeywords.Keyword LIKE " .
> $dbh->quote($term) . ")) ";
>
>                 $count++;
>                 if ($count >= 20) {
> @@ -876,11 +891,7 @@ function construct_keyword_search($dbh, $keywords,
> $namedesc) {
>                 $op = "AND ";
>         }
>
> -       if (!empty($q_keywords)) {
> -               $where_part = "AND (" . $q_keywords . ") ";
> -       }
> -
> -       return $where_part;
> +       return $q_keywords;
>  }
>
>  /**
> --
> 2.20.1
>
>

Patch

diff --git a/doc/rpc.txt b/doc/rpc.txt
index 3148ebea..b0f5c4e1 100644
--- a/doc/rpc.txt
+++ b/doc/rpc.txt
@@ -39,6 +39,10 @@  Examples
   `/rpc/?v=5&type=search&by=makedepends&arg=boost`
 `search` with callback::
   `/rpc/?v=5&type=search&arg=foobar&callback=jsonp1192244621103`
+`search` with API Version 6 for packages containing `cookie` AND `milk`::
+  `/rpc/?v=6&type=search&arg=cookie%20milk`
+`search` with API Version 6 for packages containing `cookie milk`::
+  `/rpc/?v=6&type=search&arg="cookie milk"`
 `info`::
   `/rpc/?v=5&type=info&arg[]=foobar`
 `info` with multiple packages::
diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php
index 0ac586fe..da1af9be 100644
--- a/web/lib/aurjson.class.php
+++ b/web/lib/aurjson.class.php
@@ -80,7 +80,7 @@  class AurJSON {
 		if (isset($http_data['v'])) {
 			$this->version = intval($http_data['v']);
 		}
-		if ($this->version < 1 || $this->version > 5) {
+		if ($this->version < 1 || $this->version > 6) {
 			return $this->json_error('Invalid version specified.');
 		}
 
@@ -140,7 +140,7 @@  class AurJSON {
 	}
 
 	/*
-	 * Check if an IP needs to be  rate limited.
+	 * Check if an IP needs to be rate limited.
 	 *
 	 * @param $ip IP of the current request
 	 *
@@ -192,7 +192,7 @@  class AurJSON {
 		$value = get_cache_value('ratelimit-ws:' . $ip, $status);
 		if (!$status || ($status && $value < $deletion_time)) {
 			if (set_cache_value('ratelimit-ws:' . $ip, $time, $window_length) &&
-			    set_cache_value('ratelimit:' . $ip, 1, $window_length)) {
+				set_cache_value('ratelimit:' . $ip, 1, $window_length)) {
 				return;
 			}
 		} else {
@@ -370,7 +370,7 @@  class AurJSON {
 		} elseif ($this->version >= 2) {
 			if ($this->version == 2 || $this->version == 3) {
 				$fields = implode(',', self::$fields_v2);
-			} else if ($this->version == 4 || $this->version == 5) {
+			} else if ($this->version >= 4 && $this->version <= 6) {
 				$fields = implode(',', self::$fields_v4);
 			}
 			$query = "SELECT {$fields} " .
@@ -492,13 +492,27 @@  class AurJSON {
 			if (strlen($keyword_string) < 2) {
 				return $this->json_error('Query arg too small.');
 			}
-			$keyword_string = $this->dbh->quote("%" . addcslashes($keyword_string, '%_') . "%");
 
-			if ($search_by === 'name') {
-				$where_condition = "(Packages.Name LIKE $keyword_string)";
-			} else if ($search_by === 'name-desc') {
-				$where_condition = "(Packages.Name LIKE $keyword_string OR ";
-				$where_condition .= "Description LIKE $keyword_string)";
+			if ($this->version >= 6) {
+
+				// API Version 6.
+				$namedesc = $search_by === 'name-desc';
+				$where_condition = construct_keyword_search($this->dbh,
+					$keyword_string, $namedesc, false);
+
+			} else {
+
+				// API Version 5 and below.
+				$keyword_string = $this->dbh->quote(
+					"%" . addcslashes($keyword_string, '%_') . "%");
+
+				if ($search_by === 'name') {
+					$where_condition = "(Packages.Name LIKE $keyword_string)";
+				} else if ($search_by === 'name-desc') {
+					$where_condition = "(Packages.Name LIKE $keyword_string ";
+					$where_condition .= "OR Description LIKE $keyword_string)";
+				}
+
 			}
 		} else if ($search_by === 'maintainer') {
 			if (empty($keyword_string)) {
diff --git a/web/lib/pkgfuncs.inc.php b/web/lib/pkgfuncs.inc.php
index 8c915711..e56fef77 100644
--- a/web/lib/pkgfuncs.inc.php
+++ b/web/lib/pkgfuncs.inc.php
@@ -687,8 +687,9 @@  function pkg_search_page($params, $show_headers=true, $SID="") {
 		}
 		elseif (isset($params["SeB"]) && $params["SeB"] == "n") {
 			/* Search by name. */
-			$K = "%" . addcslashes($params['K'], '%_') . "%";
-			$q_where .= "AND (Packages.Name LIKE " . $dbh->quote($K) . ") ";
+			$q_where .= "AND (";
+			$q_where .= construct_keyword_search($dbh, $params['K'], false, false);
+			$q_where .= ") ";
 		}
 		elseif (isset($params["SeB"]) && $params["SeB"] == "b") {
 			/* Search by package base name. */
@@ -696,8 +697,10 @@  function pkg_search_page($params, $show_headers=true, $SID="") {
 			$q_where .= "AND (PackageBases.Name LIKE " . $dbh->quote($K) . ") ";
 		}
 		elseif (isset($params["SeB"]) && $params["SeB"] == "k") {
-			/* Search by keywords. */
-			$q_where .= construct_keyword_search($dbh, $params['K'], false);
+			/* Search by name. */
+			$q_where .= "AND (";
+			$q_where .= construct_keyword_search($dbh, $params['K'], false, true);
+			$q_where .= ") ";
 		}
 		elseif (isset($params["SeB"]) && $params["SeB"] == "N") {
 			/* Search by name (exact match). */
@@ -709,7 +712,9 @@  function pkg_search_page($params, $show_headers=true, $SID="") {
 		}
 		else {
 			/* Keyword search (default). */
-			$q_where .= construct_keyword_search($dbh, $params['K'], true);
+			$q_where .= "AND (";
+			$q_where .= construct_keyword_search($dbh, $params['K'], true, true);
+			$q_where .= ") ";
 		}
 	}
 
@@ -833,10 +838,11 @@  function pkg_search_page($params, $show_headers=true, $SID="") {
  * @param handle $dbh Database handle
  * @param string $keywords The search term
  * @param bool $namedesc Search name and description fields
+ * @param bool $keyword Search packages with a matching PackageBases.Keyword
  *
  * @return string WHERE part of the SQL clause
  */
-function construct_keyword_search($dbh, $keywords, $namedesc) {
+function construct_keyword_search($dbh, $keywords, $namedesc, $keyword=false) {
 	$count = 0;
 	$where_part = "";
 	$q_keywords = "";
@@ -863,11 +869,20 @@  function construct_keyword_search($dbh, $keywords, $namedesc) {
 		$q_keywords .= $op . " (";
 		if ($namedesc) {
 			$q_keywords .= "Packages.Name LIKE " . $dbh->quote($term) . " OR ";
-			$q_keywords .= "Description LIKE " . $dbh->quote($term) . " OR ";
+			$q_keywords .= "Description LIKE " . $dbh->quote($term) . " ";
+		}
+		else {
+			$q_keywords .= "Packages.Name LIKE " . $dbh->quote($term) . " ";
+		}
+
+		if ($keyword) {
+			$q_keywords .= "OR EXISTS (SELECT * FROM PackageKeywords WHERE ";
+			$q_keywords .= "PackageKeywords.PackageBaseID = Packages.PackageBaseID AND ";
+			$q_keywords .= "PackageKeywords.Keyword LIKE " . $dbh->quote($term) . ")) ";
+		}
+		else {
+			$q_keywords .= ") ";
 		}
-		$q_keywords .= "EXISTS (SELECT * FROM PackageKeywords WHERE ";
-		$q_keywords .= "PackageKeywords.PackageBaseID = Packages.PackageBaseID AND ";
-		$q_keywords .= "PackageKeywords.Keyword LIKE " . $dbh->quote($term) . ")) ";
 
 		$count++;
 		if ($count >= 20) {
@@ -876,11 +891,7 @@  function construct_keyword_search($dbh, $keywords, $namedesc) {
 		$op = "AND ";
 	}
 
-	if (!empty($q_keywords)) {
-		$where_part = "AND (" . $q_keywords . ") ";
-	}
-
-	return $where_part;
+	return $q_keywords;
 }
 
 /**