From patchwork Mon Mar 9 22:23:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anatol Pomozov X-Patchwork-Id: 1555 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id F133D17864299 for ; Mon, 9 Mar 2020 22:23:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=DKIM_ADSP_CUSTOM_MED=0.001, DKIM_INVALID=1,DKIM_SIGNED=0.1,FREEMAIL_FROM=0.5,MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_MED=-2.3,SPF_HELO_NONE=0.001,T_DMARC_POLICY_NONE=0.01, T_DMARC_SIMPLE_DKIM=0.01 autolearn=ham autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Mon, 9 Mar 2020 22:23:24 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 0E58019E1617D6; Mon, 9 Mar 2020 22:23:23 +0000 (UTC) Received: from luna.archlinux.org (luna.archlinux.org [IPv6:2a01:4f8:160:3033::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: luna) by orion.archlinux.org (Postfix) with ESMTPSA id CD99719E1617CF; Mon, 9 Mar 2020 22:23:22 +0000 (UTC) Authentication-Results: orion.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FTZVdVK2 Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id C0FE02A8F2; Mon, 9 Mar 2020 22:23:22 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FTZVdVK2 Received: from luna.archlinux.org (luna.archlinux.org [127.0.0.1]) by luna.archlinux.org (Postfix) with ESMTP id A0C9E2A8EE for ; Mon, 9 Mar 2020 22:23:18 +0000 (UTC) Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by luna.archlinux.org (Postfix) with ESMTPS for ; Mon, 9 Mar 2020 22:23:18 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id 82C4619E1617CD for ; Mon, 9 Mar 2020 22:23:17 +0000 (UTC) Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by orion.archlinux.org (Postfix) with ESMTPS for ; Mon, 9 Mar 2020 22:23:17 +0000 (UTC) Received: by mail-pl1-x642.google.com with SMTP id w12so4566299pll.13 for ; Mon, 09 Mar 2020 15:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=b+GHi+ehiz64bTDbsdn7wBHO4RRQjaGD8rhZjq/7PLo=; b=FTZVdVK2DU42yIgqwx/JAuzgmpFStLGnUoYy0D8737cYf33Hr6M+VvevqVvReOM036 JCDaog7WG3V92NKu4lrp3vydmbYDSmaJVRjyYsR4B6jsOBBIwxEbZWZX33Mm8I9Y4fr7 mOOXYS20VPBAlcSQ+nzuUAypfa3kck0Krq/pJX69q3z3pOOqshYX6EjqQD3kwTXla+kD 0rk6ZWeHfon99dyDeD3SFT9Pj9i9s7iKr78+vUQXGrKHONETkrH2/lfXTa/oEvoS0VIt kQHX0A+dWPcSrNw23RFD6NcK92ZxVykqroRxl/ml7y8UQJ2QY6Li6h0lTytJ2zhQNTa0 Z7ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=b+GHi+ehiz64bTDbsdn7wBHO4RRQjaGD8rhZjq/7PLo=; b=KGa55+LK5qKwIDLGuvSgOG0V7xsIpf80Er3OQU+taYIojeAJ7JrDEe2x3ocYJZ4frt hq6oC60lXJeYylU79629PpEe3VBeu6zPAk/lvalLUp5zFT3cNFMjM4DgKYk4ao8ljYCB mww3VGC2o4MZbHbp5a/su3jxyQLkiVFE9dGyRJ73/6T5gRoFZssgrVfubhSP86wk+2zE TxyQ1TUV+Yyv1YInBkgO1YA1ypeLOcLqs40lfrkUgqkotQhsjO1BpLmdTegLN2zRz2IW LzYAqQWGA78fxf1mdqrREWPxURiBP8rd0nPBf5+c1r2dh7h3CZ3OwY8qi4xVnQXHbea+ egfw== X-Gm-Message-State: ANhLgQ1mRfX9zUaHVGt1Lw9y7Toa0x/azg2ZXxv+CkbZz0k6ItCZSLf1 Ki/ykm45U777nrmaJKDDdSqfxZq9KCE= X-Google-Smtp-Source: ADFU+vsILONLItlk5xaHYeO6sK4VFkvZoSDCuy0r5Sm73i9XtvAyqsKG5YlC7dmhe+iskUe9YpRL7Q== X-Received: by 2002:a17:902:9a86:: with SMTP id w6mr17683820plp.37.1583792595013; Mon, 09 Mar 2020 15:23:15 -0700 (PDT) Received: from wolf.hsd1.ca.comcast.net ([2601:646:8500:a860:2efd:a1ff:febf:a363]) by smtp.gmail.com with ESMTPSA id h7sm47987733pfq.36.2020.03.09.15.23.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Mar 2020 15:23:14 -0700 (PDT) From: Anatol Pomozov To: pacman-dev@archlinux.org Date: Mon, 9 Mar 2020 15:23:12 -0700 Message-Id: <20200309222312.208724-1-anatol.pomozov@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [pacman-dev] [PATCH] Add multi_curl handle to ALPM global context X-BeenThere: pacman-dev@archlinux.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@archlinux.org Sender: "pacman-dev" To be able to run multiple download in parallel efficiently we need to use curl_multi interface [1]. It introduces a set of APIs over new type of handler 'CURLM'. Create CURLM object at the application start and set it to global ALPM context. The 'single-download' CURL handle moves to payload struct. A new CURL handle is created for each payload with intention to be processed by CURLM. Note that curl_download_internal() is not ported to CURLM interface due to the fact that the function will go away soon. [1] https://curl.haxx.se/libcurl/c/libcurl-multi.html --- lib/libalpm/alpm.c | 12 +++++++++--- lib/libalpm/dload.c | 17 ++++++----------- lib/libalpm/dload.h | 1 + lib/libalpm/handle.c | 5 ----- lib/libalpm/handle.h | 2 +- 5 files changed, 17 insertions(+), 20 deletions(-) diff --git a/lib/libalpm/alpm.c b/lib/libalpm/alpm.c index d1265214..34c5b4b2 100644 --- a/lib/libalpm/alpm.c +++ b/lib/libalpm/alpm.c @@ -70,6 +70,11 @@ alpm_handle_t SYMEXPORT *alpm_initialize(const char *root, const char *dbpath, goto cleanup; } +#ifdef HAVE_LIBCURL + curl_global_init(CURL_GLOBAL_ALL); + myhandle->curlm = curl_multi_init(); +#endif + #ifdef ENABLE_NLS bindtextdomain("libalpm", LOCALEDIR); #endif @@ -104,13 +109,14 @@ int SYMEXPORT alpm_release(alpm_handle_t *myhandle) ret = -1; } - _alpm_handle_unlock(myhandle); - _alpm_handle_free(myhandle); - #ifdef HAVE_LIBCURL + curl_multi_cleanup(myhandle->curlm); curl_global_cleanup(); #endif + _alpm_handle_unlock(myhandle); + _alpm_handle_free(myhandle); + return ret; } diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index bec3ff5e..1f28c7c2 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -72,15 +72,6 @@ static char *get_fullpath(const char *path, const char *filename, return filepath; } -static CURL *get_libcurl_handle(alpm_handle_t *handle) -{ - if(!handle->curl) { - curl_global_init(CURL_GLOBAL_SSL); - handle->curl = curl_easy_init(); - } - return handle->curl; -} - enum { ABORT_SIGINT = 1, ABORT_OVER_MAXFILESIZE @@ -241,7 +232,7 @@ static size_t dload_parseheader_cb(void *ptr, size_t size, size_t nmemb, void *u } } - curl_easy_getinfo(payload->handle->curl, CURLINFO_RESPONSE_CODE, &respcode); + curl_easy_getinfo(payload->curl, CURLINFO_RESPONSE_CODE, &respcode); if(payload->respcode != respcode) { payload->respcode = respcode; } @@ -377,8 +368,9 @@ static int curl_download_internal(struct dload_payload *payload, struct sigaction orig_sig_pipe, orig_sig_int; /* shortcut to our handle within the payload */ alpm_handle_t *handle = payload->handle; - CURL *curl = get_libcurl_handle(handle); + CURL *curl = curl_easy_init(); handle->pm_errno = ALPM_ERR_OK; + payload->curl = curl; /* make sure these are NULL */ FREE(payload->tempfile_name); @@ -594,6 +586,9 @@ cleanup: unlink(payload->tempfile_name); } + curl_easy_cleanup(curl); + payload->curl = NULL; + /* restore the old signal handlers */ unmask_signal(SIGINT, &orig_sig_int); unmask_signal(SIGPIPE, &orig_sig_pipe); diff --git a/lib/libalpm/dload.h b/lib/libalpm/dload.h index 3eb7fbe1..65fcdadb 100644 --- a/lib/libalpm/dload.h +++ b/lib/libalpm/dload.h @@ -44,6 +44,7 @@ struct dload_payload { int trust_remote_name; int cb_initialized; #ifdef HAVE_LIBCURL + CURL *curl; CURLcode curlerr; /* last error produced by curl */ #endif }; diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c index 8ae08d11..0d4797bf 100644 --- a/lib/libalpm/handle.c +++ b/lib/libalpm/handle.c @@ -64,11 +64,6 @@ void _alpm_handle_free(alpm_handle_t *handle) closelog(); } -#ifdef HAVE_LIBCURL - /* release curl handle */ - curl_easy_cleanup(handle->curl); -#endif - #ifdef HAVE_LIBGPGME FREELIST(handle->known_keys); #endif diff --git a/lib/libalpm/handle.h b/lib/libalpm/handle.h index cd7104f9..9fef0fbf 100644 --- a/lib/libalpm/handle.h +++ b/lib/libalpm/handle.h @@ -59,7 +59,7 @@ struct __alpm_handle_t { #ifdef HAVE_LIBCURL /* libcurl handle */ - CURL *curl; /* reusable curl_easy handle */ + CURLM *curlm; unsigned short disable_dl_timeout; unsigned int parallel_downloads; /* number of download streams */ #endif