From patchwork Mon Oct 19 23:19:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: lesto fante X-Patchwork-Id: 1783 Return-Path: Delivered-To: patchwork@archlinux.org Received: from apollo.archlinux.org (localhost [127.0.0.1]) by apollo.archlinux.org (Postfix) with ESMTP id 87FB01BDCCF48 for ; Mon, 19 Oct 2020 23:20:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on apollo.archlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=DKIM_ADSP_CUSTOM_MED=0.001, DKIM_INVALID=1,DKIM_SIGNED=0.1,FREEMAIL_FROM=0.5,MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_MED=-2.3,RCVD_IN_MSPIKE_H4=-0.01,RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001,T_DMARC_POLICY_NONE=0.01,T_DMARC_SIMPLE_DKIM=0.01 autolearn=ham autolearn_force=no version=3.4.4 X-Spam-BL-Results: [127.0.0.19] [127.0.9.2] Received: from orion.archlinux.org (orion.archlinux.org [88.198.91.70]) by apollo.archlinux.org (Postfix) with ESMTPS for ; Mon, 19 Oct 2020 23:20:09 +0000 (UTC) Received: from orion.archlinux.org (localhost [127.0.0.1]) by orion.archlinux.org (Postfix) with ESMTP id C64731D3AD56D4; Mon, 19 Oct 2020 23:20:04 +0000 (UTC) Authentication-Results: luna.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OHc40eam Authentication-Results: orion.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OHc40eam DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=voT/Y37X6LPsHrdx6mD8MrhvJguf8JBFW7VG/fi98OM=; b=OHc40eamLnAFGfISC/VY5g12+EkfLdOLK7rJnyU1mUkJvkcFDkcQCONRmE7iZ+gP2h n75xb0aF1ayo3pr8xpvsQR5N+7iVATsav0d1AQQwD4mg/47GPikc60HKzipNxqTtLqLw 50V21iFtbm+e0+fmhmzmwOrV/XrCOMaSMEmTdXu3gtXuFn/nUwhylD6c+bwbi+Nf9JcP i/IGsx72cMk6/2/CuHV5LK6kTcqzzjZA1kUhjw/c909/P0MiKrfnxGz13jpRcAfPMYkV a2lFm/yReqXovSKanyvS6bOmg8eJcCl62QhvynpXWgICFGibqgDt7x1lq0tlVk+Hyj2u aQIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=voT/Y37X6LPsHrdx6mD8MrhvJguf8JBFW7VG/fi98OM=; b=dS0JqejMS8UJE0KA0AE6yBjnbAipnSOdTdNlV8q6hj8LTdVw5FORqr+dZYyB87DEoI SJyt6rkOlvSpHR0pbbTZwVPibLnn38yl1XGErqB8ND3ifdZdGa1AYZFUlXELY8VBZ+na 4KaBYPFmUBg3+GHrwBHwHsrBEPiltdeMBWQPoBmC1nsZwvln2LdLCdfOER4X7MdhV2g0 Cgj4HNqzOaYlfdKdCTjcH/IY2mdFe3ukX+EgeeqqMRVDKp6+uj6AxafhYrU2AoMRdr6/ UZCMmIZMXsdpgFGKOWnxMo4BxIX9p9AfSmlByNULXyyiYjaIMOtdmcrMakMuq8sk0Nh2 B2wA== X-Gm-Message-State: AOAM532wj1u9UpvBl8M/tH4fIBDKFiN5S7zcdiJFgwWK8d1hz2vP4Xcd rIl3nOml+/UZp8iHWaKJyM1s/CURSQr0sw== X-Google-Smtp-Source: ABdhPJyPaSXZtaTlhxOrBJnMcErX6Pbor2T7/eYDK753VCGZwbPwKypXTD/Eoz0yN4xOozNhv4/2Lw== X-Received: by 2002:adf:8b92:: with SMTP id o18mr78083wra.54.1603149599610; Mon, 19 Oct 2020 16:19:59 -0700 (PDT) From: lesto To: pacman-dev@archlinux.org Date: Tue, 20 Oct 2020 01:19:56 +0200 Message-Id: <20201019231956.206070-1-lestofante88@gmail.com> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Subject: [pacman-dev] [PATCH] Support parallel download with xfercommand X-BeenThere: pacman-dev@archlinux.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion list for pacman development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Discussion list for pacman development Errors-To: pacman-dev-bounces@archlinux.org Sender: "pacman-dev" Signed-off-by: lesto --- lib/libalpm/alpm.h | 10 ++++++++++ lib/libalpm/dload.c | 9 +++++++++ lib/libalpm/handle.c | 13 +++++++++++++ lib/libalpm/handle.h | 1 + src/pacman/conf.c | 5 ++++- src/pacman/conf.h | 1 + src/pacman/pacman-conf.c | 3 +++ 7 files changed, 41 insertions(+), 1 deletion(-) diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index 614a530c..c2af60e8 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -755,6 +755,11 @@ typedef void (*alpm_cb_totaldl)(off_t total); typedef int (*alpm_cb_fetch)(const char *url, const char *localpath, int force); +/** A callback for waiting for download of files + * @return 0 on success, -1 on error. + */ +typedef int (*alpm_cb_fetch_lock)(void); + /** Fetch a list of remote packages. * @param handle the context handle * @param urls list of package URLs to download @@ -787,6 +792,11 @@ alpm_cb_fetch alpm_option_get_fetchcb(alpm_handle_t *handle); /** Sets the downloading callback. */ int alpm_option_set_fetchcb(alpm_handle_t *handle, alpm_cb_fetch cb); +/** Returns the downloading lock callback. */ +alpm_cb_fetch_lock alpm_option_get_fetch_lockcb(alpm_handle_t *handle); +/** Sets the downloading lock callback. */ +int alpm_option_set_fetch_lockcb(alpm_handle_t *handle, alpm_cb_fetch_lock cb); + /** Returns the callback used to report total download size. */ alpm_cb_totaldl alpm_option_get_totaldlcb(alpm_handle_t *handle); /** Sets the callback used to report total download size. */ diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 673e769f..174d559d 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -824,6 +824,15 @@ int _alpm_download(alpm_handle_t *handle, RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1); } } + + if (handle->fetch_lockcb != NULL) { + // if fetch_lockcb is set, fetchcb is non-blocking; here we wait for all download to complete + int ret = handle->fetch_lockcb(); + if (ret == -1) { + RET_ERR(handle, ALPM_ERR_EXTERNAL_DOWNLOAD, -1); + } + } + return 0; } } diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c index 1310601a..683e678d 100644 --- a/lib/libalpm/handle.c +++ b/lib/libalpm/handle.c @@ -174,6 +174,12 @@ alpm_cb_fetch SYMEXPORT alpm_option_get_fetchcb(alpm_handle_t *handle) return handle->fetchcb; } +alpm_cb_fetch_lock SYMEXPORT alpm_option_get_fetch_lockcb(alpm_handle_t *handle) +{ + CHECK_HANDLE(handle, return NULL); + return handle->fetch_lockcb; +} + alpm_cb_totaldl SYMEXPORT alpm_option_get_totaldlcb(alpm_handle_t *handle) { CHECK_HANDLE(handle, return NULL); @@ -321,6 +327,13 @@ int SYMEXPORT alpm_option_set_fetchcb(alpm_handle_t *handle, alpm_cb_fetch cb) return 0; } +int SYMEXPORT alpm_option_set_fetch_lockcb(alpm_handle_t *handle, alpm_cb_fetch_lock cb) +{ + CHECK_HANDLE(handle, return -1); + handle->fetch_lockcb = cb; + return 0; +} + int SYMEXPORT alpm_option_set_totaldlcb(alpm_handle_t *handle, alpm_cb_totaldl cb) { CHECK_HANDLE(handle, return -1); diff --git a/lib/libalpm/handle.h b/lib/libalpm/handle.h index 9fef0fbf..dc00751b 100644 --- a/lib/libalpm/handle.h +++ b/lib/libalpm/handle.h @@ -73,6 +73,7 @@ struct __alpm_handle_t { alpm_cb_download dlcb; /* Download callback function */ alpm_cb_totaldl totaldlcb; /* Total download callback function */ alpm_cb_fetch fetchcb; /* Download file callback function */ + alpm_cb_fetch_lock fetch_lockcb; /* Download lock file callback function */ alpm_cb_event eventcb; alpm_cb_question questioncb; alpm_cb_progress progresscb; diff --git a/src/pacman/conf.c b/src/pacman/conf.c index 3a3ef605..53de73b8 100644 --- a/src/pacman/conf.c +++ b/src/pacman/conf.c @@ -157,6 +157,7 @@ int config_free(config_t *oldconfig) FREELIST(oldconfig->hookdirs); FREELIST(oldconfig->cachedirs); free(oldconfig->xfercommand); + free(oldconfig->xferlockcommand); free(oldconfig->print_format); free(oldconfig->arch); wordsplit_free(oldconfig->xfercommand_argv); @@ -319,7 +320,9 @@ static int download_with_xfercommand(const char *url, const char *localpath, for(i = 0; i <= config->xfercommand_argc; i++) { const char *val = config->xfercommand_argv[i]; if(val && strcmp(val, "%o") == 0) { - usepart = 1; + if (config->xferlockcommand == NULL) { + usepart = 1; + } val = tempfile; } else if(val && strcmp(val, "%u") == 0) { val = url; diff --git a/src/pacman/conf.h b/src/pacman/conf.h index b8a451ad..1a9d637d 100644 --- a/src/pacman/conf.h +++ b/src/pacman/conf.h @@ -130,6 +130,7 @@ typedef struct __config_t { char *xfercommand; char **xfercommand_argv; size_t xfercommand_argc; + char *xferlockcommand; /* our connection to libalpm */ alpm_handle_t *handle; diff --git a/src/pacman/pacman-conf.c b/src/pacman/pacman-conf.c index 463badf1..7c4f4cc9 100644 --- a/src/pacman/pacman-conf.c +++ b/src/pacman/pacman-conf.c @@ -259,6 +259,7 @@ static void dump_config(void) show_str("Architecture", config->arch); show_str("XferCommand", config->xfercommand); + show_str("XferLockCommand", config->xferlockcommand); show_bool("UseSyslog", config->usesyslog); show_bool("Color", config->color); @@ -366,6 +367,8 @@ static int list_directives(void) show_str("Architecture", config->arch); } else if(strcasecmp(i->data, "XferCommand") == 0) { show_str("XferCommand", config->xfercommand); + } else if(strcasecmp(i->data, "XferLockCommand") == 0) { + show_str("XferLockCommand", config->xferlockcommand); } else if(strcasecmp(i->data, "UseSyslog") == 0) { show_bool("UseSyslog", config->usesyslog);