[pacman-dev,2/2] Revert "ci: cache packages"

Message ID 20210123214004.1109216-2-emil.l.velikov@gmail.com
State New
Headers show
Series [pacman-dev,1/2] ci: use official image | expand

Commit Message

Emil Velikov Jan. 23, 2021, 9:40 p.m. UTC
This reverts commit e348ba38814c9cc7c1c9892d0451096234dc39ab.

With the above commit we started caching the downloaded packages. Based
on some testing and, it saves ~30s in the "step_script" stage while
adding 18s for "Restoring/Saving cache". A net saving of ~10s.

With earlier commit, we no longer use an ancient image which also pulls
base-devel - thus the packages we have to download is minimal.

Now comparing the uncached "step_script", vs the cached one - it is
slowed by 2-3 seconds (1:01 -> 1:03), while we eliminate the 18s (and
growing) caching.

Tl:Dr: With up-to date image, package caching in not worth it - be that
time, disk or network wise.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
[1] https://gitlab.com/xexaxo/pacman/-/jobs/980145379 - cached, empty
[2] https://gitlab.com/xexaxo/pacman/-/jobs/980131049 - cached, warm
[3] https://gitlab.com/xexaxo/pacman/-/jobs/980733379 - image+, cached
[4] https://gitlab.com/xexaxo/pacman/-/jobs/980765006 - image+, uncached
---
 .gitlab-ci.yml | 15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)

Comments

Emil Velikov Jan. 29, 2021, 5:04 p.m. UTC | #1
On Sat, 23 Jan 2021 at 21:40, Emil Velikov <emil.l.velikov@gmail.com> wrote:
>
> This reverts commit e348ba38814c9cc7c1c9892d0451096234dc39ab.
>
> With the above commit we started caching the downloaded packages. Based
> on some testing and, it saves ~30s in the "step_script" stage while
> adding 18s for "Restoring/Saving cache". A net saving of ~10s.
>
> With earlier commit, we no longer use an ancient image which also pulls
> base-devel - thus the packages we have to download is minimal.
>
> Now comparing the uncached "step_script", vs the cached one - it is
> slowed by 2-3 seconds (1:01 -> 1:03), while we eliminate the 18s (and
> growing) caching.
>
> Tl:Dr: With up-to date image, package caching in not worth it - be that
> time, disk or network wise.
>
> Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
> ---
> [1] https://gitlab.com/xexaxo/pacman/-/jobs/980145379 - cached, empty
> [2] https://gitlab.com/xexaxo/pacman/-/jobs/980131049 - cached, warm
> [3] https://gitlab.com/xexaxo/pacman/-/jobs/980733379 - image+, cached
> [4] https://gitlab.com/xexaxo/pacman/-/jobs/980765006 - image+, uncached

Fwiw I suspect that some of the pipeline data may disappear shortly,
so as a total summary:
The series changes runtime from 2:30 to 1:45 while also cutting the
traffic by 10%+.

-Emil

Patch

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index bfb7e6d2..a0b9dced 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,19 +1,12 @@ 
 variables:
   MAKEFLAGS: "-j10"
   VERBOSE: 1
-  PACMAN_OPTS: --needed --noconfirm --cachedir .pkg-cache
-
-cache:
-  key: pkgs-v1
-  paths:
-    # For some reason Gitlab CI only supports storing cache/artifacts in a path relative to the build directory
-    - .pkg-cache
 
 .arch-test:
   image: archlinux:base-devel
   before_script:
     - >
-      pacman -Syu $PACMAN_OPTS
+      pacman -Syu --needed --noconfirm
       git
       gpgme libarchive curl
       python
@@ -37,14 +30,14 @@  arch-debug:
 arch-docs:
   extends: .arch-test
   script:
-    - pacman -Syu $PACMAN_OPTS asciidoc
+    - pacman -Syu --needed --noconfirm asciidoc
     - meson -Ddoc=enabled build
     - ninja -C build
 
 arch-clang:
   extends: .arch-test
   script:
-    - pacman -Syu $PACMAN_OPTS clang
+    - pacman -Syu --needed --noconfirm clang
     - CC=clang meson build
     - ninja -C build
     - fakechroot meson test -C build
@@ -87,7 +80,6 @@  arch-no-nls:
 
 debian:
   image: debian:bullseye
-  cache: {}
   before_script:
     - apt update
     - >
@@ -103,7 +95,6 @@  debian:
 
 fedora:
   image: fedora
-  cache: {}
   before_script:
     - >
       dnf -y install