Skip to content

Improper cache path handing for mounted caches #1667

@Jetblack-io

Description

@Jetblack-io

Minimal .gitlab-ci.yml illustrating the issue

---
create cache:
  stage: create_cache
  script:
  - mkdir -p cache1 cache2
  - touch cache1/foo cache2/bar
  cache:
  - key: cachekey
    paths:
    - cache1
    - cache2

list cache:
  stage: list_cache
  script:
  - find . -name ".git" -prune -o -type f
  cache:
  - key: cachekey
    paths:
    - cache1
    - cache2

Launching that with --mounted-cache flag results in duplicate files.

Expected behavior

Proper cache path handling with --mount-cache flag

Host information

Ubuntu 24.04.2
gitlab-ci-local 4.62.0

Containerd binary

Docker 28.2.2

Additional context

Running gitlab-ci-local with --mount-cache flag and multiple cache paths creates a single docker volume for each key.
But it does not preserve specified paths within the docker volume, combining files from all specified paths instead.
Without that flag everything seems to work fine, but copying the cache back and forth takes a LOT of time on IO constrained systems, especially with large caches.

Mounted caches seem to work relatively well with --privileged flag, and I even got them to work properly with --no-umask flag,
but the structure inside the volume is wrong and unreliable when multiple paths are used.

Right now the only workaround is using exactly one path per cache key, but I think that proper behaviour could be achieved either by preserving paths inside volume, or by creating a separate named volume for each path.

(A friend without GitHub account asked me to post this)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions