question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Cannot list packages on main screen with verdaccio-aws-s3-storage configured

See original GitHub issue

Your Environment

  • verdaccio version: 5.18.0
  • node version: from official docker image verdaccio/verdaccio:5.18.0
  • package manager: npm, from official docker image verdaccio/verdaccio:5.18.0
  • os: docker image verdaccio/verdaccio:5.18.0
  • platform: kubernetes (helm)

Describe the bug

After installing chart into kubernetes cluster with preprovisioned aws-s3-storage plugin, application cannot list packages, but can reach them via direct URL.

To Reproduce

  1. Configuration block from mostly base helm values.
  2. Install into cluster with following values,
  3. Upload packages into S3 Verdaccio/storage/ (backup from working old version 4.x.x). Couple GBs.
  4. Proceed into UI.

Expected behavior Everything ok on old 4.x.x set up, but…

Screenshots, server logs, package manager log UI reachable, but 0 packages (changed host due security reasons): image

Can reach a package (changed host and package name due security reasons): image

Configuration files Part of very basic values:

 configMap: |
  # necessary (see https://github.com/verdaccio/verdaccio/issues/673)
  storage: /verdaccio/storage/data
  
  store:
    aws-s3-storage:
      keyPrefix: Verdaccio/storage/
      bucket: ${bucket}
      region: ${region}
      accessKeyId: ${access_key_id}
      secretAccessKey: ${secret_access_key}

  auth:
    htpasswd:
      file: /verdaccio/storage/htpasswd
      max_users: 4
      algorithm: bcrypt
      rounds: 10

  uplinks:
    npmjs:
      url: https://registry.npmjs.org/
      max_fails: 100

  packages:
    '@*/*':
      access: $all
      publish: $authenticated
      proxy: npmjs

    '**':
      access: $all
      publish: $authenticated
      proxy: npmjs

  middlewares:
    audit:
      enabled: true

  max_body_size: 100m

  server: 
    keepAliveTimeout: 60

  logs: {type: stdout, format: pretty, level: debug}

Dockerfile

FROM verdaccio/verdaccio:5.18.0

ENV NODE_ENV=production
USER root
RUN npm i -g verdaccio-aws-s3-storage@10.3.0

Environment information EKS 1.22, Helm 3.8.2, Chart 4.10.3, verdaccio-aws-s3-storage@10.3.0, verdaccio@5.18.0

Debugging output

  • $ NODE_DEBUG=request verdaccio display request calls (verdaccio <–> uplinks)
  • $ DEBUG=verdaccio* verdaccio enable extreme verdaccio debug mode (verdaccio api)
  • $ npm -ddd prints:
  • $ npm config get registry prints:

Plugin init starts (changed bucket name due security reasons):

verdaccio:plugin:loader loading plugin /usr/local/lib/node_modules/verdaccio/build/plugins/aws-s3-storage
verdaccio:plugin:loader plugin /usr/local/lib/node_modules/verdaccio/build/plugins/aws-s3-storage not found
verdaccio:plugin:loader loading plugin verdaccio-aws-s3-storage
verdaccio:plugin:loader plugin aws-s3-storage is an npm package
erdaccio:plugin:loader plugin is ES6
{"level":30,"time":1670257501552,"pid":8,"hostname":"verdaccio-64b7b6bf68-txp4f","content":"aws-s3-storage","prefix":"verdaccio","msg":"plugin successfully loaded: @{prefix}-@{content}"}
{"level":20,"time":1670257501552,"pid":8,"hostname":"verdaccio-64b7b6bf68-txp4f","keyPrefix":"Verdaccio/storage/","bucket":"MY_BUCKET","msg":"s3: [_getData] bucket: @{bucket} prefix: @{keyPrefix}"}
{"level":20,"time":1670257502155,"pid":8,"hostname":"verdaccio-64b7b6bf68-txp4f","keyPrefix":"Verdaccio/storage/","bucket":"MY_BUCKET","msg":"s3: [_sync] bucket: @{bucket} prefix: @{keyPrefix}"}
{"level":20,"time":1670257502285,"pid":8,"hostname":"verdaccio-64b7b6bf68-txp4f","msg":"s3: [_sync] sucess"}

Also found empty response from /-/verdaccio/data/packages: image

Issue Analytics

  • State:open
  • Created 10 months ago
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
juanpicadocommented, Dec 7, 2022

Sure, just fyi don’t expect much movement on this until the end of the year, but I have the feeling you have permission issues maybe you can check this in the meantime https://verdaccio.org/blog/2021/04/14/verdaccio-5-migration-guide specially new permissions section for Docker

1reaction
sromaniuk-scommented, Dec 6, 2022

Hello @juanpicado ! Thanks for your response. Yes, I’m migrating from v4 to v5.

Read more comments on GitHub >

github_iconTop Results From Across the Web

aws-s3: Inconsistent package list in cluster deployment #317
The bug demostrates that verdaccio with s3 backend is stateful, with package-list cached in memory, and caused inconsistent and racing issues in ...
Read more >
Configuration File - Verdaccio
The default configuration has support for scoped packages and allows any user to access all packages, but only authenticated users to ...
Read more >
verdaccio-aws-s3-storage - npm
Start using verdaccio-aws-s3-storage in your project by running `npm i ... TypeScript icon, indicating that this package has built-in type ...
Read more >
Get Started with AWS S3 - Webiny
Creating an AWS S3 Bucket. In order to use the SDK, we have to configure the SDK package by loading it into our...
Read more >
Site Deployment - Documentation - Fleek docs
File configuration¶ · image public docker image, default to node:slim · command no command is executed by default · baseDir build command is...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found