"publish --force" and "unpublish VERSION" seem to be broken

Describe the bug

I tried to look at #1359 and #1405 but couldn’t find a solution. I’m using ~~https://github.com/Remitly/verdaccio-s3-storage~~ https://registry.verdaccio.org/-/web/detail/verdaccio-aws-s3-storage as a storage plugin, but you can comment the whole storage key in configuration and it will be the same (except for one thing that is pointed out below).

npm publish --force

If I try to overwrite an existing package version with a new one:

npm publish --force --registry MY_REPO

The result is:

• old package is removed from the remote storage and the web UI doesn’t show it
• the npm command fails with:

npm ERR! code EPUBLISHCONFLICT
npm ERR! publish fail Cannot publish over existing version.
npm ERR! publish fail Update the 'version' field in package.json and try again.
npm ERR! publish fail 
npm ERR! publish fail To automatically increment version numbers, see:
npm ERR! publish fail     npm help version

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/marco/.npm/_logs/2019-08-10T13_46_04_312Z-debug.log

• running the same exact command again publishes successfully

npm unpublish --force SPECIFIC_VERSION

If I try to unpublish the package first to avoid errors:

npm unpublish "@test/my-pack@1.0.0-beta" --force --registry MY_REPO

The result is:

• npm exits with no errors and says that the package has been removed
• the package still shows up on the AWS S3 storage and on the web UI
• trying to publish the same package from now on will result in failures (this doesn’t happen when using local storage: publishing again works as with the command above)

The only way I found to recover from this situation was:

• removing the .tgz archive manually from the storage
• restart the verdaccio server

These commands don’t work as expected.

To Reproduce

I’m using the following Dockerfile to speed up testing:

FROM verdaccio/verdaccio:latest

USER root

ENV NODE_ENV=production

# Install "envsubst" to replace ENV_VARS in config
RUN apk add gettext libintl

# Install AWS S3 Plugin
RUN yarn add verdaccio-aws-s3-storage --production=true --no-lockfile --registry https://registry.verdaccio.org

# Copy configuration and .htpasswd
COPY ./config.yml  $VERDACCIO_APPDIR/conf/config.yaml

# Fix permissions
RUN chown -R $VERDACCIO_USER_UID:root $VERDACCIO_APPDIR/conf/config.yaml \
    && chmod -R g=u $VERDACCIO_APPDIR/conf/config.yaml

USER $VERDACCIO_USER_UID

# Set user credentials, replace ENV_VARS in config and finally run Verdaccio
CMD echo "$HTPASSWD" > /verdaccio/storage/htpasswd \
    && envsubst \$AWS_S3_BUCKET,\$AWS_S3_PATH_PREFIX,\$AWS_DEFAULT_REGION,\$AWS_S3_ENDPOINT,\$AWS_S3_PATH_STYLE,\$AWS_ACCESS_KEY_ID,\$AWS_SECRET_ACCESS_KEY < $VERDACCIO_APPDIR/conf/config.yaml > $VERDACCIO_APPDIR/conf/config.yaml \
    && $VERDACCIO_APPDIR/bin/verdaccio --config $VERDACCIO_APPDIR/conf/config.yaml --listen $VERDACCIO_PROTOCOL://0.0.0.0:$VERDACCIO_PORT

This docker-compose.yml should run your registry:

version: '2.1'

services:
  verdaccio:
    image: testing/private-verdaccio-s3:latest
    environment:
      AWS_S3_BUCKET: 
      AWS_S3_PATH_PREFIX: 
      AWS_ACCESS_KEY_ID: 
      AWS_SECRET_ACCESS_KEY: 
      AWS_DEFAULT_REGION:
      AWS_S3_ENDPOINT: 
      AWS_S3_PATH_STYLE: "true"
      # User is demo:demo
      HTPASSWD: |
        demo:$apr1$uq2Uw53i$dS7GoGvNuupumWvXjOJ.R1
    ports:
      - "4874:4873"

Configuration file config.yml:

#
# This is the config file used for the docker images.
# It allows all users to do anything, so don't use it on production systems.
#
# Do not configure host and port under `listen` in this file
# as it will be ignored when using docker.
# see https://verdaccio.org/docs/en/docker#docker-and-custom-port-configuration
#
# Look here for more config file examples:
# https://github.com/verdaccio/verdaccio/tree/master/conf
#

# path to a directory with all packages
storage: /verdaccio/storage/data

# comment this section entirely to disable AWS S3
store:
  aws-s3-storage:
    bucket: $AWS_S3_BUCKET
    keyPrefix: $AWS_S3_PATH_PREFIX
    region: $AWS_DEFAULT_REGION
    endpoint: $AWS_S3_ENDPOINT
    s3ForcePathStyle: $AWS_S3_PATH_STYLE
    accessKeyId: $AWS_ACCESS_KEY_ID
    secretAccessKey: $AWS_SECRET_ACCESS_KEY

web:
  title: Verdaccio

auth:
  htpasswd:
    file: /verdaccio/storage/htpasswd
    max_users: -1

security:
  api:
    jwt:
      sign:
        expiresIn: 60d
        notBefore: 1
  web:
    sign:
      expiresIn: 7d
      notBefore: 1

# a list of other known repositories we can talk to
uplinks:
  npmjs:
    url: https://registry.npmjs.org/

packages:
  '**':
    access: $all
    publish: $all
    unpublish: $all

    # if package is not available locally, proxy requests to 'npmjs' registry
    proxy: npmjs

middlewares:
  audit:
    enabled: true

# log settings
logs:
  - { type: stdout, format: pretty, level: http }

listen:
  - 0.0.0.0:4873

Put these 3 files in a folder called verdaccio-test, then build and run everything:

cd /path/to/verdaccio-test
docker build -t testing/private-verdaccio-s3:latest .
docker-compose up -d

# Attach to logs
docker logs -f verdaccio-test_verdaccio_1

# use http://127.0.0.1:4874

Expected behavior

Packages should be deleted correctly.

Conclusion

I hope that everything is clear enough, please feel free to ask for further information.

Thank you for your time.

[favoyang]

Here’s the breakdown for all bugs mentioned in the issue.

TLDR: jump to the last paragraph.

Bug 1 - npm unpublish -f failed with error, but package is actually removed by verdaccio.

• storage: s3
• reason: verdaccio returns 404 error when removing package folder at the last step, which is harmless for s3 storage. Ignore the error fixed the issue.
• fix: https://github.com/verdaccio/monorepo/pull/289

Fact 2 - npm unpublish [-f] pkg@version work as expect after patching PR https://github.com/verdaccio/monorepo/pull/289

Bug 3 - npm publish -f (republish) failed with error, but the package@version file is removed by verdaccio (npm publish -f == npm unpublish -f). Thus run the publish command again will succeed, because verdaccio thinks you’re publishing a new version.

• storage: all
• registry.npm.org behaviors
  • republish same version is not allowed, returns error 403 - “Cannot publish over previously published version”
  • unpublish pkg@version is allowed. However, publish just unpublished pkg@version is not allowed, returns error 403. That means you can not use this trick to overwrite a published version. This is not considered as a bug, because it is consistent as above.
  • the only way to fix issues in a published version is to publish a new version instead.
• reason
  • when work with registry.npm.org
    • the npm client requests publish -f, the server returns a 403 (permission error)
    • the npm client stops here

    npm http fetch PUT 403 https://registry.npmjs.org/com.bastianblokland.enumgenerator 6995ms
    npm verb stack Error: 403 Forbidden - PUT https://registry.npmjs.org/com.bastianblokland.enumgenerator - You cannot publish over the previously published versions: 1.6.1.
    npm verb stack     at C:\Program Files\nodejs\node_modules\npm\node_modules\npm-registry-fetch\check-response.js:104:15
    npm verb stack     at processTicksAndRejections (internal/process/task_queues.js:93:5)
    npm verb statusCode 403
    

  • when work with verdaccio,
    • the npm client requests publish -f, the server returns 409 (conflict error)
    • the npm client continue to work because of -f, fetches package info, the server returns 200
    • the npm client fetches package info again, the server returns 304 (no change)
    • the npm client requests to delete the pkg@version, the server returns 201 (successfully deleted)
    • the npm client raises version conflict error. Here the issue, the npm client shall skip the version check, because of -f.
    • this end up with the result that pkg@version being removed, with no pkg@version get published. (aka publish -f pkg@version == unpubish pkg@version)

    npm http fetch PUT 409 http://127.0.0.1:4873/com.bastianblokland.enumgenerator 78ms
    npm http fetch GET 200 http://127.0.0.1:4873/com.bastianblokland.enumgenerator?write=true 30ms
    npm WARN publish Forced publish over com.bastianblokland.enumgenerator@1.6.1
    npm http fetch GET 304 http://127.0.0.1:4873/com.bastianblokland.enumgenerator?write=true 35ms (from cache)
    npm http fetch DELETE 201 http://127.0.0.1:4873/com.bastianblokland.enumgenerator/-rev/3-3da368dddcba208e 13ms
    npm verb stack Error: Cannot publish com.bastianblokland.enumgenerator@undefined over existing version.
    npm verb stack     at ConflictError (C:\Program Files\nodejs\node_modules\npm\node_modules\libnpmpublish\publish.js:211:24)
    npm verb stack     at patchMetadata (C:\Program Files\nodejs\node_modules\npm\node_modules\libnpmpublish\publish.js:164:11)
    npm verb stack     at C:\Program Files\nodejs\node_modules\npm\node_modules\libnpmpublish\publish.js:63:22
    npm verb stack     at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
    npm verb stack     at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
    npm verb stack     at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18)
    npm verb stack     at Promise._settlePromise0 (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:619:10)
    npm verb stack     at Promise._settlePromises (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:699:18)
    npm verb stack     at _drainQueueStep (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\async.js:138:12)
    npm verb stack     at _drainQueue (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\async.js:131:9)
    npm verb stack     at Async._drainQueues (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\async.js:147:5)
    npm verb stack     at Immediate.Async.drainQueues [as _onImmediate] (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\async.js:17:14)
    npm verb stack     at processImmediate (internal/timers.js:439:21)
    npm verb pkgid com.bastianblokland.enumgenerator
    npm verb cwd C:\Users\favoy\Documents\projects\verdaccio\enum-generator-unity
    npm verb Windows_NT 10.0.18363
    npm verb argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "--registry" "http://127.0.0.1:4873" "publish" "-f" "--verbose"
    npm verb node v12.13.1
    npm verb npm  v6.12.1
    npm ERR! code EPUBLISHCONFLICT
    npm ERR! publish fail Cannot publish over existing version.
    npm ERR! publish fail Update the 'version' field in package.json and try again.
    npm ERR! publish fail
    npm ERR! publish fail To automatically increment version numbers, see:
    npm ERR! publish fail     npm help version
    npm verb exit [ 1, true ]
    

  • fix: won’t fix. It is not a verdaccio server issue, but a npm client issue.

Based on the analysis

• I suggest verdaccio merge https://github.com/verdaccio/monorepo/pull/289 to fix npm unpublish -f bug for s3 storage.
• @liarco because of the publish -f bug is not fixable by verdaccio. I suggest you run npm unpublish pkg@version, then npm publish to simulate the npm publish -f.

[favoyang]

@liarco I have verified the issue only appears in S3, and fixed in https://github.com/verdaccio/monorepo/pull/289, see the PR description for details.