Scratch org not listed in Scratch section when authenticated via JWT

Summary

We use a server-based automation (bitbucket pipelines) to create new scratch orgs, then authenticate from a dev’s local machine to the new scratch using auth:jwt:token. When we do this, the scratch org is listed in the first (non-scratch) section of force:org:list, and expiration date info is not available.

Steps To Reproduce:

NOTE you will need two systems with sfdx installed to reproduce this issue. It may be possible to fake this by doing all steps on one system, but deleting the org file from ~/.sfdx after step 3, but I have not tested that scenario.

1. Ensure that your dev hub is configured for JWT auth and that you have your server key file and client key.
2. On machine 1, authenticate to your dev hub via JWT.
3. Using any repo, On machine 1, create a scratch org. Make note of the username created.
4. On machine 2, authenticate to your dev hub via JWT.
5. On machine 2, authenticate to the new scratch org using JWT. For example, sfdx auth:jwt:grant --clientid ${!CONSUMER_KEY} --jwtkeyfile ./server.key -r https://test.salesforce.com --username [e-mail] -a bugreport
6. On machine 2, run sfdx force:org:list.

Expected result

The new scratch org (alias bugreport) should be listed in the second section of output, as a scratch org, with an expiration date.

Actual result

The new scratch org is listed in the first section. No expiration date information is available.

Additional information

This is NOT a duplicate of #845; creating a scratch and using it on the same system does not exhibit this issue.

This is actually behavior that has existed for quite a while, however, prior to a recent release (I think it changed in v54.4), running force:org:display for the new org would fix the issue, by updating the org file in ~/.sfdx. This is no longer the case.

If you compare org files in ~/.sfdx, comparing a scratch created on the local system with a JWT-auth’d scratch created elsewhere, you can see the the JWT version is missing the keys createdOrgInstance, created, and expirationDate. Manually adding the expirationDate key to the file with a reasonable value is sufficient to make force:org:list display the org as a scratch org.

SFDX CLI Version(to find the version of the CLI engine run sfdx --version):

sfdx-cli/7.93.1-762bca056d darwin-x64 node-v14.15.4

SFDX plugin Version(to find the version of the CLI plugin run sfdx plugins --core)

@oclif/plugin-autocomplete 0.3.0 (core)
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.2 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.10.0 (core)
@oclif/plugin-update 1.4.0-3 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-trust 3.6.0 (core)
alias 1.1.8 (core)
auth 1.5.1 (core)
config 1.2.7 (core)
generator 1.1.5 (core)
salesforcedx 51.5.0 (core)
├─ limits 1.0.4 (core)
├─ schema 1.0.4 (core)
├─ custom-metadata 1.0.11 (core)
├─ apex 0.1.4 (core)
├─ org 1.5.0 (core)
├─ user 1.2.0 (core)
├─ @salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
├─ salesforce-alm 51.6.3 (core)
└─ templates 51.3.0 (core)
sfdx-cli 7.93.1 (core)
sfpowerkit 2.8.6
telemetry 1.1.1 (core)

OS and version:

macOS version 10.14.6

[mshanemc]

https://github.com/forcedotcom/cli/blob/main/releasenotes/README.md#51140-may-27-2021---cli-71030

[jayree]

@jclark-dot-org can you please try to run force:org:shape:list and check if the scratch orgs are listed in the correct section again.