Skip to content

E2E - QA: init, deploy, versions list, config link, 2nd deploy#7367

Open
phyllis-sy-wu wants to merge 1 commit intopsyw-0421-E2E-teardown-polishfrom
psyw-0421-E2E-QA-A-app-dev
Open

E2E - QA: init, deploy, versions list, config link, 2nd deploy#7367
phyllis-sy-wu wants to merge 1 commit intopsyw-0421-E2E-teardown-polishfrom
psyw-0421-E2E-QA-A-app-dev

Conversation

@phyllis-sy-wu
Copy link
Copy Markdown
Contributor

@phyllis-sy-wu phyllis-sy-wu commented Apr 22, 2026

WHY are these changes introduced?

Automates test A of the manual QA flow (tracked in #7368 / sub-issue #7369). The existing app-deploy.spec.ts only covered a single init→deploy→versions-list cycle. Test A extends it to the full manual flow, adding coverage for:

  • Version-tagged deploy + message persistence
  • app config link creating a second app (interactive PTY prompts)
  • Deploying against the linked secondary config
  • Verifying each app's active version is correctly isolated

This closes the gap for the config link-to-new-app codepath, which previously had no E2E coverage.

WHAT is this pull request doing?

Expands tests/app-deploy.spec.ts to the full A scope:

  1. shopify app init --template reactRouter (primary app)
  2. shopify app deploy --version v1 + assert active on primary
  3. shopify app versions list --json + assert versionTag matches
  4. shopify app config link --config secondary (via PTY) → creates a new app within the primary app's organization
  5. shopify app deploy --config secondary --version v2 + assert active on secondary
  6. shopify app versions list --config secondary --json + assert isolation from primary

Also adds helpers in setup/app.ts (config-link PTY driver, JSON-mode versions list, shared assertion for active-version checks) reused across this and future A–F tests.

How to test your changes?

# One-shot
pnpm --filter e2e exec playwright test app-deploy

# With live CLI output
DEBUG=1 pnpm --filter e2e exec playwright test app-deploy

Requires .env with E2E_ACCOUNT_EMAIL, E2E_ACCOUNT_PASSWORD, E2E_ORG_ID.

Runtime ~2 min. Resources created during the test are the two apps (primary + secondary) — teardown deletes both automatically.

Example

DEBUG=1 pnpm --filter e2e exec playwright test app-deploy
Expand for complete log
cli % DEBUG=1 pnpm --filter e2e exec playwright test app-deploy
[e2e][auth] global setup starting

To run this command, log in to Shopify.
User verification code: LLTN-KJVC
👉 Open this link to start the auth process: https://accounts.shopify.com/activate-with-code?device_code%5Buser_code%5D=LLTN-KJVC
✔ Logged in.
✔ Current account: genghis-khan-identity-1-2025-03-31@shopify.com.
[e2e][auth] browser sessions established for admin + dev dashboard
[e2e][auth] global setup done, config at /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/global-auth/XDG_CONFIG_HOME

Running 1 test using 1 worker

     1 tests/app-deploy.spec.ts:64:3 › App deploy › init, deploy, versions list, config link, deploy to secondary

[e2e][w0] ----- TEST: init, deploy, versions list, config link, deploy to secondary ----- 
[e2e][auth] copying session from global setup
[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/create-app/bin/run.js
[e2e][w0][cli] app init --template reactRouter --flavor javascript --name E2E-deploy1-1776876378681 --package-manager pnpm --local --organization-id 161686155 --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X
╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Initializing project with `pnpm`                                            │
│  Use the `--package-manager` flag to select a different package manager.     │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯


╭─ success ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  e2-e-deploy1-1776876378681 is ready for you to build!                       │
│                                                                              │
│  Next steps                                                                  │
│    • Run `cd e2-e-deploy1-1776876378681`                                     │
│    • For extensions, run `shopify app generate extension`                    │
│    • To see your app, run `shopify app dev`                                  │
│                                                                              │
│  Reference                                                                   │
│    • Shopify docs [1]                                                        │
│    • Shopify Dev MCP, [2] connect your AI assistant to development           │
│      resources                                                               │
│    • For an overview of commands, run `shopify app --help`                   │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://shopify.dev
[2] https://shopify.dev/docs/apps/build/devmcp

[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app deploy --version E2E-v1-1776876395516 --message E2E A primary deployment --force --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
╭─ warning ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  The `--force` flag is deprecated and will be removed in the next major      │
│  release.                                                                    │
│                                                                              │
│  Use `--allow-updates` for CI/CD environments, or `--allow-updates           │
│  --allow-deletes` if you also want to allow removals.                        │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Your configuration file has been modified                                   │
│                                                                              │
│  The `include_config_on_deploy` field is no longer supported, since all      │
│  apps must now include configuration on deploy. It has been removed from     │
│  your configuration file.                                                    │
│                                                                              │
│  See Shopify CLI documentation. [1]                                          │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://shopify.dev/docs/apps/build/cli-for-apps/app-configuration#build

╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Using shopify.app.toml for default values:                                  │
│                                                                              │
│    • Org:             core-build-develop-admin-web-e2e                       │
│    • App:             E2E-deploy1-1776876378681                              │
│                                                                              │
│   You can pass `--reset` to your command to reset your app configuration.    │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯


Releasing a new app version as part of E2E-deploy1-1776876378681



╭─ success ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  New version released to users.                                              │
│                                                                              │
│  E2E-v1-1776876395516 [1]                                                    │
│  E2E A primary deployment                                                    │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://dev.shopify.com/dashboard/161686155/apps/351353143297/versions/93525
3475329

[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app versions list --json --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
[
  {
    "createdAt": "2026-04-22 16:46:39",
    "createdBy": "gid://shopify/User/187591014",
    "versionTag": "E2E-v1-1776876395516",
    "status": "active",
    "versionId": "gid://shopify/Version/935253475329",
    "message": "E2E A primary deployment"
  },
  {
    "createdAt": "2026-04-22 16:46:33",
    "createdBy": "gid://shopify/User/187591014",
    "versionTag": "e2e-deploy1-1776876378681-1",
    "status": "inactive",
    "versionId": "gid://shopify/Version/935253278721",
    "message": ""
  }
]
[e2e][w0][cli] spawn: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app config link --config secondary --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
?  Which organization is this work for?
✔  core-build-develop-admin-web-e2e (161686155)

?  Create this project as a new app on Shopify?
✔  Yes, create it as a new app

?  App name:
✔  E2E-deploy2-1776876378681

╭─ success ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  shopify.app.secondary.toml is now linked to "E2E-deploy2-1776876378681" on  │
│   Shopify                                                                    │
│                                                                              │
│  Using shopify.app.secondary.toml as your default config.                    │
│                                                                              │
│  Next steps                                                                  │
│    • Make updates to shopify.app.secondary.toml in your local project        │
│    • To upload your config, run `shopify app deploy`                         │
│                                                                              │
│  Reference                                                                   │
│    • App configuration [1]                                                   │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://shopify.dev/docs/apps/tools/cli/configuration

[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app deploy --version E2E-v2-1776876408185 --message E2E A secondary deployment --config secondary --force --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
╭─ warning ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  The `--force` flag is deprecated and will be removed in the next major      │
│  release.                                                                    │
│                                                                              │
│  Use `--allow-updates` for CI/CD environments, or `--allow-updates           │
│  --allow-deletes` if you also want to allow removals.                        │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Your configuration file has been modified                                   │
│                                                                              │
│  The `include_config_on_deploy` field is no longer supported, since all      │
│  apps must now include configuration on deploy. It has been removed from     │
│  your configuration file.                                                    │
│                                                                              │
│  See Shopify CLI documentation. [1]                                          │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://shopify.dev/docs/apps/build/cli-for-apps/app-configuration#build

╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Using shopify.app.secondary.toml for default values:                        │
│                                                                              │
│    • Org:             core-build-develop-admin-web-e2e                       │
│    • App:             E2E-deploy2-1776876378681                              │
│                                                                              │
│   You can pass `--reset` to your command to reset your app configuration.    │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯


Releasing a new app version as part of E2E-deploy2-1776876378681



╭─ success ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  New version released to users.                                              │
│                                                                              │
│  E2E-v2-1776876408185 [1]                                                    │
│  E2E A secondary deployment                                                  │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://dev.shopify.com/dashboard/161686155/apps/351353241601/versions/93525
3737473

[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app versions list --json --config secondary --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
[
  {
    "createdAt": "2026-04-22 16:46:51",
    "createdBy": "gid://shopify/User/187591014",
    "versionTag": "E2E-v2-1776876408185",
    "status": "active",
    "versionId": "gid://shopify/Version/935253737473",
    "message": "E2E A secondary deployment"
  },
  {
    "createdAt": "2026-04-22 16:46:46",
    "createdBy": "gid://shopify/User/187591014",
    "versionTag": "e2e-deploy2-1776876378681-1",
    "status": "inactive",
    "versionId": "gid://shopify/Version/935253573633",
    "message": ""
  }
]

[e2e][w0] ----- Teardown: app E2E-deploy1-1776876378681 ----- 
[e2e][w0][browser] deleting app
[e2e][w0][browser] app found, deleting
[e2e][w0][browser] app deleted

[e2e][w0] ----- Teardown: app E2E-deploy2-1776876378681 ----- 
[e2e][w0][browser] deleting app
[e2e][w0][browser] app found, deleting
[e2e][w0][browser] app deleted
  ✓  1 tests/app-deploy.spec.ts:64:3 › App deploy › init, deploy, versions list, config link, deploy to secondary (1.5m)

  1 passed (2.2m)

Post-release steps

N/A

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows) — E2E tests run on macOS/Linux in CI
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact
  • The change is user-facing — no user-facing changes (test infrastructure only), no changeset needed

Copy link
Copy Markdown
Contributor Author

phyllis-sy-wu commented Apr 22, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions
Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/analytics.d.ts
@@ -19,7 +19,6 @@ interface EnvironmentData {
     env_device_id: string;
     env_cloud: string;
     env_package_manager: string;
-    env_install_package_manager: string;
     env_is_global: boolean;
     env_auth_method: string;
     env_is_wsl: boolean;
packages/cli-kit/dist/public/node/monorail.d.ts
@@ -2,7 +2,7 @@ import { JsonMap } from '../../private/common/json.js';
 import { DeepRequired } from '../common/ts/deep-required.js';
 export { DeepRequired };
 type Optional<T> = T | null;
-export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.22";
+export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.21";
 export interface Schemas {
     [MONORAIL_COMMAND_TOPIC]: {
         sensitive: {
@@ -129,7 +129,6 @@ export interface Schemas {
             env_ci_platform?: Optional<string>;
             env_device_id?: Optional<string>;
             env_package_manager?: Optional<string>;
-            env_install_package_manager?: Optional<string>;
             env_package_manager_workspaces?: Optional<boolean>;
             env_plugin_installed_any_custom?: Optional<boolean>;
             env_plugin_installed_shopify?: Optional<string>;

@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-improve-cli-log-format branch from 9d0a3aa to bfb1932 Compare April 22, 2026 14:52
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 26a7e70 to 95f628a Compare April 22, 2026 14:52
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-improve-cli-log-format to graphite-base/7367 April 22, 2026 15:15
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 95f628a to f0005dd Compare April 22, 2026 15:15
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-teardown-polish April 22, 2026 15:15
@phyllis-sy-wu phyllis-sy-wu linked an issue Apr 22, 2026 that may be closed by this pull request
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-teardown-polish to graphite-base/7367 April 22, 2026 17:17
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from f0005dd to 633cc3c Compare April 22, 2026 17:17
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-improve-cli-log-format April 22, 2026 17:17
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-improve-cli-log-format to graphite-base/7367 April 22, 2026 18:41
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 633cc3c to 8e559ab Compare April 22, 2026 18:41
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-teardown-polish April 22, 2026 18:41
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-teardown-polish to graphite-base/7367 April 22, 2026 19:31
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 8e559ab to 64d0528 Compare April 22, 2026 20:46
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-teardown-polish April 22, 2026 20:46
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-teardown-polish to graphite-base/7367 April 22, 2026 21:03
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 64d0528 to e952944 Compare April 22, 2026 21:14
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-teardown-polish April 22, 2026 21:14
@phyllis-sy-wu phyllis-sy-wu marked this pull request as ready for review April 22, 2026 21:23
@phyllis-sy-wu phyllis-sy-wu requested a review from a team as a code owner April 22, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

app-deploy

1 participant