#1148

refactor(api): remove Dataset production compatibility aliases scheduled for 0.19.0

Openiamgpwants to merge intofrom

Goal and scope

Honours the 0.19.0 removal promise in docs/reference/public-name-compatibility.md (#1020) before the 0.19.0 release (#1146) ships. Removes every Dataset production "pipeline" compatibility alias listed there.

Non-goals: the Jobs UI /pipelines URLs, the pipelines capability page ID that gates them, Dagster/DLT native pipeline names, and run-evidence PipelineRun records are unchanged. These are retained contracts in the same document.

Change map and risk

Behaviour changes (pre-announced removal):

  • Python aliases ObservatoryDatasetPipeline, ObservatoryPipelineStage, ObservatoryPipelineList removed (module __getattr__ shim and TYPE_CHECKING aliases).
  • ObservatoryDatasetProfile.pipeline: validation alias, warning validator, and deprecated computed response field removed. A pipeline input key is now ignored.
  • Production model last_run: validation alias, warning validator, and deprecated computed response field removed.
  • GET /api/observatory/pipelines removed (now 404), along with its security_manifest.py entry.
  • Profile sections.pipelines key removed. sections.production remains.
  • Dataset capability metadata on the pipelines page now reports profile_sections=["production"] and read_models=["dataset-production", "dataset-profile"].
  • Counters phlo.legacy.dataset_production_name.uses and phlo.legacy.dataset_production_route.uses removed. They had no other users.

Mechanical: regenerated docs/reference/http-api.md and generated/http-api.json (272 endpoints). Updated the route counts in the unified-api-phase-0.md inventory (68 to 67 Observatory routes; 107 non-v1). Updated the glossary and the python-api.md removal schedule. public-name-compatibility.md now records the removal as completed in 0.19.0.

First-party consumers checked: the Observatory frontend (packages/phlo-observatory) does not call the old route or read pipeline, last_run on production, sections.pipelines, or the capability read_models/profile_sections values. Its last_run uses are the separate transform model. phlo-mcp has no references. No consumer needed migration.

Verification

All results are on this revision with fixtures and the local test suite. No live services were used.

  • make check: all steps pass (py test: 7557 passed, 4 skipped).

  • uv run pytest packages/phlo-api/tests/test_observatory_api.py packages/phlo-api/tests/test_observatory_capabilities.py packages/phlo-api/tests/test_v1_contract.py: 140 passed.

  • npm --prefix packages/phlo-observatory/src/phlo_observatory test: 297 passed, including the OpenAPI client-contract check contracts.test.mjs.

  • npm --prefix packages/phlo-observatory/src/phlo_observatory run build: succeeds.

  • make docs-build: succeeds. make docs-reference-check reports the docs are current.

  • Not a fix. The compatibility test was replaced by test_dataset_production_legacy_names_are_removed, which asserts that the aliases, fields, route (404 and absent from OpenAPI), and section key are gone.

Recovery and maintainer decisions

The maintainer decided to remove the aliases in 0.19.0 rather than move the date. Recovery is a source revert. No data, migrations, or persisted identifiers are involved. External clients still on the old names must move to the canonical names listed in public-name-compatibility.md. Zero-use telemetry was not confirmed; the release-date deadline applies instead.


    

Activity

303
coldtea-pr-lens[bot]8h ago
  • Redraw · drawn at e1d1471, the branch has moved on

Architecture


Data flow


Review

✨ Nothing flagged · reviewed e1d1471


TIP

Open a diagram on the canvas, then press W or click play to walk through the change one step at a time

🪧 More tips
  • Run npx skills add coldteadotai/pr-lens, then tell your coding agent: "Diagram the change you just made with PR Lens and attach it to the pull request."
  • Run npx @coldtea/pr-lens-cli analyze --base origin/main on a branch, then npx @coldtea/pr-lens-cli render .pr-lens/graph.json. Same lenses, your own model key, before the pull request exists
  • Click the link under each diagram to open it on a canvas you can zoom, pan and step through
  • The diagrams are links. Click one to open it on the canvas, then press W or click play to walk through the change
  • The CLI's render reads .github/pr-lens.yml and applies your renames, exclusions and lane pins at draw time
  • Set github.comment.collapsed: true in .github/pr-lens.yml to fold the comment behind one View architecture… row. Drawing still runs as before
  • Set github.draw: on-demand in .github/pr-lens.yml and PR Lens stops drawing pull requests on its own. Comment @pr-lens draw on a pull request when you want that one drawn
  • Add .github/workflows/pr-lens.yml with coldteadotai/pr-lens/packages/action@v0 and your model provider's key as its api-key to run PR Lens from your own CI. Any /chat/completions endpoint works
  • After a push, tick Redraw to draw the latest commit
  • Switch GitHub to dark mode and the diagrams follow. The moving dots are this pull request's data in motion
  • Set github.description: false in .github/pr-lens.yml to stop PR Lens adding buttons to your PR descriptions

Thanks for using PR Lens! It's free for OSS. If it helped, a star helps other teams find it. 

phlo-agent[bot]8h ago

Reviewed e1d1471 against the removal promise in docs/reference/public-name-compatibility.md. No blocking correctness, security, or compatibility problem found.

What I checked

  • Scope is complete. All eight listed surfaces are removed: the three Python aliases and module __getattr__, ObservatoryDatasetProfile.pipeline (validator, computed field, setter), production last_run, the /api/observatory/pipelines route, its security_manifest.py classification, sections.pipelines, the capability metadata values, and both phlo.legacy.dataset_production_* counters. No residue of these names is left in the changed modules.
  • Retained contracts are untouched. The pipelines capability page ID, the Jobs UI /pipelines/* routes, Dagster/DLT native pipeline names, and run-evidence PipelineRun appear nowhere in the diff, matching the non-goals.
  • No in-repo caller of the removed route or fields. The Observatory Jobs client reads api/v1/jobs and api/v1/runs (src/lib/data/api/pipelines.ts), the UI has no dataset-profile view reading sections or pipeline, and the remaining legacy-caller inventory in docs/reference/unified-api-remaining-acceptance.md lists no /api/observatory/pipelines consumer. The capability metadata change therefore has no reader to migrate.
  • The route deletion is actually verified, not just re-documented. test_v1_contract.py::test_documented_route_decisions_partition_the_mounted_inventory asserts documented == mounted and re-checks the phase-0 family and suffix counts, so the regenerated generated/http-api.json (272 endpoints) and the 68→67 / 6→5 edits in unified-api-phase-0.md are checked against the mounted app rather than prose.
  • The replacement test has the right shape. test_dataset_production_legacy_names_are_removed asserts absence of the aliases, response fields, section key, route (404 and absent from OpenAPI) and the silently ignored pipeline input key.

One sequencing dependency

The docs now read "removed in 0.19.0", but main is still pre-0.19.0: PR #1146 is open on this same base (7feced4d). Merge this before 0.19.0 is cut; if the release branch is generated first, the removal lands in a later version while the 0.19.0 docs claim it is already gone.

Validation still outstanding

Hosted checks on e1d1471 had not started when I looked (only the container contracts and PR Lens checks had completed; security / introduced dependency risk was queued), so the passing counts in the description are the local make check and frontend runs, not CI. This change has no UI state change, so browser evidence is not required; external Python and JSON clients remain unobservable, as the description already records.

admin changed the title from refactor(api)!: remove Dataset production compatibility aliases scheduled for 0.19.0 to refactor(api): remove Dataset production compatibility aliases scheduled for 0.19.08h ago