Review Process
The multi-stage PR lifecycle for a scraper - draft, CTD review, PDW review, staging QA, launch.
city-scrapers (core + consumer repos)city-scrapers-core and the per-city repos like city-scrapers-fortxReview process
The steps below reflect the current team workflow. Statuses, reviewer assignments, and branch names evolve over time - confirm specifics with your team lead before acting on them.
A scraper PR passes through three distinct review phases before it reaches production. The full lifecycle:
Draft PR openedContributor
Spider file, mixin (if factory pattern), test file, and saved fixtures are in. The draft flag signals 'ready for a first pass', not merge consideration.
CTD reviewCTD team lead
A CTD reviewer runs the spider and checks output against the field rubric (QA), then reads the code (logic, style, edge cases, test coverage).
Ready for reviewContributor + PDW
Contributor resolves the first-pass feedback and unmarks draft. PDW reviewers are added; the Airtable backlog record moves to 'Ready to integrate to staging'.
Merge to stagingPDW
PDW completes its review and merges the PR into the staging branch - the PR itself stays open. The spider's output goes live on the staging site.
CB QA on stagingCity Bureau
City Bureau checks output against the public-site standard. Feedback goes back on the still-open PR; fixes are pushed and re-merged to staging.
Sign-off and launchPDW merges
CB approval is recorded in the Airtable backlog ('Ready for launch'). The PR merges to main, closes, and output begins appearing on the public site.
Why the PR stays open through staging
The staging merge is not the end of the PR. Merging into staging makes the
spider's output visible on the staging environment of Documenters.org, where
the City Bureau team evaluates it against the public-site standard. Keeping
the PR open means CB feedback lands on the same thread - the contributor
pushes fixes to the same branch and it gets re-merged to staging as needed.
What each phase checks
- CTD review is two checks in one: a QA pass (run the spider, compare output to the source site against the rubric) and a code review (logic, style, edge cases, test coverage).
- PDW review is a secondary code look plus minor QA - PDW owns operations on the Documenters side.
- CB staging QA is data QA only: does the output meet the standard expected on the public site.
Tracking through Airtable
The scraper's record in the Airtable backlog mirrors the lifecycle:
- Ready-for-review -> "Ready to integrate to staging" (confirm the exact label with your team lead).
- CB sign-off -> "Ready for launch" (same caveat).
The backlog entry is created automatically when the PR opens - see Airtable sync for how that automation works.
Last updated on