workflow: check TiDB code PRs weekly to update docs#22801
workflow: check TiDB code PRs weekly to update docs#22801hfxsd wants to merge 433 commits intopingcap:masterfrom
Conversation
ti-chi-bot
Bot
added
the
missing-translation-status
ti-chi-bot
Bot
added
the
size/XL
hfxsd
added
translation/no-need
hfxsd
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a Python script to automate the weekly identification of merged TiDB PRs that may require documentation updates based on labels, keywords, and file paths. Feedback includes addressing a logic error in the time window calculation to avoid timezone mismatches and overlapping reports, using word boundaries for keyword matching to prevent false positives, removing redundant keywords, and implementing error handling for GitHub API requests.
| start_date = start_sh.date().isoformat() | ||
| end_date = end_sh.date().isoformat() | ||
|
|
||
| query = f"repo:{SOURCE_REPO} is:pr is:merged merged:{start_date}..{end_date}" |
There was a problem hiding this comment.
The current search query uses date strings (e.g., 2023-10-23..2023-10-30), which has two significant issues:
- Timezone Mismatch: GitHub interprets these dates in UTC. Since your window is calculated in Shanghai time (UTC+8), PRs merged between 00:00 and 08:00 Shanghai time on the start date will be missed (as their UTC date is the previous day).
- Overlapping Reports: The
..syntax in GitHub search is inclusive. A PR merged on theend_datewill be included in this week's report and also in next week's report (where it will be thestart_date).
To fix this, use full ISO 8601 timestamps with timezone offsets and ensure the range is exclusive of the end time.
References
- Technical accuracy and terminology consistency are important for documentation workflows. (link)
| score -= 1 | ||
| reasons.append(f"Only maintenance labels: {', '.join(hit_negative_labels)}") | ||
|
|
||
| kw_hits = sorted({kw for kw in POSITIVE_KEYWORDS if kw in text}) |
There was a problem hiding this comment.
The substring matching kw in text can lead to false positives for short keywords. For example, "api" will match "capital" or "rapid", and "sql" will match "mysql". Consider using regular expressions with word boundaries (\b) to ensure keywords are matched as whole words only.
| with urllib.request.urlopen(req, timeout=30) as resp: | ||
| return json.loads(resp.read().decode("utf-8")) |
There was a problem hiding this comment.
The urllib.request.urlopen call lacks error handling. If the GitHub API returns a non-200 status code (e.g., 403 due to rate limiting or 401 due to an invalid token), the script will raise an unhandled HTTPError. Consider adding a try-except block to provide a more informative error message or handle retries for transient issues.
ti-chi-bot
Bot
added
the
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Update the weekly docs-check workflow to scan merged code PRs across the pingcap organization instead of only pingcap/tidb. Introduce SOURCE_ORG and EXCLUDED_REPOS env vars (defaulting to pingcap/docs and pingcap/docs-cn), update job/name strings, commit/title templates, and the PR body/link to reflect the org-wide scan and excluded repos.
Add automation to apply heuristic weekly doc updates to docs-cn. - Add scripts/apply_weekly_docs_cn_updates.py: reads the scan JSON, maps PRs/paths to target docs, and appends a "weekly code sync" section to matched docs; writes a summary to docs-cn/weekly-doc-sync/applied-doc-updates.json. - Update workflow .github/workflows/tidb-pr-weekly-doc-check.yml to copy the scan JSON, run the new script when updates are needed, and include the update summary in the created PR (also adjust PR copy text). - Update scripts/check_tidb_prs_and_create_docs_cn_pr.py to include changed_files in the PR payload so the apply script can resolve target docs. This automates applying simple, heuristic doc notes from the weekly PR scan and generates a summary for maintainers to review.
ti-chi-bot
Bot
added
size/XXL
Refactor the weekly doc-check workflow into a scan job that discovers candidate PRs and a downstream create-pr-per-source job that processes each candidate in parallel. The scan job now emits outputs (needs_update, candidates_count, candidates_matrix, paths, window dates) and uploads a JSON/report artifact; a TARGET_BRANCH_MAP env var is introduced to map source base branches to docs target branches. apply_weekly_docs_cn_updates.py was rewritten to apply updates for a single source PR, add a per-PR marker/note, and write an applied-<repo>-<pr>.json summary. check_tidb_prs_and_create_docs_cn_pr.py now accepts TARGET_BRANCH_MAP, records source_base_branch for PRs, builds a matrix of candidates, and writes candidates_count/candidates_matrix outputs for the workflow.
Add an EXTRA_REPOS env var to the workflow and job so additional repos (default tikv/tikv,tikv/pd) can be passed into the doc-check job. Parse EXTRA_REPOS in the script and merge it into the list of source repositories, then remove any EXCLUDED_REPOS before returning the final sorted repo list. This allows including extra repositories in the weekly TiDB PR documentation checks.
1 participant
First-time contributors' checklist
What is changed, added or deleted? (Required)
This PR redesigns the weekly docs sync automation from a single aggregated docs-cn PR into a per-source-PR workflow.
What it does:
Problems this solves:
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?