flash: add multi-target flash asset generation#168
Merged
Conversation
keerthi-go
requested review from
bjordiscollaku,
simonbeaudoin0935 and
vishwasudupa
as code owners
shoudil
reviewed
Jun 3, 2026
Add parse-boards-config.py, download-boot-bins.sh, and gen-flash-dirs.sh to support building flash_<board>_<storage>/ directories for multiple hardware targets sharing the same system image. Supports both the new multi-board boards: list in qcom-product.conf and legacy single-board flat keys for backward compatibility. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Rewrite gen-flash-dirs.sh for correct per-storage system image placement: output is <target>/<storage>/, rootfs.img at root, efi.bin sector-size matched (4096 for ufs/nvme, 512 for emmc), dtb.bin in spinor always and in other storages only when no spinor exists for that target. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Move parse-boards-config.py, download-boot-bins.sh, and gen-flash-dirs.sh from bootloader/ into a new flash/ directory dedicated to flash asset assembly. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Rename schema keys for clarity: boards: -> targets: boot_binaries_url: -> boot_bin_url: qcom_ptool_platform: -> ptool_platform: cdt_binaries_url: -> cdt_url: Parser retains backward compat with old flat key names. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Relative paths for --ptool-dir, --boot-bins-dir, --system-images, and --output-dir become invalid after pushd into PTOOL_WORK. Resolve all to absolute paths immediately after argument validation. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
The workflow pins qcom-ptool to commit 7b33652 where the scripts (gen_partition.py, ptool.py, gen_contents.py) live at the repo root, not under qcom_ptool/. Remove the qcom_ptool/ subdirectory prefix from all three invocations. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Some boot binary archives (e.g. Hamoa) already contain partition_<storage>/ subdirectories with both partition files and boot binaries pre-combined, rather than flat boot binaries requiring a separate ptool run. Add pre_partitioned: true target key. When set: - Storage types are discovered from partition_<storage>/ dirs inside the extracted archive instead of from qcom-ptool platforms/ - partition_<storage>/ contents are copied directly into <target>/<storage>/ skipping ptool generation - efi.bin and dtb.bin are still placed per Table 1 rules Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Fix 1: Post-process rawprogram*.xml in nvme/ufs/emmc to rewrite rootfs.img -> ../../rootfs.img so qdl resolves it relative to the XML location. Avoids copying the bulky rootfs.img into every flash subdir. Fix 2: HAS_SPINOR confirmed reset per loop iteration — dtb.bin now correctly placed in qcm6490-idp ufs/emmc since HAS_SPINOR stays false for that target. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
For pre-partitioned targets, run gen_partition.py on the spinor partitions.conf to produce partition_spinor.xml as input to gen_contents.py (no full ptool.py run needed). After gen_contents.py writes contents.xml, patch OS storage file_path values from template defaults (NVME/, UFS/) to actual per-target paths (<board>/nvme/, <board>/ufs/). Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Add optional contents_xml_in key to the targets: schema. parse-boards-config.py emits it in the JSON array. gen-flash-dirs.sh already passes --contents-xml-in from the workflow; no script change needed here. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Pre-partitioned archives ship rawprogram XMLs with
filename="" for OS partitions. Patch them after copy:
nvme/ufs/emmc: label=efi -> efi.bin,
label=rootfs -> ../../rootfs.img
spinor: label=dtb_a/dtb_b -> dtb.bin
Non-pre-partitioned targets already have filenames set
by ptool partition_map so these seds are no-ops for them.
Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Follow same rule as dtb.bin placement: - Target WITH spinor: one contents.xml in <target>/spinor/ with OS paths patched to ../nvme/, ../ufs/, ../emmc/ - Target WITHOUT spinor: one contents.xml per storage dir using the per-storage contents.xml.in from qcom-ptool Template resolution order for spinor targets: 1. contents_xml_in key from BOARDS_JSON 2. --contents-xml-in arg (CONTENTS_XML_IN env) 3. qcom-ptool platforms/<ptool>/spinor/contents.xml.in Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
CX_PRE and CX_CXI jq expressions had unescaped inner quotes causing a jq parse error at runtime. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
contents.xml templates reference rootfs.img at the storage dir (. or ../nvme/) but rootfs.img lives at ../../rootfs.img relative to any storage subdir. Patch the file_path element after <file_name>rootfs.img</file_name> to ../../ in both spinor and no-spinor contents.xml after generation. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Change 1: parse-boards-config.py rewritten as 25-line json.load() — replaces 130-line custom YAML parser. Accepts targets.json directly; emits [] if absent. Change 2: gen-flash-dirs.sh always runs ptool for all storage types. Storage discovery always uses qcom-ptool platforms/. pre_partitioned branches removed from storage discovery and ptool generation. Change 3: gen-flash-dirs.sh copies boot bins first, runs ptool second — ptool naturally overwrites archive partition files. copy_boot_bins() allowlist removed. Boot bin location auto-detected: partition_<storage>/ subdir if present, else flat archive root. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
Replace two python3 -c inline blocks that patched rootfs.img file_path in contents.xml with a single xmlstarlet ed command. xmlstarlet is XML-aware and handles both the plain file_path and flavor-attributed file_path cases with one XPath expression. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
gen-flash-dirs.sh: - rootfs.img missing is now exit 1 not WARN - SUPPORTED_STORAGES empty -> exit 1 with clear message - rawprogram XML patching: sed -> xmlstarlet (robust against attribute order and multi-line formatting) - contents.xml storage path patching: sed -> xmlstarlet - CDT multiple-match warning: fix empty-string miscount (grep -c . -> wc -l with null guard) - cleanup_flash_dir: add clarifying comment that it only runs on partition_<storage>/ subdirs, not PTOOL_WORK root - pre_partitioned: update header to say documentation-only - realpath -m (GNU extension) -> mkdir -p + realpath - CX_CXI template resolution: add order comment download-boot-bins.sh: - TOP_DIRS glob -> find -type d (handles top-level files alongside subdirs correctly) - md5sum -> sha256sum for cache key naming - wget failures: explicit [ERROR] + exit 1 - stale cdt_binaries_url -> cdt_url in header comment - TOP_DIRS assumption: add clarifying comment parse-boards-config.py: - pre_partitioned: add comment explaining string not bool Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
gen-flash-dirs.sh: - Rewrite header: minimal, accurate, no table labels - Remove duplicate mkdir -p OUTPUT_DIR - Remove BOARD_COUNT_CX duplicate, reuse BOARD_COUNT - Add xmlstarlet to dependency check at startup - Fix collapsed xmlstarlet OS storage path lines - Simplify CDT block: one .bin per zip, no count/fallback - Fix SUPPORTED_STORAGES empty guard - Fix CDT_COUNT empty-string miscount (wc -l + null guard) - rootfs.img missing: WARN -> exit 1 - realpath -m -> mkdir -p + realpath - cleanup_flash_dir: add clarifying comment on scope - pre_partitioned: update header to documentation-only - CX_CXI template resolution: add order comment - Remove all Table 1/Table 2 label references - Fix rawprogram/contents.xml patching: sed -> xmlstarlet download-boot-bins.sh: - Fix stale boot_binaries_url -> boot_bin_url in comments - board -> target in description - TOP_DIRS glob -> find -type d - md5sum -> sha256sum for cache key - wget failures: explicit [ERROR] + exit 1 parse-boards-config.py: - pre_partitioned: add string-not-bool comment Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
pre_partitioned was documented but never consumed at runtime. Boot bin location is auto-detected from archive structure. Remove from schema, defaults, and header comment. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
keerthi-go
force-pushed
the
dev/multi-target-support
branch
from
cbf20b6 to
b91b620
Compare
Replace fragile xmlstarlet post-patching of rawprogram XMLs
with gen_partition.py -m partition_map, matching the
qcom-deb-images approach. Filenames are now set at partition
table generation time:
nvme/ufs/emmc: efi=efi.bin, rootfs=../../rootfs.img,
dtb_a/b=dtb.bin, cdt=cdt.bin (if present)
spinor: dtb_a/b=dtb.bin, cdt=cdt.bin (if present)
Labels not present in partitions.conf are silently ignored
by gen_partition.py. CDT resolution moved before ptool so
the cdt filename is available for the -m map.
Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
bjordiscollaku
previously approved these changes
Jun 16, 2026
bjordiscollaku
left a comment
bjordiscollaku
left a comment
Contributor
There was a problem hiding this comment.
Looks good. Post merge of https://github.com/qualcomm-linux/qcom-distro-images/pull/84, consider moving these tools into qcom-distro-images/scripts/ci/.
abickett
reviewed
Jun 16, 2026
Missing boot bins, efi.bin, dtb.bin, or CDT (when configured) all produce an unusable flash directory. Promote these from [WARN] to [ERROR] + exit 1 so failures are caught immediately rather than silently producing broken output. Remaining [WARN]: skipping spinor contents.xml when template is missing — Axiom is affected but qdl still works. Signed-off-by: Keerthi Gowda <kbalehal@qti.qualcomm.com>
|
Approving for enablement now - please add the SHA validation in followup. and we can consider move to distro-images |
abickett
approved these changes
Jun 17, 2026
bjordiscollaku
approved these changes
Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a dedicated
flash/directory with three scripts that implementdata-driven multi-target flash asset generation. All targets share the
same
rootfs.imgbuilt once at the output root; each target gets itsown
<target>/<storage>/flash directory with its own boot binaries,partition files, efi.bin, dtb.bin, and contents.xml.
New files
flash/parse-boards-config.pyReads
targets.jsonand emits a validated JSON array of target entries.If
targets.jsonis absent, emits[](non-flat-meta config — flash steps skipped).flash/download-boot-bins.shDownloads boot binary archives and CDT archives per target.
Deduplicates by URL — targets sharing the same silicon family download the ZIP only once.
flash/gen-flash-dirs.shCore assembly script:
qcom-ptool/platforms/<ptool_platform>/partition_<storage>/subdir if present, else flat archive rootcontents.xml:contents.xmlin<target>/spinor/, OS paths patched to../nvme/,../ufs/contents.xmlper storage dir, all paths at.;rootfs.imgpath patched to../../rootfs.img(shared at output root)