rootfs: dependency-resolved --local-debs installation, APT-based kernel delivery, and deferred bootloader update

[bjordiscollaku]

Summary

build-rootfs.sh gains a dependency-resolving install path for locally built .deb packages (--local-debs), drops the hard requirement on --kernel-package, and defers update-grub until every package-delivery path has completed. With these changes the kernel can arrive via any of three routes: direct deb (--kernel-package), overlay-manifest APT source (--overlay), or a staged local APT repository (--local-debs). In all cases the generated bootloader configuration reflects the final installed package set, regardless of which route delivered the kernel.

Two supporting changes complete the PR: a workspace-isolation fix that makes the script safe to invoke repeatedly from the same checkout (required for multi-kernel CI loops), and removal of a board-specific DTB/GRUB mutation that did not belong in this distribution-agnostic tool.

Primary consumer: the multi-kernel distro pipeline in qcom-distro-images #84, which invokes this script once per kernel variant.

Why

Four independent problems, fixed in four scoped commits:

1. Kernel ingestion was rigid and did not resolve dependencies. --kernel-package was mandatory, blocking flows where the kernel comes from APT or overlay manifests. Plain dpkg -i cannot resolve dependency graphs, which is a problem for kernel package sets that span multiple debs (for example canonical-layout kernels split across linux-image-* and linux-modules-* with inter-package dependencies).
2. update-grub ran too early. It executed immediately after the direct kernel dpkg -i, before overlay-manifest and local-debs installs. When the effective kernel arrives through one of those later paths, GRUB was generated against stale kernel state.
3. The script could delete itself. ROOTFS_DIR was $WORKDIR/rootfs, which collides with the repository's own rootfs/ source directory when invoked from the repo root. The preprocessing stage's rm -rf "$ROOTFS_DIR" could remove rootfs/scripts/build-rootfs.sh mid-run, which breaks any loop that calls the script more than once per workspace.
4. Board policy in a generic tool. A Debian-only block probed for glymur-crd.dtb, symlinked it to /boot/dtb, and injected a devicetree directive into grub.cfg. That is platform-specific boot policy hardcoded into shared rootfs assembly.

What changed (one commit per concern)

1. rootfs: isolate build workspace from source tree (dd57d03)

Runtime extraction/build directory renamed from $WORKDIR/rootfs to $WORKDIR/rootfs_work. The generated-state directory and the checked-in source directory no longer share a path, so repeated invocations in a single CI job can no longer destroy the script sources. Path-safety fix only; no other behavior change.

2. rootfs: add local-debs flow; kernel deb optional (5dd6591)

New repeatable CLI option --local-debs <path>:

• accepts a .deb file or a directory of .deb files, and may be given multiple times,
• validates every path up front (hard error on a nonexistent path; empty directories tolerated),
• stages all discovered debs into /opt/local-debs inside the rootfs.

Inside the chroot a throwaway local APT repository is built from the staged debs:

1. install dpkg-dev (--no-install-recommends),
2. generate the index with dpkg-scanpackages . /dev/null > Packages,
3. register deb [trusted=yes] file:///opt/local-debs ./ in sources.list.d/local-debs.list,
4. extract package names from deb metadata (dpkg-deb --field <deb> Package),
5. apt-get install -y <names>, so APT computes the install order and satisfies inter-package dependencies across the provided set and the configured remote sources.

Why an APT repository instead of dpkg -i: dpkg -i installs exactly what it is handed and fails on unmet dependencies. The local-repo approach lets APT resolve dependency chains (for example linux-image and linux-modules) deterministically.

In the same commit, --kernel-package becomes optional, with conditional validation, copy, and install (a skip message when absent). --product-conf and --seed remain the only required arguments. With no --local-debs, every local-repo stage is a no-op.

3. rootfs: run update-grub after all package installs (35cdcac)

update-grub (and the subsequent grub.cfg normalization and manifest-delta capture) moves from immediately after the kernel dpkg -i to after all install sources complete. The explicit invocation is kept because the zz-update-grub kernel hook skips itself in a chroot (no /run/systemd/system). The bootloader state now reflects whichever kernel was installed last, and the kernel delivery path no longer changes GRUB correctness.

4. rootfs: drop Debian-specific DTB injection (dc80c48)

Removes the glymur-crd.dtb probe, the /boot/dtb symlink, and the grub.cfg devicetree injection. DTB and boot policy belong to the target-specific layers that own board behavior (in the distro pipeline, FIT DTB generation is handled per kernel variant by qcom-distro-images CI), not to the generic image builder.

Install sequence inside the chroot (after this PR)

1. Capture base package manifest.
2. Firmware dpkg -i (if --firmware).
3. Kernel dpkg -i (if --kernel-package).
4. User and account setup.
5. Overlay manifest packages: APT sources and listed packages (if --overlay).
6. Local-debs packages via the local APT repository (if --local-debs).
7. update-grub, explicit, after all installs.
8. grub.cfg normalization (search --label system, strip root=/dev/*).
9. Post-install manifest capture and base/post delta.

CLI contract

Argument	Before	After
--product-conf	required	required
--seed	required	required
--kernel-package	required	optional
--firmware	optional	optional
--overlay	optional	optional
--variant	optional	optional
--local-debs	n/a	new, optional, repeatable (file or directory)

Compatibility

• Existing callers are unaffected: passing --kernel-package works exactly as before, and the new behavior is opt-in.
• update-grub timing: for callers whose kernel arrives only via --kernel-package (and whose overlays do not install kernels), the resulting grub.cfg is identical; it is simply generated later. For overlay or local-debs delivered kernels, the previous behavior was incorrect and is now fixed.
• The DTB injection removal is a deliberate behavioral change for Debian targets that relied on the injected devicetree directive. The only consumer (trixie/generic) is retired from the nightly matrix in qcom-distro-images #84; board DTB policy is now owned by target-specific layers.

Known trade-off (candidate follow-up)

The staged debs under /opt/local-debs and the local-debs.list APT source entry remain in the shipped image (the index stays self-consistent, so on-device apt update keeps working). This mirrors the existing behavior of the kernel deb being copied to / and left there, but it does add image payload. A cleanup pass (remove staged debs and the source entry after install) is a reasonable follow-up.

Files changed

• rootfs/scripts/build-rootfs.sh (+111/-50)

Validation

• Static: bash -n rootfs/scripts/build-rootfs.sh.
• End to end: the validation sweep on qcom-distro-images #84 was dispatched with qcom-build-utils-ref=feat/rootfs-local-debs-overlay-kernel-bootflow. Every rootfs in those runs was assembled by this script, covering both the single-deb qcom-next closure and the multi-deb canonical closure (linux-image plus linux-modules resolved by APT), across latest and pinned modes and server and desktop matrix targets.