feat(iotedge): update Azure IoT stack to 1.6.0 and rust toolchain to 1.97.1 - #687
Conversation
JanZachmann
left a comment
There was a problem hiding this comment.
One likely build breaker in the rust 1.97.1 rename, plus two smaller things, all inline. One more finding cannot be inline because the file is not in the diff: doc/examples/iotedge-apparmor/deployment.json:18 still pins mcr.microsoft.com/azureiotedge-agent:1.5, while 1.6.0 ships image = "mcr.microsoft.com/azureiotedge-agent:1.6" in edgelet/contrib/config/linux/default.toml (installed as config.toml.default, and the same value is the commented default in config.toml.edge.template). Worth bumping the example in this PR.
What I checked and found correct: rust source and all snapshot checksums against static.rust-lang.org; the patch applies cleanly on wrynose and matches oe-core 1.97.1 file for file; RUSTVERSION ?= "1.97.1%" wins over tcmode-default.inc; both SRCREVs equal the upstream 1.6.0 tags and match edgelet/Cargo.lock; the generated manifests match the locks exactly (306 crates and 306 checksums each, plus the 26 git members the .inc files override); all retained patches still apply and every dropped-patch claim holds at the new revisions. Dropping fix_docker_engine_api_version.patch is safe: iotedge 1.6.0 asks for /v1.41 and docker-moby v29.3.0 has defaultMinAPIVersion = "1.40".
For the release note: the installed default edge agent image moves to mcr.microsoft.com/azureiotedge-agent:1.6.
JanZachmann
left a comment
There was a problem hiding this comment.
Round 2. Both round-1 blockers are fixed and I re-checked them against upstream rather than against the description: the generated oe.patch applies clean at the pinned oe-core commit 5d1aa5c8, and after applying, meta/recipes-devtools/rust/ matches upstream master except the later refactors that are not tied to the version bump, while rust-target-config.bbclass differs from master by exactly the two commits you named as skipped. Details in the threads; the vendor/*.a one is resolved.
Three things are still open, none of them a reason to hold the code as written:
The clang/LLVM 0043-LoopUnroll-… patch from the same upstream commit is still outside the patch scope, and it does reach this build through llvm-project-source. It fixes non-deterministic codegen, not wrong codegen, so it is a decision rather than a defect - see the scope thread.
doc/examples/iotedge-apparmor/deployment.json:18 still pins azureiotedge-agent:1.5 while 1.6.0 ships :1.6 as its default. Outside the diff, so there is no thread for it.
The branch is unbuilt. Both new rust hunks land in do_package and in the generated target JSON, so a build of one aarch64 and one arm32 machine is the verification that counts here. That is why this is a comment and not an approval - the code matches upstream, but nothing has executed it yet.
oe-core wrynose (OE 6.0.2) ships rustc 1.94.1, but iotedge 1.6.0's edgelet pulls sysinfo 0.39.x, whose MSRV is 1.95. Bump the toolchain instead of downgrading the dependency. The rust recipes are patched in oe-core in place via the existing kas patch mechanism rather than vendored into meta-omnect: oe-core's meta is first in BBPATH, so a vendored rust_*.bb's "require recipes-devtools/rust/ rust-source.inc" would resolve back to oe-core's old copy. The patch is derived from wrynose's own 1.94.1 recipe set, which is class-compatible - oe-core master's set is not, it carries a work-shared source refactor plus cargo_common/rust-common changes that would disrupt every rust recipe in the build. It renames the four recipes rust, cargo, libstd-rs and rust-cross-canadian from 1.94.1 to 1.97.1, updates the source sha256 and the patch list in rust-source.inc, moves the bootstrap snapshot from 1.93.0 to 1.96.0, and replaces files/ with the 1.97.1 patch set. LLVM needs no version bump; wrynose already ships 22.1.3 and rust 1.97 requires >= 20. Three content changes are taken from upstream on top of the rename. The 1.97.1 bump also deletes vendored archives from the installed library sources in all three rust_do_install variants; without it do_package fails, because 1.97 vendors the library workspace into <src>/library/vendor - which the recipe copies into the package - and dwarfsrcfiles rejects wit-bindgen's non-ELF archive. The two version-coupled rust-target-config.bbclass hunks come along as well: the aarch64 AAPCS64 non-leaf frame pointer from the same commit, and the arm/armv7 eabi/eabihf target ABI from the 1.96.0 bump. Upstream's armv8a AArch32 and MIPS class changes are not tied to a version bump and no machine we build needs them. The third reaches outside recipes-devtools/rust: the same upstream commit adds 0043-LoopUnroll-Iterate-exit-fold-candidates-in-determini.patch to the clang recipe's patch list, an LLVM backport that makes LoopUnroll and LoopPeel iterate deterministically. It is version-coupled because rust links oe-core's LLVM rather than a bundled one - rust_1.97.1.bb sets download-ci-llvm to false and points llvm-config at STAGING_BINDIR - so every rust recipe in the image draws its codegen from that tree. Upstream's 0043 number is kept because its 0042 comes from an unrelated commit that wrynose does not carry. RUSTVERSION is retargeted so the PREFERRED_VERSION_* values it feeds no longer point at the removed 1.94.1. Since kas takes a single p001 per repo, oe.patch is regenerated as the concatenation of oe_layerdir.patch and the new oe_rust-1.97.1.patch, and p001 for openembedded-core now points at it. Signed-off-by: Marcel Lilienthal <134974+mlilien@users.noreply.github.com>
Move iotedge, aziot-edged and aziot-identityd to the 1.6.0 line, with the crate manifests regenerated by cargo-bitbake. aziot-identityd goes from the 1.6.0-rc.1 commit that was pinned for the wrynose migration to the 1.6.0 release tag. Its crate list came out byte-identical to rc.1, so both recipe patches still apply unchanged. iotedge and aziot-edged move to Azure/iotedge 1.6.0. The version-specific includes are renamed and the generated recipes regenerated; the crates.io dependency set grew from 226 to 306 entries. iot_identity_service_rev stays at the aziot-identityd SRCREV above - iotedge 1.6.0's edgelet/Cargo.lock pins exactly that commit, and its 26 git members are unchanged, so the remap paths in omnect_rust_aziot-identityd_deps.bbclass remain valid. RUSTFLAGS += "-D warnings" is kept, but the three fix_linter_findings patches are dropped because upstream 1.6.0 absorbed all of them: [workspace.lints.rust] now declares the unexpected_cfgs check-cfg values, External carries #[allow(dead_code)], the dead Uid(u32) field is gone, and ossl110 is declared and set by openssl-build's build script. fix_docker_engine_api_version.patch is dropped as well. Upstream moved the Docker Engine API to v1.41, which is compatible with the v1.40 minimum of Docker 29.5, and deleted the swagger file the patch edited. Forcing v1.44 again would regress compatibility with older daemons. Requires rustc >= 1.95 for sysinfo 0.39.x; see the preceding commit. Signed-off-by: Marcel Lilienthal <134974+mlilien@users.noreply.github.com>
p001 for openembedded-core points at the generated oe.patch, so nothing references oe_layerdir.patch by name any more and editing it has no effect and raises no error. Record the rule where both readers look: a README next to the patches and a comment at the p001 entry. The note deliberately does not go into oe.patch itself - that would break the byte-equality with the concatenation it is verified against. Signed-off-by: Marcel Lilienthal <134974+mlilien@users.noreply.github.com>
The AppArmor example manifest still pinned the 1.5 runtime images while the distro now ships iotedge 1.6.0. edgeAgent and edgeHub have to track the installed IoT Edge line - Azure does not support running a 1.6 agent against a 1.5 hub - so all three pins move together rather than only the agent: azureiotedge-agent, azureiotedge-hub and the SimulatedTemperatureSensor sample module. Bare minor tags are kept, matching the existing style in the file. Example material only: nothing here is built into the image, and the CI apparmor test copies just the omnect-temp-sensor profile from this directory, not the manifest. Signed-off-by: Marcel Lilienthal <134974+mlilien@users.noreply.github.com>
The concatenation rule was documented but unchecked: editing an oe_*.patch and forgetting to regenerate left kas applying a stale oe.patch with nothing reporting it. Add a pull-request job running the same cmp the README prescribes. LC_ALL is set on the step environment rather than on cat, because the shell expands and collates the glob before cat ever runs, and glob order is the order the hunks apply in - C and en_US.UTF-8 sort oe_A/oe_a/oe_B differently. Drop the README's source table: it restated `ls kas/patches/oe_*.patch` and needed an edit whenever a patch was added or dropped. The retirement rule it carried survives as prose, and the file reads as the example it is meant to be. Signed-off-by: Marcel Lilienthal <134974+mlilien@users.noreply.github.com>
|
Round 2 items closed. Builds. One aarch64 machine ( One correction on your It is still the right hunk to carry:
Scope item: The README and CI-enforcement points from the One caveat on reproducing the arm32 result: dehndetect pulls |
The branch gate listed main, kirkstone and scarthgap, so pull requests targeting wrynose skipped the conventional-commit check entirely. Signed-off-by: Marcel Lilienthal <134974+mlilien@users.noreply.github.com>
JanZachmann
left a comment
There was a problem hiding this comment.
Approving. All three threads are resolved and nothing is open from my side.
What I checked against upstream rather than against the description, across the rounds: the generated oe.patch applies clean at the pinned oe-core commit 5d1aa5c8 and stays byte-identical to cat kas/patches/oe_*.patch; after applying, meta/recipes-devtools/rust/ matches upstream master except the refactors that are not tied to the version bump, rust-target-config.bbclass differs only by the two class commits you deliberately skipped, and common.inc only by the SPDX string, the 22.1.8 sha256sum and 0042. 0043-LoopUnroll-… is byte-identical to upstream and applies with no offset against the real llvmorg-22.1.3 sources. On the iotedge side, both SRCREVs equal the upstream 1.6.0 tags, the generated manifests match the locks exactly, every retained patch still applies and every dropped one is genuinely absorbed upstream, and all three example images have a 1.6 tag on MCR for amd64, arm32v7 and arm64v8.
The build result and the CI green are yours, not mine - I did not run the image build.
Moves the Azure IoT stack to the 1.6.0 line, which forces a Rust toolchain bump.
Three commits, reviewable independently.
feat(rust): bump toolchain to 1.97.1oe-core wrynose (OE 6.0.2) ships rustc 1.94.1, but iotedge 1.6.0's edgelet pulls
sysinfo0.39.x, whose MSRV is 1.95. Bumping the toolchain was preferred over downgrading the dependency.The rust recipes are patched in oe-core in place via the kas patch mechanism rather than vendored into meta-omnect: oe-core's
metais first inBBPATH, so a vendoredrust_*.bb'srequire recipes-devtools/rust/rust-source.incwould resolve back to oe-core's old copy.The patch is derived from wrynose's own 1.94.1 recipe set, which is class-compatible. oe-core master's set is deliberately not used — it carries a work-shared source refactor (
common-source.inc,rust-source_1.97.1.bb), the@LLVM_CONFIG_PATH@/STAGING_BINDIR_CROSSllvm-config rework, andcargo_common/rust-common/cargoclass changes that would disrupt every Rust recipe in the build. It:rust,cargo,libstd-rsandrust-cross-canadianfrom_1.94.1to_1.97.1rust-source.incfiles/with the 1.97.1 patch setUpstream does its own bumps the same way —
git mvplusrust-snapshot.inc/rust-source.incmetadata, with no content change torust_*.bb(checked for the 1.95.0, 1.96.0 and 1.96.1 bumps). The 1.97.1 bump is the exception, so three content changes are taken from it as well:rust_1.97.1.bb: delete vendored archives from the installed library sources in all threerust_do_installvariants. Without itdo_packagefails — 1.97 vendors the library workspace into<src>/library/vendor, which the recipe copies into the package, anddwarfsrcfilesrejects wit-bindgen 0.57.1's non-ELF archive (oe/package.py,source_info(), exempts only clang+LTO).meta/classes-recipe/rust-target-config.bbclass: the aarch64 AAPCS64 non-leaf frame pointer from the same commit, plus the arm/armv7eabi/eabihftarget ABI from the 1.96.0 bump. The arm half is not a breaker for us (rustc 1.97.1 permits hard-float with an unspecifiedcfg(target_abi), and every arm32 machine we build is hard-float), but it belongs to the same version-bump pair and a soft-float arm32 target would be rejected outright without it.meta/recipes-devtools/clang/:0043-LoopUnroll-Iterate-exit-fold-candidates-in-determini.patchplus itscommon.incSRC_URIline — an LLVM backport makingLoopUnrollandLoopPeeliterate deterministically. Version-coupled because rust links oe-core's LLVM rather than a bundled one (rust_1.97.1.bbsetsdownload-ci-llvm = Falseand pointsllvm-configat${STAGING_BINDIR}/${STAGING_BINDIR_NATIVE}), so every Rust recipe in the image draws its codegen from that tree.llvm-project-source.bbis the sole consumer of that patch list — it is the only recipe reachingcommon.incthroughcommon-clang.incwithout also requiringcommon-source.inc, which blanksSRC_URI. Upstream's0043number is kept: its0042comes from789d416dd8(clang-tools-extra: disable tests), which is not version-coupled and which wrynose does not carry, so keeping the number lines up on a later rebase instead of colliding.Not adopted, deliberately: upstream's armv8a AArch32 Thumb fixes and MIPS ABI class changes (not tied to a version bump, and no machine we build needs them) and the snapshot self-contained-lld patchelf loop (only fails on build hosts without a
/lib64compat path).LLVM needs no version change: wrynose already ships 22.1.3 and rust 1.97 requires only >= 20.
RUSTVERSIONis retargeted so thePREFERRED_VERSION_*values it feeds no longer point at the removed 1.94.1.Per the one-
p001-per-repo convention,kas/patches/oe.patchis regenerated as the concatenation ofoe_layerdir.patchand the newoe_rust-1.97.1.patch.feat(iotedge): update Azure IoT stack to 1.6.0Crate manifests regenerated with
cargo bitbake.iot_identity_service_revstays at the aziot-identityd SRCREV, since iotedge 1.6.0'sedgelet/Cargo.lockpins exactly that commit and its 26 git members are unchanged, so the remap paths inomnect_rust_aziot-identityd_deps.bbclassremain valid.RUSTFLAGS += "-D warnings"is kept, but four patches are dropped:fix_linter_findings-*patches — upstream 1.6.0 absorbed all of them ([workspace.lints.rust]now declares theunexpected_cfgscheck-cfg values,Externalcarries#[allow(dead_code)], the deadUid(u32)field is gone, andossl110is declared and set by openssl-build's build script)fix_docker_engine_api_version.patch— upstream moved the Docker Engine API to v1.41, compatible with the v1.40 minimum of Docker 29.5, and deleted the swagger file the patch edited. Forcing v1.44 again would regress compatibility with older daemons.docs(kas): document oe.patch generationp001for openembedded-core now points at the generatedoe.patch, so nothing referencesoe_layerdir.patchby name any more and editing it has no effect and raises no error. A newkas/patches/README.mdplus a comment at thep001entry record the rule. The note is deliberately not a header insideoe.patch, which would break byte-equality withcat kas/patches/oe_*.patch.Verification
oe.patchbyte-equalscat kas/patches/oe_*.patch, and applies to a pristine5d1aa5c806worktree withgit apply --check --whitespace=nowarn, exactly how kas applies it.0043-LoopUnroll-…is byte-identical to upstream's copy, and was test-applied against thellvm-project-22.1.3.srctarball${BASEURI}resolves to: both hunks land at the line numbers in the patch header (LoopPeel.cpp11 and 1122,LoopUnroll.cpp17 and 516) withgit apply --checkclean, so no fuzz and no offset. None of the pinnedclang/patch files touch either source file, so its position after0041rather than upstream's0042is immaterial. After applying,common.incdiffers from upstream master only by the SPDX license string, the 22.1.8sha256sumand0042.rust-snapshot.incis byte-identical,rust-source.incdiffers only by the work-shared block,rust_1.97.1.bbandrust-target-config.bbclassonly by the refactors and the deliberately-skipped hunks listed above.