Metalink support: v3 (dnf) + v4 (aria2), with client geo override#1
Merged
Conversation
Serve a Metalink 4.0 document (.meta4 / application/metalink4+xml) listing the candidate mirrors for a file, ordered by preference, with the source file hashes. This enables client-side mirror selection and failover (dnf/librepo, aria2, ...) instead of the single-target 302. - New MetalinkRenderer producing the RFC 5854 XML (generator, published, file/size/hash, url with location + priority). - Triggered by the ?meta4 query param or an Accept: application/metalink4+xml header (new METALINK request type). - Hash types follow the IANA registry (sha-256/sha-1/md5); only the hashes actually computed for the file are emitted. - Client geolocation override (mainly for testing on private IPs): ?country / ?continent feed the restriction and primary-country logic, ?lat / ?lon set the client coordinates consumed by the distance-based ranking. Refs videolabs#49 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhiB9FMKecvFyETJRYY8Nb
Metalink documents are meant to let the client fail over across mirrors, so don't trim the candidate list down to 5 (as done for the redirect/json path) and don't randomize the order: treat metalink like mirrorlist, i.e. return the full list deterministically ordered by score. The redirect and JSON paths are unchanged. Refs videolabs#49 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhiB9FMKecvFyETJRYY8Nb
Real-world testing showed dnf/librepo cannot consume the RFC 5854 (v4) document: librepo only parses Metalink 3.0 (metalinker.org), i.e. <files>/<file>/<resources>/<url preference=...> with hashes wrapped in <verification> and dashless hash type names. This is the format Fedora's MirrorManager actually serves. - New Metalink3Renderer (triggered by ?metalink or Accept: application/metalink+xml) producing the v3 layout dnf expects. The v4 renderer (?meta4) is kept for aria2/wget2 and standard compliance. - Fix the <file name> to the basename (e.g. "repomd.xml") instead of the full path: librepo matches it by exact string equality, so a full path was silently rejected. Applies to both v3 and v4. - Treat metalink (v3 and v4) like mirrorlist in the selection engine so the full candidate list is returned. Validated end to end: `dnf makecache` via metalink= succeeds (exit 0, "Metadata cache created") on both Fedora dnf5 and OpenMandriva (cooker, dnf5 5.2.16), fetching repomd.xml straight from a mirror. Refs videolabs#49 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QhiB9FMKecvFyETJRYY8Nb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Metalink support to mirrorbits so clients can select mirrors, fail over and verify integrity themselves, instead of relying on a single 302 redirect.
Endpoints
?metalink→ Metalink 3.0 (metalinker.org namespace). This is the format dnf/librepo actually consumes (themetalink=directive in a.repo). This is the main deliverable for OpenMandriva: it lets dnf clients move away from the single-mirror 302 towards client-side selection and failover.?meta4→ Metalink 4.0 (RFC 5854), for Metalink-aware download clients such as aria2 / wget2 (ISO and large-file downloads). Being upstreamed separately as Add Metalink 4 (RFC 5854) support videolabs/mirrorbits#228.Both are also triggered by the matching
Accept:header (application/metalink+xml/application/metalink4+xml).Behaviour
?mirrorlist, the full candidate list is returned (no 5-mirror cap, no random reordering) so clients can fail over across all mirrors, and the per-mirrorScoreweighting is honoured (e.g. to offload abf-downloads by giving it a negative score).<file name>is the basename: librepo matches it by exact string equality, so a full path is silently rejected.sha256/sha1/md5inside<verification>; v4 uses the IANA namessha-256/sha-1/md5. Only the checksums actually computed for the file are emitted.?country/?continentfeed the restriction and primary-country logic,?lat/?lonset the client coordinates used by the distance ranking.Tested
go test ./http/... ./mirrors/...passes;go vetclean.dnf makecacheviametalink=...?metalinksucceeds on Fedora dnf5 and OpenMandriva (cooker, dnf5 5.2.16), fetching repomd.xml straight from a mirror.?meta4, downloads from a mirror and verifies the checksum.Scoredrops down the list despite geographic proximity.Notes