Proposal: Hard-fork vnl — drop ITK_USE_SYSTEM_VXL and stop tracking upstream vxl

[hjmjohnson]

Proposal

Stop supporting ITK_USE_SYSTEM_VXL (external vxl) and stop tracking
upstream vxl/vxl for the vendored copy under Modules/ThirdParty/VNL/.
Treat ITK's vnl as a hard fork — vendored, pruned, and evolved on
ITK's schedule and ITK's needs alone.

NOTE: Pruning VXL of items not needed by ITK (2208 → 1115 files, ~50% reduction) greatly reduces the maintenance surface area, and greatly reduces the cmake complexity.

Motivation

A symbol-level audit of ITK's vxl usage shows a narrow, concentrated
footprint:

• ITK uses only core/vnl, core/vnl/algo, core/vcl (already
  zero vcl_* includes in ITK source — fully migrated), and
  v3p/netlib.
• Zero references in ITK to core/vbl, core/vgl, core/vul,
  core/vil, core/vsl, core/vpl, or any of contrib/.
• Top consumers (mention counts in ITK source): vnl_vector (405),
  vnl_matrix (202), vnl_vector_fixed (81), vnl_matrix_fixed
  (73), vnl_nonlinear_minimizer (50), vnl_math (45),
  vnl_lbfgsb (21), vnl_svd (19), vnl_lbfgs (18),
  vnl_matrix_inverse (17), vnl_quaternion (15), vnl_amoeba
  (14), vnl_cost_function (14), vnl_sparse_matrix (14),
  vnl_symmetric_eigensystem (6), vnl_fft_base (7).
• ITK's existing VXL_BUILD_CORE_NUMERICS_ONLY=ON already restricts
  the build to roughly this set, but the source tree under
  Modules/ThirdParty/VNL/src/vxl/ still ships everything.

A survey of the 124 visible forks of vxl/vxl shows essentially no
divergent downstream community. Every ITK-contributor fork
(hjmjohnson, thewtex, blowekamp, N-Dekker, bradking,
dzenanz, seanm) is 0 commits ahead of upstream — pure PR-staging
mirrors. The only meaningfully divergent fork is one researcher's
computer-vision fork (rfabbri/vxl-1). Outside ITK, the active
contributor base is one other organization with computer-vision
priorities that diverge from ITK's medical-imaging-numerics priorities.

The two groups end up burdening each other:

• ITK carries only config, core, v3p, vcl in the source tree, but changes to these core have often required upstream VXL changes in vbl/vgl/vul/vil/vsl/vpl/contrib that ITK does not use, build, or test.
• Upstream vxl carries API and ABI commitments around vnl that ITK
  doesn't need but cannot break unilaterally.
• Cross-org review burden on PRs that one side cares about and the
  other tolerates.

What this unlocks

Treating vnl as ITK-owned vendored code (no external option, no
upstream tracking) gives us freedom to:

1. Aggressively prune. Delete every header outside the closure of
   what ITK actually compiles. The source tree shrinks by a large
   majority.
2. Hide / shrink the public surface. Most of the vnl API is not
   exposed in ITK's public headers. The unexposed portion can be
   rewritten, renamed, made internal::, or replaced behind ITK's
   own thin facades without backward-compat concerns.
3. Stage a migration to Eigen and std.
   • vnl_math → <numbers> / <cmath>
   • vnl_*_fixed<T,N,M> → Eigen::Matrix<T,N,M>
   • vnl_matrix / vnl_vector → Eigen::MatrixXd / VectorXd
   • vnl_svd / vnl_symmetric_eigensystem / vnl_sparse_matrix →
     Eigen equivalents
   • Optimizers (vnl_lbfgs, vnl_lbfgsb, vnl_amoeba,
     vnl_levenberg_marquardt, vnl_powell,
     vnl_conjugate_gradient) phase last — they need a numerical
     regression suite before swap because ITK's registration
     framework depends on their exact behavior.
4. Land performance improvements (alignment, SIMD, view types,
   move-only semantics, allocator-aware containers) that would be
   ABI/API-breaking upstream.

What this costs

• ITK is no longer a consumer of upstream vxl bug fixes. We accept
  that — the bug-fix flux from upstream into ITK's slice has been
  small in recent years.
• ITK contributors who maintain the 0-ahead mirror forks lose the
  ability to round-trip a vnl fix through both projects. In practice
  these PRs have been rare and can still be done by hand if an
  upstream-relevant fix surfaces.
• ITK_USE_SYSTEM_VXL users (if any exist in production) need a
  migration path. A deprecation cycle of one minor release is
  reasonable.

Proposed sequence

1. Decision on this issue (community discussion, vote if needed).
2. Deprecate ITK_USE_SYSTEM_VXL in the next minor release with a
   warning at configure time.
3. Remove the option in the release after that. Vendored copy
   becomes the only path.
4. Prune the vendored tree to the closure of what ITK builds.
5. Begin migration in priority order: vnl_math and
   vnl_*_fixed first (mechanical), vnl_matrix/vector second
   (bulk but mechanical), solvers third, optimizers last with
   numerical regression coverage.

Discussion points

• Are there ITK_USE_SYSTEM_VXL users in production we need to survey?
• Any objection to the rename of the vendored module from "VXL" to
  something more honest about its scope (e.g., "VNLForITK") once
  the prune lands?
• Should the optimizer phase be coordinated with the registration
  framework's own modernization roadmap?

[hjmjohnson]

Five-year contributor analysis (2021-05-11 → 2026-05-11)

• The disjointness is striking: core/vnl got 839 ITK touches vs 2 VSI touches; contrib/brl got 224 VSI touches vs 0 ITK touches. The two communities literally don't share working files — the only coupling is API compatibility on a code path that ITK developers maintain for the other's benefit.
• VSI's commit count being higher than ITK's (215 vs 165) initially looks like "VSI carries more weight," but file-touch and LOC churn flip that 4.6× the other way.

This is the evidence behind the "two communities, divergent needs" claim
in the proposal above. Data computed from vxl/vxl@master, non-merge
commits only.

Commits by affiliation group

Group	Commits	Files touched	Insertions	Deletions
ITK-affiliated	165	3,750	110,803	79,511
Vision Systems Inc (VSI)	215	813	37,577	24,302
Other / packagers	8	11	40	16

ITK has fewer commits, but they are almost all in the foundational core/vnl support files ~4.6× the file-touches and ~3× the LOC churn — characteristic of tree-wide modernization (C++17 migration, deprecation
removal, header hygiene) rather than feature work.

Per-contributor breakdown

ITK-affiliated (159 commits, 96% of "ITK" group):

Author	Affiliation	Commits
Hans Johnson	U. Iowa / ITK maintainer	131
Sean McBride	ITK contributor	10
Niels Dekker	LUMC / ITK contributor	8
Andrei Sandor	McGill / ITK-adjacent	5
Dženan Zukić	Kitware / ITK maintainer	4
Tom Vercauteren	KCL / ITK-adjacent	2
Matt McCormick	Kitware / ITK maintainer	1
Brad King	Kitware (CMake/ITK)	1
Bradley Lowekamp	NIH / ITK maintainer	1
Jon Haitz Legarreta	ITK contributor	1
David Stoup	Kitware	1

Vision Systems Inc (215 commits):

Author	Commits
Joseph Mundy	114
Drew Gilliam	68
Nathan Harbison	18
Eben Aceto	5
Calvin Bauer	4
Daniel Crispell	4
Andy Neff	1
mundyj	1

Where each group works — the disjointness proof

Top directories touched (file-touch counts, non-merge commits, 5y):

Directory	ITK touches	VSI touches
core/vnl	839	2
core/vil	518	41
core/vgl	348	22
core/vgui	315	0
core/vil1	273	0
core/vbl	114	6
core/vul	107	0
core/vsl	91	0
v3p/netlib	84	2
core/vpgl	345	224
core/vrel	141	100
contrib/brl	0	224
contrib/rpl	0	119
contrib/gel	0	20

The two groups are working on essentially disjoint parts of the
repository. ITK does not maintain contrib/* at all. VSI touches
core/vnl (the only directory ITK ships) exactly twice in five years.

What this means for the proposal

1. ITK is the de-facto maintainer of core/vnl and v3p/netlib, (and of itemes ITK does not usecore/vbl, core/vul,
   core/vgui, core/vil1, core/vsl) . No one else has touched these in any volume.
2. VSI maintains contrib/brl, contrib/rpl, contrib/gel,
   core/vpgl, and core/vrel — none of which ITK consumes. Deprecation or performance improvements in core/vnl often require ITK resources to update the contrib/brl directory as well.
3. The compatibility constraint flows one way: every modernization
   ITK wants in core/vnl has to be vetted against VSI's deep use of
   that same tree (via vpgl, vrel, brl/...) even though VSI does
   not contribute back to core/vnl improvements.
4. The "preserve upstream API" tax is paid by ITK alone to keep a
   relationship VSI is not actively investing in for the core/vnl
   areas ITK ships.

A hard fork lets ITK keep maintaining / peformance improving/ ELIMINATING what it actually needs, and
lets VSI keep vxl/vxl evolving for the parts they actually use, without either group blocking the other.

Method

git log master --since=2021-05-11 --until=2026-05-11 --no-merges \
    --pretty='%ae|%an' | sort | uniq -c | sort -rn

git log master --since=2021-05-11 --until=2026-05-11 --no-merges \
    --author='<pattern>' --name-only --pretty=format: \
  | awk -F/ '{print $1"/"$2}' | sort | uniq -c | sort -rn

git log master --since=2021-05-11 --until=2026-05-11 --no-merges \
    --author='<pattern>' --shortstat

Group patterns:

• ITK: hans-johnson|hans.j.johnson|dzukic|dzenan|mccormick|brad.king|blowekamp|N.Dekker|sean@rogue|legarreta|vercauteren|sandor|david.stoup
• VSI: visionsystemsinc|vsi-ri|mundyj

[dzenanz]

This seems fine with me. VNLforITK name seems reasonable. Have you asked anyone in VSI group for their opinion about the matter?

[blowekamp]

• Zero references in ITK to core/vbl, core/vgl, core/vul,
  core/vil, core/vsl, core/vpl, or any of contrib/.

We don't have these directories in ITK. These messages and plan don't look like they were review close enough by a human before posting.

[hjmjohnson]

@blowekamp I have modified each of those directories in VXL many many times. They are not in ITK proper, but I am managing them in VXL, and in USE_SYSTEM_VXL as well. They are part of my ITK burden even though they are excluded in the ITK proper ingestion. I kept them in the message because they do contribute to the burden.

[hjmjohnson]

This seems fine with me. VNLforITK name seems reasonable. Have you asked anyone in VSI group for their opinion about the matter?

@dzenanz

I have been communicating with the VSI group regarding VXL upstream changes. They have been very receptive to the backward-compatible changes and performance improvement changes being made to VXL. To this point, a lot of effort has been taken to avoid being disruptive to the VSI group, and they are supportive of ITK improving VNL in ways that are not disruptive to them

Some of the foreseen next steps for more Eigen interactions would introduce changes to VNL that would be a lot easier to do safely within the confines of ITK alone, but would require a lot more effort to verify outside of ITK in a community-wide backward-compatible way (we can be backward compatible within ITK for the limited API surface area).

[blowekamp]

@blowekamp I have modified each of those directories in VXL many many times. They are not in ITK proper, but I am managing them in VXL, and in USE_SYSTEM_VXL as well. They are part of my ITK burden even though they are excluded in the ITK proper ingestion. I kept them in the message because they do contribute to the burden.

Thanks, that is a lot more informative than much of the message above.

ITK carries vbl/vgl/vul/vil/vsl/vpl/contrib in the source tree it
does not use, build, or test.

[hjmjohnson]

@blowekamp Update the sentence to carry that information -> "ITK carries only config, core, v3p, vcl in the source tree, but changes to these core have often required upstream VXL changes in vbl/vgl/vul/vil/vsl/vpl/contrib that ITK does not use, build, or test."

I see how the original message was both wrong and misleading without the expanded context.