Add Apache Arrow provider

[dabla]

Added Apache Arrow provider in Airflow and implemented basic AdbcHook.
The AdbcHook implements the DBApiHook, so it can be reused across all SQL related operators.
I could also be used to test integration with Apache DataFusion.

@zeroshade

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[potiuk]

Just for anyone looking here - this is a draft for discussion between me, @dabla and @zeroshade - we will still need to start a DISCUSSION thread for the new provider - and we think Arrow and ADBC is a good addition. But we have to first discuss the approach :)

[vikramkoka]

As @potiuk mentioned, I believe this needs a devlist conversation first.
Looking forward to it

[potiuk]

As @potiuk mentioned, I believe this needs a devlist conversation first. Looking forward to it

Yep. This one is mostly to gather learnigs, get feedback from @zeroshade and see how we can turn it into a "convincing" devlist proposal - by showing some use cases and small POC of implementation and what it allows :).

We'll experiment a bit with it and gather our thoughts and see what can come out of it.

[github-actions]

uv.lock on main just moved via commit 4fe401c and this PR currently conflicts.

Quickest fix:

git fetch upstream main && git rebase upstream/main
rm uv.lock && uv lock
git add uv.lock && git rebase --continue
git push --force-with-lease

Automated nudge — ignore if you're not ready to rebase. This comment is updated in place on future uv.lock bumps.

[potiuk]

Did a full pass over the new provider — structurally it's a clean standard-template provider, licenses/deps are permissive (adbc-driver-*/pyarrow are Apache-2.0), headers are present, provider.yaml wiring is correct, and no raise AirflowException / no assert in the hook. Two of my earlier points: the extras-passthrough ask is now handled via named db_kwargs/conn_kwargs keys (the safe allowlist form — good); and ignore my earlier --keywords suggestion, the dot form (apache.arrow) is the template convention every provider uses, so pyproject.toml is correct as-is.

Not approving yet — the main gap is test coverage on the extras feature itself (details inline), plus a few small fixes. A couple of smaller observations beyond the inline notes:

• hooks/adbc.py — from more_itertools import chunked is a direct import, but more-itertools isn't a declared dependency of this provider (only transitive via common-sql). Either add it as a direct dependency or use common_sql/an alternative.
• tests/.../test_adbc.py — a few mock.MagicMock() are created without spec=, inconsistent with the spec=Cursor usages elsewhere in the same file; AGENTS asks for spec/autospec when mocking.

Also: please rebase onto latest main — new providers churn uv.lock/root pyproject.toml, and a fresh rebase keeps CI meaningful before merge.

---

This review was drafted by an AI-assisted tool and confirmed by an Airflow maintainer. The findings below are observations, not blockers; an Airflow maintainer — a real person — will take the next look at the PR. If you think a finding is mis-applied, please reply on the PR and a maintainer will weigh in.

More on how Airflow handles maintainer review: contributing-docs/05_pull_requests.rst.