io: Support 64 bits position/byte-count

[pcanal]

No description provided.

[github-actions]

Test Results

    1 files      1 suites   6h 57m 18s ⏱️
3 386 tests 3 285 ✅ 101 💤 0 ❌
3 386 runs  3 386 ✅   0 💤 0 ❌

Results for commit 4248799.

♻️ This comment has been updated with latest results.

[jblomer]

I have difficulties parsing the first sentence of the commit message of the first commit. Also, this sentence mentions three control bits. Isn't it just two (up to kByteCountMask, i.e. 0x40000000)?

We may want to swap the reading and writing commits so that the code always works correctly on small objects.

[jblomer]

If both startpos and bcnt are provided, the code may add the position to fByteCountStack (it doesn't if the serialized object doesn't have a byte count). That means that the caller of ReadVersion needs to pop from the stack, correct? This looks a bit brittle and should at least be documented.

[pcanal]

If the user does not provide the address of their bytecount variable, they will not be given the value of the byte count and thus they can not and will not call CheckByteCount.
If the user has provided the address, we assume they are using it in a call to CheckByteCount.
I will add to the doc string that the CheckByteCount call is (now) required.

[pcanal]

We may want to swap the reading and writing commits so that the code always works correctly on small objects.

Currently the reading commit is first and should be able to read small buffers ... or am I missing something?

[pcanal]

I have difficulties parsing the first sentence of the commit message of the first commit.

New wording:

io: Add data structure for 64 bit byte count.
    
In order to support 64 bits byte counts, which do not fit in the space reserved
for them in the stream (4 bytes minus 2 control bits) and to work around the
fact that the variables that holds the position and the byte count information in
Streamer functions are only 32 bits (see arguments to TBufferFile::WriteVersion
and TBufferFile::ReadVersion), we need to pass them indirectly when they needs 64 bits

Since the streaming is inherently serial, we can leverage the
sequence of calls and cache the 64 bits values in a queue.

The bytecount that can not be stored in place inside the io stream will be held
in a collection (fByteCounts) that need to be stored externally to the
buffer.

Also, this sentence mentions three control bits. Isn't it just two (up to kByteCountMask, i.e. 0x40000000)?

Correct.

[pcanal]

@jakob This PR is kept open but put on Draft mode to keep the conversation going. However because of the branch naming limitation, I have to move the testing (and merging) to a new PR: #21183 which has a cleaned up (final?) list of commits.