Releases · apache/bookkeeper

Release list

Release 4.18.0 Latest

Latest

hangc0276 released this 10 Jun 21:57

release-4.18.0

cd06340

4.18.0

Release 4.18.0 includes multiple important features, improvements, bug fixes and dependency/CVE updates.

The technical details of this release are summarized below.

Breaking Changes

Require Java 17+ for building and running BookKeeper PR #4446
[improve][conf] Change the default value of readOnlyModeOnAnyDiskFullEnabled to true PR #4520
BP-69: Remove bookkeeper-slogger module PR #4764
Remove deprecated method setAllocatorPoolingConcurrenncy PR #4431
Remove deprecated class MathUtils PR #4274

Features

BP-69: Adopt slog for structured logging PR #4751
BP-69: Convert bookkeeper-common to slog (phase 1) PR #4754
BP-69: Convert stats and allocator modules to slog (phase 2) PR #4755
BP-69: Convert bookkeeper-server to slog (phase 3) PR #4756
BP-69: Convert stream/distributedlog modules to slog (phase 5) PR #4757
BP-69: Convert http, tools, benchmark and metadata-drivers to slog (phase 4+6) PR #4758
BP-69: Convert remaining SLF4J usages in non-test code to slog PR #4765
BP-69: Allow caller to pass a parent slog Logger to created/opened ledgers PR #4766
BP-69: Inherit LedgerHandle logger context in ledger operations PR #4767
BP-69: Print slog context attrs in log4j2; add OTel JSON example PR #4770
BP-68: Delete cookie as part of decommission API PR #4500
Delete cookie as part of decommission API PR #4592
BP-66: support throttling for zookeeper read during rereplication PR #4258
[BP-62] Expose batchReadUnconfirmedAsync to ReadHandle PR #4739
feat: migrate native-io implementation from C to Rust PR #4738
Migrate BookkeeperProtocol from protobuf-java to LightProto PR #4780
Migrate DataFormats from protobuf-java to LightProto PR #4779
Migrate stream non-gRPC protos to LightProto PR #4781
Migrate stream gRPC protos to LightProto PR #4783
Migrate from Yahoo DataSketches to Apache DataSketches 7.0.1 (KLL) PR #4774
Support bookie server config TCP keep-alive PR #4762
[client] new API to check if BookKeeper client is connected to metadata service PR #4342
feat: allow user config AsyncLogger in log4j2 PR #4269

Improvements

Bookie

[improve] add metrics: total_entry_log_space_bytes PR #4507
improve: Enrich GC metrics to better analyze GC behavior and the time consumption of each phase PR #4384
add metric for rocksdb getLastEntryInLedger to help find out bottleneck PR #4529
Set ThreadFactory for Executor in EntryLoggerAllocator PR #4562
Add temp dir property for native library PR #4533
Apply WRITE_BUFFER_WATER_MARK setting to child channels in BookieNetty Server PR #4337
SingleDirectoryDbLedgerStorage skip optimistic cache put sometimes PR #4306
Log only overridden config in BookieServer startup PR #4768
Remove unused journaltime PR #4006

Client

LedgerHandle: eliminate unnecessary synchronization on LedgerHandle.getLength() PR #4516
Optimize bounded batch reads by predicting entry count PR #4741
improve: Improve host address resolution logic for bookie id PR #4588
Rename the client high priority thread pool PR #4496

Tools

[improve] Update readledger command to read all the entries from a bookie when first entry and last entry ids are not provided PR #4692
Optimize ListActiveLedgersCommand PR #4602
Optimize bookie usage information while using command line option --help PR #4241

Bugs

Bookie

fix BookKeeper can't startup cause by 'IOException: Recovery log xxx is missing' PR #4740
Prevent double flush due to race in SingleDirectoryDbLedgerStorage PR #4305
Fix RocksDB configuration path handling on Windows PR #4407
Issue 4503: Added check into BufferedChannel's read to avoid endless loop PR #4506
[Fix] ConcurrentLongHashMap throw ArrayIndexOutOfBoundsException PR #4771
Add workaround for ZOOKEEPER-3825 related to numeric IPs and DNS names with multiple numeric IPs and support on Java 17 PR #4719
Fix async log appender not print error log when bookie starting exceptionally PR #4475
fix: throw reject when SingleThreadExecutor drainTo in progress and queue is empty PR #4488

Client

Fix batch reads hanging after digest mismatch retries are ignored PR #4789
Fix WriteSet use-after-recycle in sequence reads PR #4788
Fix NPE in PendingAddOp.maybeTimeout() when clientCtx is null after recycling PR #4760
Fix race condition NPE in V3 response handling during timeout check PR #4737
[client][fix] Bookie WatchTask may be stuck PR #4481
Fix LedgerHandle batchReadUnconfirmedAsync: use slog log instead of LOG PR #4782
#4574: fix sanity check asyncBatchReadEntries PR #4579
fix: Add strict hostname resolution and loopback check for advertised address PR #4595
[fix] Change log level to debug when check parameters in BatchRead PR #4485

Others

[FIX] Fix IllegalThreadStateException in ComponentStarter shutdown hook PR #4733
Fix potential jetcd-core shading problem PR #4526
Fix potential class conflict during jetcd-core-shaded shading process PR #4532
compatible fix wrong key for getHttpServerTrustStorePassword PR #4301
[fix] fix string format for exception in RocksdbKVStore PR #4448
fix: permission denied in the docker image PR #4464
[fix] Fix issue where options for sanity test command are ignored PR #4691
Fix duplicated call zkGetLogSegmentNames in getLogSegmentNames PR #4050
Fix the pid occupied check when use bookkeeper-daemon.sh start or stop PR #3113

Build, CI and Tests

Introduce /bkbot Command to Control CI Workflow Runs via PR Comments PR #4673
[Doc] Introduce CI Workflow Management with BookKeeper Bot PR #4674
[ci] Add /bkbot rerun-failure command for job-level failure rerun PR #4746
Upgrade website to Docusaurus 3 and use Node 24 to build PR #4723
ci: add java21(LTS) Compatibility Check PR #4350
build: add java21 and windows daily build PR #4292
ci: add typo ci check and fix typos PR #4375
Improve CI: add caching for BookKeeper old release downloads and improve maven caching and error logs [PR #4722](http...

Assets 2

Release 4.17.3

zymap released this 20 Jan 08:34

release-4.17.3

d0e68ee

4.17.3

Release 4.17.3 includes multiple bug fixes and few dependency updates.

Apache BookKeeper users are encouraged to upgrade to 4.17.3 if you are using 4.17.x.
The technical details of this release are summarized below.

Bugs

Fix: Garbage collection stuck on corrupt entry log file PR #4544
[fix] Failed read entries after multiple decommissioning PR #4613
[fix]BK stays at read_only state even if the disk is empty PR #4640
Bring back the old public method name which removed from #4640 PR #4682
[fix]compile error of the file native_io_jni.c for the env jdk11 & windows PR #4665
Only stop Gc for the fill disk for DbLedgerStorage PR #4661
[fix]Wrong error code(-102) when opening a deleted ledger PR #4657
Log all the error in the GarbageCollectorThread PR #4649

Improvements

support equalsAndHashcode for BookieServerInfo PR #4686
Supports configuring TCP Keepalive related parameters in Bookie Client. PR #4683
add rate limit for zk read rate in gc. PR #4645

Dependency updates

Upgrade netty to 4.1.130 Final PR #4699
[security] Replace net.jpountz.lz4:lz4 with at.yawk.lz4:lz4-java PR #4696
Upgrade OpenTelemetry to 1.56.0, Otel instrumentation to 2.21.0 and Otel semconv to 1.37.0 PR #4690
Exclude commons-collections from dependencies PR #4689
Remove commons-configuration2 and commons-beanutils from top level dependencies PR #4648

Full Changelog: release-4.17.2...release-4.17.3

Assets 2

Release 4.17.2

StevenLuMT released this 05 Jul 22:23

release-4.17.2

75f65df

Release 4.17.2 includes multiple bug fixes and few dependency updates.

Apache BookKeeper users are encouraged to upgrade to 4.17.2 if you are using 4.17.x.
The technical details of this release are summarized below.

Highlights

Bugs

Fix a NPE bug after refactor recycler of BookieClient PR #4610
Fix Memory Leak In Netty Recycler of Bookie Client PR #4609
Fix the data loss issue that caused by the wrong entry log header PR #4607
Fix the coredump that occurs when calling KeyValueStorageRocksDB.count after rocksdb has been closed PR #4581
Fix and improve locating RocksDB config files PR #4560
[fix] Write stuck due to pending add callback by multiple threads PR #4557
Fix SST files not being cleaned up in the locations folder PR #4555
Fix tune-runner-vm action to run correctly on ubuntu-24.04 image PR #4536
[cli] Fix: recover command doesn't accept rate limit parameter PR #4535
fix pendingDeletedLedgers do not remove ledger error PR #4525
Fix region aware placement policy disk weight dose not update. PR #4522
Fix check read failed entry memory leak issue. PR #4513
[fix] remove in address2Region while bookie left to get correct rack info PR #4504
fix: install netcat-openbsd instead of netcat in test image build PR #4476
[fix][ci] Fix OWASP Dependency Check download by using NVD API key PR #4473
Fix ReadOnlyLedgerHandle leak issue when checkAllLedgers. PR #4468
fix[rocksdb]: fix error rocksdb default config for CFOptions PR #4466
[fix] Fix data lost after when writing ledger and deleting legder execute concurrency PR #4462
[BugFix] Fix to prevent resource leaks in 3 classes PR #4449
Correct RackawareEnsemblePlacementPolicyImpl defaultRack when the bookie is not available. PR #4439
Fix the completionObjects leak problem. PR #4285
Issue 2161: set metrics endpoint content-type PR #4208

Improvements

Adjust DNS cache expiration in tests to reduce test timeout failures PR #4586
Change the new ensemble log to info level PR #4566
Add documentation to bk_server.conf about RocksDB config PR #4561
Reduce metadataLock contention in LedgerHandle PR #4549
[improve] when rackaware failed to choose a bookie, print out the list of ensemble. PR #4482
Ensure that formatVersion is specified for all RocksDB databases PR #4559
Set default format_version to 5 for RocksDB databases PR #4480
[improve] Optimize reorderReadSequence to Check WriteSet PR #4478
Update Release Script Instructions and Python Publishing Scripts PR #4458
Enable ZooKeeper client to establish connection in read-only mode PR #4244
Allocator support exitOnOutOfMemory config. PR #3984

Dependency updates

Upgrade commons-beanutils to 1.11.0 to address CVE-2025-48734 PR #4608
Migrate deprecated commons-configuration 1.x to commons-configuration 2.x PR #4604
Upgrade Jetty to 9.4.57.v20241219 to mitigate CVE-2024-6763 PR #4600
Upgrade Netty to 4.1.121.Final PR #4597
Upgrade to grpc 1.72.0 PR #4591
Upgrade OpenTelemetry version and align versions using BOMs PR #4589
Upgrade Netty to 4.1.119, tcnative to 2.0.70 and io_uring to 0.0.26 PR #4584
Upgrade Apache Commons libraries to compatible versions PR #4582
Bump vertx.version from 4.5.7 to 4.5.11 to address CVE-2024-8391 PR #4545
Upgrade to Netty 4.1.115.Final to address CVE-2024-47535 PR #4524
Upgrade Zookeeper to 3.9.3 to address CVE-2024-51504 PR #4523
Upgrade gRPC to 1.70.0 PR #4512
Upgrade protobuf to 3.25.5 to address CVE-2024-7254 PR #4508
Upgrade to Netty 4.1.113.Final and netty-tcnative 2.0.66.Final PR #4502
Bump grpc from 1.56.0 to 1.64.0 to address CVE list PR #4344

Details

https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.17.2+is%3Amerged+

Assets 2

Release 4.16.7

StevenLuMT released this 03 Jul 09:19

release-4.16.7

aea1fea

Release 4.16.7 includes multiple bug fixes and few dependency updates.

Apache BookKeeper users are encouraged to upgrade to 4.16.7 if you are using 4.16.x.
The technical details of this release are summarized below.

Highlights

Bugs

Fix the data loss issue that caused by the wrong entry log header PR #4607
Fix the coredump that occurs when calling KeyValueStorageRocksDB.count after rocksdb has been closed PR #4581
Fix and improve locating RocksDB config files PR #4560
[fix] Write stuck due to pending add callback by multiple threads PR #4557
Fix SST files not being cleaned up in the locations folder PR #4555
Fix tune-runner-vm action to run correctly on ubuntu-24.04 image PR #4536
[cli] Fix: recover command doesn't accept rate limit parameter PR #4535
fix pendingDeletedLedgers do not remove ledger error PR #4525
Fix region aware placement policy disk weight dose not update. PR #4522
Fix check read failed entry memory leak issue. PR #4513
[fix] remove in address2Region while bookie left to get correct rack info PR #4504
fix: install netcat-openbsd instead of netcat in test image build PR #4476
[fix][ci] Fix OWASP Dependency Check download by using NVD API key PR #4473
fix[rocksdb]: fix error rocksdb default config for CFOptions PR #4466
[fix] Fix data lost after when writing ledger and deleting legder execute concurrency PR #4462
[BugFix] Fix to prevent resource leaks in 3 classes PR #4449
Correct RackawareEnsemblePlacementPolicyImpl defaultRack when the bookie is not available. PR #4439
Fix the completionObjects leak problem. PR #4285

Improvements

Adjust DNS cache expiration in tests to reduce test timeout failures PR #4586
Change the new ensemble log to info level PR #4566
Add documentation to bk_server.conf about RocksDB config PR #4561
Reduce metadataLock contention in LedgerHandle PR #4549
[improve] when rackaware failed to choose a bookie, print out the list of ensemble. PR #4482
Ensure that formatVersion is specified for all RocksDB databases PR #4559
Set default format_version to 5 for RocksDB databases PR #4480
[improve] Optimize reorderReadSequence to Check WriteSet PR #4478
Update Release Script Instructions and Python Publishing Scripts PR #4458
Enable ZooKeeper client to establish connection in read-only mode PR #4244
Improve auto-recovery noise log when some bookie down. PR #4118
Allocator support exitOnOutOfMemory config. PR #3984

Dependency updates

Upgrade Jetty to 9.4.57.v20241219 to mitigate CVE-2024-6763 PR #4600
Upgrade Netty to 4.1.121.Final PR #4597
Upgrade Netty to 4.1.119, tcnative to 2.0.70 and io_uring to 0.0.26 PR #4584
Upgrade Apache Commons libraries to compatible versions PR #4582
Bump vertx.version from 4.5.7 to 4.5.11 to address CVE-2024-8391 PR #4545
Upgrade to Netty 4.1.115.Final to address CVE-2024-47535 PR #4524
Upgrade Zookeeper to 3.9.3 to address CVE-2024-51504 PR #4523
Upgrade protobuf to 3.25.5 to address CVE-2024-7254 PR #4508
Upgrade to Netty 4.1.113.Final and netty-tcnative 2.0.66.Final PR #4502

Details

https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.16.7+is%3Amerged+

Assets 2

Release 4.17.1

hezhangjian released this 26 Jun 13:44

release-4.17.1

f233320

Release 4.17.1 includes multiple bug fixes and few dependency updates.

Apache BookKeeper users are encouraged to upgrade to 4.17.1 if you are using 4.17.x.
The technical details of this release are summarized below.

Highlights

Bugs

Fix TimedRunnable log NPE PR #4425
Fix Auditor ignoring bookies shut down before Auditor start PR #4419
Fix lost prometheus metric in OrderedExecutor PR #4374
Fix: resource leak when JournalChannel is not fully initialized PR #4340
Fix: bookie http endpoint info always return 0.0.0.0 PR #4325
Fix disk weight ensemble infinite loop bug PR #4324
Fix guava shade error in distributedlog PR #4319
Fix ThreadRegistry#register behavior to ensure correct Prom metrics PR #4300
Fix: reference counting (retain/release) in PerChannelBookieClient PR #4293
Fix ByteBuf release/retain in PerChannelBookClient PR #4289
Tests: miss test log in prometheus-metrics-provider module PR #4279
Fixed creation of temporary dir in NativeUtils PR #4262
Fix ArrayIndexOutOfBoundsException caused by optimistic lock PR #4066
Prevent bookie shutdown due to rest api when bookie prohibits readOnlyMode PR #3972
Fix wrong implementation for percentile in bookkeeper-benchmark PR #3864

Improvements

Adjust Log Level for LedgerFencedException in WriteEntryProcessor PR #4327
Improve: change scheduleAtFixedRate to scheduleWithFixedDelay in GarbageCollectorThread PR #4296
Remove unused code from ByteBufVisitor PR #4383
Use vertx blockingHandlers to run Bookkeeper http handlers which could be blocking PR #4266

Dependency updates

Bump jetcd from 0.5.0 to 0.7.7 PR #3849
Bump netty from 4.1.107.Final to 4.1.108.Final to address CVE list PR #4426
Bump jackson from 2.13.4.20221013 to 2.17.1 to address CVE list PR #4345
Upgrade vertx-core to 4.5.7 to address CVE-2024-1300 PR #4265

Details

https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.17.1+is%3Amerged+

Assets 2

Release 4.16.6

hezhangjian released this 26 Jun 13:43

release-4.16.6

06eca72

Release 4.16.6 includes multiple bug fixes and few dependency updates.

Apache BookKeeper users are encouraged to upgrade to 4.16.6 if you are using 4.16.x.
The technical details of this release are summarized below.

Highlights

Bugs

Fix TimedRunnable log NPE PR #4425
Fix Auditor ignoring bookies shut down before Auditor start PR #4419
Fix lost prometheus metric in OrderedExecutor PR #4374
Fix: resource leak when JournalChannel is not fully initialized PR #4340
Fix: bookie http endpoint info always return 0.0.0.0 PR #4325
Fix disk weight ensemble infinite loop bug PR #4324
Fix guava shade error in distributedlog PR #4319
Fix ThreadRegistry#register behavior to ensure correct Prom metrics PR #4300
Fix: reference counting (retain/release) in PerChannelBookieClient PR #4293
Fix ByteBuf release/retain in PerChannelBookClient PR #4289
Tests: miss test log in prometheus-metrics-provider module PR #4279
Fix ArrayIndexOutOfBoundsException caused by optimistic lock PR #4066
Prevent bookie shutdown due to rest api when bookie prohibits readOnlyMode PR #3972
Fix wrong implementation for percentile in bookkeeper-benchmark PR #3864

Improvements

Adjust Log Level for LedgerFencedException in WriteEntryProcessor PR #4327
Improve: change scheduleAtFixedRate to scheduleWithFixedDelay in GarbageCollectorThread PR #4296
Remove unused code from ByteBufVisitor PR #4383
Use vertx blockingHandlers to run Bookkeeper http handlers which could be blocking PR #4266

Dependency updates

Bump jetcd from 0.5.0 to 0.7.7 PR #3849
Bump netty from 4.1.94.Final to 4.1.108.Final to address CVE list PR #4426
Bump jackson from 2.13.4.20221013 to 2.17.1 to address CVE list PR #4345
Upgrade vertx-core to 4.5.7 to address CVE-2024-1300 PR #4265

Details

https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.16.6+is%3Amerged+

Assets 2

Release 4.17.0

hezhangjian released this 25 May 02:44

release-4.17.0

cce4b64

Release 4.17.0 includes multiple important features, improvements, bug fixes and some dependencies CVE fixes.

The technical details of this release are summarized below.