Skip to content

avoid memory allocation from the heap class.#2224

Merged
yu-shipit merged 2 commits into
filodb:developfrom
yu-shipit:develop
May 8, 2026
Merged

avoid memory allocation from the heap class.#2224
yu-shipit merged 2 commits into
filodb:developfrom
yu-shipit:develop

Conversation

@yu-shipit
Copy link
Copy Markdown
Contributor

@yu-shipit yu-shipit commented May 7, 2026

Pull Request checklist

  • The commit(s) message(s) follows the contribution guidelines ?
  • Tests for the changes have been added (for bug fixes / features) ?
  • Docs have been added / updated (for bug fixes / features) ?

🚀 Performance Optimization: Avoid Memory Allocation from Heap Class

Commit Details

📋 Summary

This commit introduces a significant performance optimization by removing helper functions that caused unnecessary memory allocations and replacing them with inline NaN handling logic. This change improves performance in time-series aggregation functions while maintaining correct NaN handling behavior.

🔧 Changes Made

Primary Changes

1. ✅ Removed Helper Functions to Avoid Memory Allocation

Eliminated three helper functions from QueryUtils that were causing heap allocations:

  • maxIgnoreNaN(a: Double, aTs: Long, b: Double, bTs: Long): (Double, Long)
  • minIgnoreNaN(a: Double, aTs: Long, b: Double, bTs: Long): (Double, Long)
  • lastIgnoreNaN(a: Double, aTs: Long, b: Double, bTs: Long): (Double, Long)

Why this improves performance:

  • Each function call created tuple objects (Double, Long) on the heap
  • In high-frequency aggregation operations, this caused significant GC pressure
  • Removing these functions eliminates millions of temporary object allocations

2. ✅ Inlined NaN Handling Logic

Replaced function calls with direct inline logic in MinOverTimeChunkedFunctionD:

Before (heap allocation):

val (resultValue, resultTimestamp) = QueryUtils.minIgnoreNaN(min, minTimestamp, nextVal, nextTimestamp)
min = resultValue
minTimestamp = resultTimestamp

After (no allocation):

// Handle NaN values properly for timestamp functions
if (min != min) { // min is NaN
  min = nextVal
  minTimestamp = nextTimestamp
} else if (nextVal != nextVal) { // nextVal is NaN, keep current min
  // Do nothing - skip NaN values
} else if (nextVal <= min) {
  min = nextVal
  minTimestamp = nextTimestamp
}

BREAKING CHANGES

If this PR contains a breaking change, please describe the impact and migration
path for existing applications.
If not please remove this section.

Breaking changes may include:

  • Any schema changes to any Cassandra tables
  • The serialized format for Dataset and Column (see .toString methods)
  • Over the wire formats for Akka messages / case classes
  • Changes to the HTTP public API
  • Changes to query parsing / PromQL parsing

Other information:

@yu-shipit yu-shipit changed the title avoid heap memory allocation through specilized case class. avoid memory allocation from the heap class. May 7, 2026
@yu-shipit yu-shipit merged commit 5d9de27 into filodb:develop May 8, 2026
1 check passed
yu-shipit added a commit to yu-shipit/FiloDB that referenced this pull request May 8, 2026
* avoid memory allocation from the heap class.

* fix timestamp for all NaN cases.

---------

Co-authored-by: Yu Zhang <yzhang999272@apple.com>
yu-shipit added a commit that referenced this pull request May 9, 2026
* avoid memory allocation from the heap class.

* fix timestamp for all NaN cases.

---------

Co-authored-by: Yu Zhang <yzhang999272@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants