JSON file detection with SOROBAN and Impl all done

[tzar-deek]

Soroban Storage & Footprint Analysis — Implementation Report

Overview

I implemented four new Soroban analyzers for the GasGuard codebase to detect storage inefficiencies, classify storage entries, identify footprint expansion risks, and analyze footprint access patterns. All four analyzers include comprehensive test suites with 41 tests passing.

---

Issue 1: Soroban Storage Entry Classifier

File: packages/analyzers/soroban/storage/storage-entry-classifier.ts (489 lines)
Tests: packages/analyzers/soroban/storage/__tests__/storage-entry-classifier.spec.ts (231 lines, 10 tests)

I built a classifier that categorizes Soroban storage entries into semantic types: configuration, access_control, user_data, counter, cache, state, metadata, token, mapping, and unknown.

My approach combines two analysis strategies:

1. Name-based classification — I scan storage key names against keyword lists for each category. For example, keys containing admin, fee, threshold are classified as configuration; keys with role, paused, whitelist map to access_control. I score each category using exact matches, segment matches (split by _), and substring matches, then pick the highest-scoring category.

2. Usage-based fallback — When name-based classification yields low confidence, I analyze the access pattern. Write-once-read-many suggests configuration; write-only temporary storage suggests cache; access with extend_ttl suggests user_data.

I also detect misclassifications where the storage tier doesn't match the semantic category:

• Cache data stored in persistent storage → suggests moving to temporary
• Configuration in temporary storage → suggests moving to persistent
• User data in instance storage → suggests persistent with TTL
• Temporary entries using extend_ttl → suggests persistent

Each classification includes a confidence level (high, medium, low) and human-readable reasons explaining why the category was chosen.

---

Issue 2: Inefficient Soroban Temporary Storage Usage

File: packages/analyzers/soroban/storage/inefficient-temporary-storage-analyzer.ts (335 lines)
Tests: packages/analyzers/soroban/storage/__tests__/inefficient-temporary-storage.spec.ts (208 lines, 10 tests)

I created a detector that identifies seven inefficiency patterns in how contracts use Soroban's temporary storage tier:

Rule	Pattern	Description
SOROBAN-TMP-01	repeated_access	Same temporary key accessed 3+ times — should use a local variable
SOROBAN-TMP-02	write_only_no_read	Temporary key written but never read in the same transaction
SOROBAN-TMP-03	should_be_persistent	Temporary key uses extend_ttl, indicating it needs longer persistence
SOROBAN-TMP-04	should_be_local	Temporary storage accessed inside a loop — causes expensive per-iteration ops
SOROBAN-TMP-05	should_be_persistent	Configuration-like keys (fee, rate, admin) stored in temporary
SOROBAN-TMP-06	over_fragmented	8+ unique temporary keys — suggests over-fragmentation
SOROBAN-TMP-07	should_be_local	Ephemeral keys (nonce, session, temp) stored in persistent storage

My implementation resolves Symbol::new variable bindings to their string literals so I track keys by their logical name rather than their variable reference. I also detect ephemeral data misplaced in persistent storage by scanning for keywords like nonce, session, ephemeral, cache.

---

Issue 3: Soroban Storage Footprint Expansion

File: packages/analyzers/soroban/storage/storage-footprint-expansion-analyzer.ts (313 lines)
Tests: packages/analyzers/soroban/storage/__tests__/storage-footprint-expansion.spec.ts (202 lines, 10 tests)

I implemented a detector that identifies six patterns causing a contract's storage footprint to grow unboundedly:

Rule	Pattern	Risk Level	Description
SOROBAN-EXP-01	unbounded_collection_growth	unbounded	Collection growth (push/insert) inside a loop with storage writes
SOROBAN-EXP-02	no_cleanup_mechanism	unbounded	Growing collections without remove/pop/clear or length checks
SOROBAN-EXP-03	growing_map_in_storage	bounded_but_high	Collection types (Vec, Map, etc.) in storage without cleanup
SOROBAN-EXP-04	dynamic_key_pattern	unbounded	Dynamic key generation (format!, to_string) with storage writes
SOROBAN-EXP-05	unbounded_key_generation	unbounded	Parameterized DataKey variants combined with collection growth
SOROBAN-EXP-06	append_without_bound	bounded_but_high	Conditional append without corresponding removal condition

I restructured the detection logic so that growing_map_in_storage is checked independently of growth-method detection. This ensures contracts using Map.set() (rather than push/insert) are still flagged when they store collections in persistent storage without cleanup.

---

Issue 4: Soroban Footprint Access Analyzer

File: packages/analyzers/soroban/footprint/footprint-access-analyzer.ts (369 lines)
Tests: packages/analyzers/soroban/footprint/__tests__/footprint-access-analyzer.spec.ts (234 lines, 12 tests)

I built an analyzer that examines access patterns within Soroban transaction footprints to identify eight pattern types:

Rule	Pattern	Description
SOROBAN-FPA-01	hot_key	Key accessed 5+ times — candidate for instance storage
SOROBAN-FPA-02	cold_key	Key accessed only once — consider lazy loading
SOROBAN-FPA-03	loop_access	Key accessed inside a loop — hoist or buffer
SOROBAN-FPA-04	imbalanced_access	Read/write ratio > 3:1 — cache more aggressively
SOROBAN-FPA-05	read_heavy	Function with reads >> writes — batch reads
SOROBAN-FPA-06	write_heavy	Function with writes >> reads — consolidate writes
SOROBAN-FPA-07	high_overlap	Functions with >70% footprint overlap — consolidate

My implementation builds per-function footprint profiles that track which keys each function accesses, read/write counts, and loop access counts. I also compute a footprint overlap score between function pairs to identify candidates for consolidation.

I refactored the template literal expressions to extract Array.from(access.functions)[0] into a local fnName variable. This resolved a TypeScript parser issue where } inside template literals was being misinterpreted as the closing brace of an object literal.

---

Infrastructure Changes

I updated the module export indexes:

• packages/analyzers/soroban/storage/index.ts — Added exports for all three new storage analyzers
• packages/analyzers/soroban/footprint/index.ts — Added export for the footprint access analyzer

---

Technical Decisions

Lexical analysis approach: I followed the existing codebase pattern of using regex-based lexical analysis with the shared utilities in packages/analyzers/soroban/common/source-utils.ts. I used maskNonCode() to strip comments and string literals before scanning, extractFunctions() for brace-aware function body extraction, and blockStackAt() / isInLoop() for loop detection.

Symbol resolution: I pre-scan for Symbol::new(&env, "key") and symbol_short! definitions to build a variable-to-key map. This lets me track storage accesses by their logical key name rather than their variable reference. I run this regex on the original source (not the masked version) because string literals are blanked out in the masked copy.

Key extraction from method calls: I use the same pattern as the existing storage-in-loop-analyzer.ts — a regex that matches up to the method's opening paren, then extractArgs() with balanced-paren matching to correctly extract arguments even with nested parentheses.

---

Testing

I wrote 41 tests across 4 test suites covering:

• Empty/edge case handling
• Correct classification and pattern detection
• Multi-pattern scenarios
• Metric accuracy
• Misclassification detection
• Threshold boundaries

All 41 tests pass. The 5 pre-existing test failures in dataflow/ and resources/ledger/ directories are unrelated to my changes.

---

Files Created

File	Lines
storage-entry-classifier.ts	489
inefficient-temporary-storage-analyzer.ts	335
storage-footprint-expansion-analyzer.ts	313
footprint-access-analyzer.ts	369
storage-entry-classifier.spec.ts	231
inefficient-temporary-storage.spec.ts	208
storage-footprint-expansion.spec.ts	202
footprint-access-analyzer.spec.ts	234
Total	2,381

Files Modified

File	Change
storage/index.ts	Added 3 exports
footprint/index.ts	Added 1 export
footprint-access-analyzer.ts	Extracted local variables to fix template literal parser issue
storage-footprint-expansion-analyzer.ts	Restructured detection logic for growing_map_in_storage

RELATED ISSUES:

• CLOSES Implement Soroban Storage Entry Classifier #888
• CLOSES Detect Inefficient Soroban Temporary Storage Usage #889
• CLOSES Detect Soroban Storage Footprint Expansion #891
• CLOSES Implement Soroban Footprint Access Analyzer #892

[drips-wave]

@tzar-deek Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits