Skip to content

perf(py): batch-path allocation and syscall nits #1349

Description

@dekobon

Summary

Four allocation/syscall inefficiencies on the Python bindings' batch
path, found during the #1238 review. None is a correctness bug; all are
per-file costs on entry points now documented for large-tree use
(analyze_paths(root, skip_generated=False)).

Findings

  1. Double stat per fileanalyze_path
    (big-code-analysis-py/src/analysis.rs) runs path.is_dir()
    immediately before read_file_with_eol, whose first line is
    fs::metadata(path). Two stat(2) per file, every file, both entry
    points. The directory probe only matters when the read cannot
    produce Ok(Some), so it can move off the hot path — but the
    EISDIR-masking comment in analyze_path documents subtle platform
    behaviour, so the restructure needs care rather than a drive-by.
  2. FuncSpace.name allocated before the gates — the owned path
    String is built ahead of both the read gate and the generated
    filter and dropped unused for every skipped file. The non-UTF-8
    check must stay ahead of the read (error-ordering contract); the
    allocation need not.
  3. attach_or_keep clones the whole metrics JSON per file per
    injector
    (big-code-analysis-py/src/batch.rs) to have a fallback,
    even when the injector is a no-op (untracked file). With
    vcs=True, vcs_per_function=True that is two full-JSON clones per
    file. An injector signature of PyResult<Option<String>>
    (None = unchanged) removes the clone on both the no-op and success
    paths.
  4. VcsRepoCache::resolve_root allocates two PathBufs per file on
    a cache hit
    — one to build the lookup key, one to clone the value.
    Arc<Path> values (refcount bump on hit) or a borrowed-key lookup
    removes both.

Non-findings (checked during the same review)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpythonPython language support / metrics

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions