-
Notifications
You must be signed in to change notification settings - Fork 53
🐛 Implement filepath filtering in gRPC providers for chained conditions #1004
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 24 commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
92d8909
:bug: Implement filepath filtering in gRPC providers for chained cond…
tsanders-rh 00aa5d6
Fix test expectations for Windows case-insensitive comparison
tsanders-rh 48bc002
Fix missing imports in filepath filtering tests
tsanders-rh 571864d
Fix nodejs filepath filtering test compilation issues
tsanders-rh 9d9e3b1
Fix incorrect path generation in benchmark tests
tsanders-rh 5bbe00a
:recycle: Refactor duplicate normalizePathForComparison to shared fun…
tsanders-rh a1b8c5a
:construction: Use FileSearcher for comprehensive filepath filtering
tsanders-rh 5fbc1c9
:recycle: Move NormalizePathForComparison test to provider/lib_test.go
tsanders-rh a1d3414
:bug: Always use FileSearcher to apply all filepath constraints
tsanders-rh 43f8ba1
:zap: Parallelize file search and symbol retrieval for better perform…
tsanders-rh 634b491
:fire: Remove accidentally committed test scripts and temporary docs
tsanders-rh 03015ec
:bug: Handle FileSearcher errors gracefully to avoid false negatives
tsanders-rh aaab5f9
:recycle: Use channel-based approach to eliminate race conditions
tsanders-rh 4e0217a
:white_check_mark: Update demo golden files with correct filtering be…
tsanders-rh ece91be
:bug: Fix java provider to allow dependency code analysis
tsanders-rh 36a81f8
:test_tube: Trigger CI to test dependency analysis fix
tsanders-rh cfa5f27
:twisted_rightwards_arrows: Merge upstream/main to resolve conflicts
tsanders-rh 4d6b6ba
:sparkles: Use provider init config includedPaths in nodejs provider
tsanders-rh f8bfe6f
:zap: Optimize nodejs provider path normalization
tsanders-rh f31d2f4
:fire: Remove local testing replace directive from java provider go.mod
tsanders-rh 76b0b0e
:bug: Fix nodejs rules being filtered out by empty dependency folder …
tsanders-rh f3982e8
:wrench: Fix nodejs provider config to use empty array instead of arr…
tsanders-rh 28ed870
:sparkles: Add filepaths support to nodejs provider condition
tsanders-rh bdcfa3c
:sparkles: Add FileSearcher integration to Java provider for comprehe…
tsanders-rh e9dab9e
Fix nodejs provider includedPaths being silently ignored
tsanders-rh 7662a26
Add missing go.sum entries for java provider
tsanders-rh ad48415
:wrench: Add replace directive to java provider go.mod to fix CI
tsanders-rh 68d24f5
:white_check_mark: Update demo golden files for nodejs provider fixes
tsanders-rh 674255c
Fix unsafe type assertion in Java provider filepath filtering
tsanders-rh c52ac90
Fix filepath filtering to apply file-level constraints for includedPaths
tsanders-rh 069703e
Parallelize file search and language server query for performance
tsanders-rh 8d5e49f
Fix log message to reference correct variable
tsanders-rh b2659c4
Revert parallelization to fix test failure
tsanders-rh 2c8875d
Re-implement parallelization with correct constraint logic
tsanders-rh d67d7f4
Removing replace from go mod and updating the test workflow to handle…
shawn-hurley File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
244 changes: 244 additions & 0 deletions
244
external-providers/dotnet-external-provider/pkg/dotnet/filepath_filtering_test.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,244 @@ | ||
| package dotnet | ||
|
|
||
| import ( | ||
| "strconv" | ||
| "testing" | ||
|
|
||
| "github.com/konveyor/analyzer-lsp/provider" | ||
| "go.lsp.dev/protocol" | ||
| "go.lsp.dev/uri" | ||
| ) | ||
|
|
||
| func TestFilepathFiltering(t *testing.T) { | ||
| // Create test locations | ||
| createLocation := func(path string) protocol.Location { | ||
| return protocol.Location{ | ||
| URI: uri.URI(path), | ||
| Range: protocol.Range{ | ||
| Start: protocol.Position{Line: 10, Character: 5}, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| tests := []struct { | ||
| name string | ||
| locations []protocol.Location | ||
| includedFilepaths []string | ||
| excludedFilepaths []string | ||
| expectedCount int | ||
| expectedPaths []string | ||
| }{ | ||
| { | ||
| name: "no filtering - all locations included", | ||
| locations: []protocol.Location{ | ||
| createLocation("file:///project/src/Program.cs"), | ||
| createLocation("file:///project/src/Utils.cs"), | ||
| }, | ||
| includedFilepaths: []string{}, | ||
| excludedFilepaths: []string{}, | ||
| expectedCount: 2, | ||
| expectedPaths: []string{"/project/src/Program.cs", "/project/src/Utils.cs"}, | ||
| }, | ||
| { | ||
| name: "included paths filtering", | ||
| locations: []protocol.Location{ | ||
| createLocation("file:///project/src/Program.cs"), | ||
| createLocation("file:///project/src/Utils.cs"), | ||
| createLocation("file:///project/tests/ProgramTests.cs"), | ||
| }, | ||
| includedFilepaths: []string{"/project/src/Program.cs"}, | ||
| excludedFilepaths: []string{}, | ||
| expectedCount: 1, | ||
| expectedPaths: []string{"/project/src/Program.cs"}, | ||
| }, | ||
| { | ||
| name: "excluded paths filtering", | ||
| locations: []protocol.Location{ | ||
| createLocation("file:///project/src/Program.cs"), | ||
| createLocation("file:///project/src/Utils.cs"), | ||
| createLocation("file:///project/tests/ProgramTests.cs"), | ||
| }, | ||
| includedFilepaths: []string{}, | ||
| excludedFilepaths: []string{"/project/tests/ProgramTests.cs"}, | ||
| expectedCount: 2, | ||
| expectedPaths: []string{"/project/src/Program.cs", "/project/src/Utils.cs"}, | ||
| }, | ||
| { | ||
| name: "both included and excluded paths", | ||
| locations: []protocol.Location{ | ||
| createLocation("file:///project/src/Program.cs"), | ||
| createLocation("file:///project/src/Utils.cs"), | ||
| createLocation("file:///project/src/Config.cs"), | ||
| createLocation("file:///project/tests/ProgramTests.cs"), | ||
| }, | ||
| includedFilepaths: []string{ | ||
| "/project/src/Program.cs", | ||
| "/project/src/Utils.cs", | ||
| "/project/src/Config.cs", | ||
| }, | ||
| excludedFilepaths: []string{"/project/src/Config.cs"}, | ||
| expectedCount: 2, | ||
| expectedPaths: []string{"/project/src/Program.cs", "/project/src/Utils.cs"}, | ||
| }, | ||
| { | ||
| name: "no false positives - similar filenames", | ||
| locations: []protocol.Location{ | ||
| createLocation("file:///project/src/Service.cs"), | ||
| createLocation("file:///project/src/ServiceImpl.cs"), | ||
| createLocation("file:///project/src/ServiceFactory.cs"), | ||
| }, | ||
| includedFilepaths: []string{"/project/src/Service.cs"}, | ||
| excludedFilepaths: []string{}, | ||
| expectedCount: 1, | ||
| expectedPaths: []string{"/project/src/Service.cs"}, | ||
| }, | ||
| { | ||
| name: "URI normalization - different schemes match", | ||
| locations: []protocol.Location{ | ||
| createLocation("file:///project/src/Program.cs"), | ||
| createLocation("file:/project/src/Utils.cs"), | ||
| }, | ||
| includedFilepaths: []string{ | ||
| "file:///project/src/Program.cs", // Triple slash | ||
| "/project/src/Utils.cs", // Plain path | ||
| }, | ||
| excludedFilepaths: []string{}, | ||
| expectedCount: 2, | ||
| expectedPaths: []string{"/project/src/Program.cs", "/project/src/Utils.cs"}, | ||
| }, | ||
| { | ||
| name: "path cleaning - .. and . resolved", | ||
| locations: []protocol.Location{ | ||
| createLocation("file:///project/src/../src/Program.cs"), | ||
| createLocation("file:///project/./src/Utils.cs"), | ||
| }, | ||
| includedFilepaths: []string{ | ||
| "/project/src/Program.cs", | ||
| "/project/src/Utils.cs", | ||
| }, | ||
| excludedFilepaths: []string{}, | ||
| expectedCount: 2, | ||
| expectedPaths: []string{"/project/src/Program.cs", "/project/src/Utils.cs"}, | ||
| }, | ||
| } | ||
|
|
||
| for _, tt := range tests { | ||
| t.Run(tt.name, func(t *testing.T) { | ||
| // Build maps for O(1) lookups | ||
| excludedPathsMap := make(map[string]bool, len(tt.excludedFilepaths)) | ||
| for _, excludedPath := range tt.excludedFilepaths { | ||
| normalizedPath := provider.NormalizePathForComparison(excludedPath) | ||
| excludedPathsMap[normalizedPath] = true | ||
| } | ||
|
|
||
| includedPathsMap := make(map[string]bool, len(tt.includedFilepaths)) | ||
| for _, includedPath := range tt.includedFilepaths { | ||
| normalizedPath := provider.NormalizePathForComparison(includedPath) | ||
| includedPathsMap[normalizedPath] = true | ||
| } | ||
|
|
||
| // Filter locations | ||
| filteredLocations := []protocol.Location{} | ||
| for _, loc := range tt.locations { | ||
| normalizedPath := provider.NormalizePathForComparison(string(loc.URI)) | ||
|
|
||
| // Check if excluded | ||
| if excludedPathsMap[normalizedPath] { | ||
| continue | ||
| } | ||
|
|
||
| // Check if included | ||
| if len(includedPathsMap) > 0 && !includedPathsMap[normalizedPath] { | ||
| continue | ||
| } | ||
|
|
||
| filteredLocations = append(filteredLocations, loc) | ||
| } | ||
|
|
||
| // Verify count | ||
| if len(filteredLocations) != tt.expectedCount { | ||
| t.Errorf("Expected %d locations, got %d", tt.expectedCount, len(filteredLocations)) | ||
| } | ||
|
|
||
| // Verify expected paths are present | ||
| foundPaths := make(map[string]bool) | ||
| for _, loc := range filteredLocations { | ||
| normalizedPath := provider.NormalizePathForComparison(string(loc.URI)) | ||
| foundPaths[normalizedPath] = true | ||
| } | ||
|
|
||
| for _, expectedPath := range tt.expectedPaths { | ||
| normalizedExpectedPath := provider.NormalizePathForComparison(expectedPath) | ||
| if !foundPaths[normalizedExpectedPath] { | ||
| t.Errorf("Expected path %q not found in filtered locations", expectedPath) | ||
| } | ||
| } | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| func BenchmarkFilepathFiltering(b *testing.B) { | ||
| // Create test data with many locations and scoped paths | ||
| locations := make([]protocol.Location, 10000) | ||
| for i := 0; i < 10000; i++ { | ||
| path := uri.URI("file:///project/src/File" + strconv.Itoa(i) + ".cs") | ||
| locations[i] = protocol.Location{ | ||
| URI: path, | ||
| Range: protocol.Range{ | ||
| Start: protocol.Position{Line: 10, Character: 5}, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| includedPaths := make([]string, 100) | ||
| for i := 0; i < 100; i++ { | ||
| includedPaths[i] = "/project/src/File" + strconv.Itoa(i) + ".cs" | ||
| } | ||
|
|
||
| b.Run("with map optimization", func(b *testing.B) { | ||
| for n := 0; n < b.N; n++ { | ||
| // Build maps | ||
| includedPathsMap := make(map[string]bool, len(includedPaths)) | ||
| for _, includedPath := range includedPaths { | ||
| normalizedPath := provider.NormalizePathForComparison(includedPath) | ||
| includedPathsMap[normalizedPath] = true | ||
| } | ||
|
|
||
| // Filter | ||
| filtered := []protocol.Location{} | ||
| for _, loc := range locations { | ||
| normalizedPath := provider.NormalizePathForComparison(string(loc.URI)) | ||
| if len(includedPathsMap) > 0 && !includedPathsMap[normalizedPath] { | ||
| continue | ||
| } | ||
| filtered = append(filtered, loc) | ||
| } | ||
| } | ||
| }) | ||
|
|
||
| b.Run("without map optimization (nested loops)", func(b *testing.B) { | ||
| for n := 0; n < b.N; n++ { | ||
| // Filter with nested loops | ||
| filtered := []protocol.Location{} | ||
| for _, loc := range locations { | ||
| normalizedLocPath := provider.NormalizePathForComparison(string(loc.URI)) | ||
|
|
||
| if len(includedPaths) > 0 { | ||
| found := false | ||
| for _, includedPath := range includedPaths { | ||
| normalizedIncludedPath := provider.NormalizePathForComparison(includedPath) | ||
| if normalizedLocPath == normalizedIncludedPath { | ||
| found = true | ||
| break | ||
| } | ||
| } | ||
| if !found { | ||
| continue | ||
| } | ||
| } | ||
|
|
||
| filtered = append(filtered, loc) | ||
| } | ||
| } | ||
| }) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.