Skip to content
This repository was archived by the owner on Jun 26, 2026. It is now read-only.

Add semantic search and new fields mapping#24

Open
pteich wants to merge 9 commits into
mainfrom
feat/semantic-search
Open

Add semantic search and new fields mapping#24
pteich wants to merge 9 commits into
mainfrom
feat/semantic-search

Conversation

@pteich

@pteich pteich commented May 4, 2026

Copy link
Copy Markdown
Contributor

This PR changes the following:

  • Adds lexical vs semantic search modes via mode=<lexical|semantic>, with semantic mode requiring a target resource.
  • Builds OpenSearch neural queries from SearchIndex.spec.semanticFields, targeting semantic_fields..
  • Moves SearchIndex resolution to the provider workspace default root:providers:search, with root:orgs still used for org lookup and as a legacy fallback.
  • Switches filter queries, aggregations, and filter-value extraction to filterable_fields..
  • Exposes configured field metadata from SearchIndex through resource metadata responses.

Refers to https://github.com/platform-mesh/search-operator/issues/17

@pteich pteich requested review from a team as code owners May 4, 2026 07:16
if end > total {
end = total
}
end := min(start+chunkSize, total)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

return nil, fmt.Errorf("no active SearchIndex resources found for org %q", org)
}

sort.Slice(refs, func(i, j int) bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to sort here? Doesn't OpenSearch sort it for us? Even if we take out items the order should be preserved.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sort is only for deterministic ordering after listing SearchIndex CRs from KCP, not for OpenSearch result ordering.

return out
}

func prefixedFields(prefix string, fields []string) []string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those prefixes related to semantic and exact fields or something else?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both as all (searchable) fields are prefixed now.

return SearchResponse{}, fmt.Errorf("%w: filters require a resource", ErrInvalidRequest)
}

if mode == SearchModeSemantic && resource == "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the mode for an entire query semantic as soon as one semantic fields is included in the query? I'm thinking of a query within two fields where one is semantic and the other one isn't.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now it's a dedicated mode that a user has to activate. But you are right we should maybe change this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants