-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Co-locate function parameter names and types with Signature::with_parameters
#19524
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Add ParameterKind plus with_parameter/with_parameters builders on Signature to pair parameter names with types or coercions, reusing arity validation. Document the migration path for ergonomic signature construction. Expand signature tests to include new builder success cases and address mismatched counts, duplicate names, and variadic failures. Refactor substr function signature setup to utilize the new parameter builder, reducing duplication in specifying coercions and names.
Eliminate duplicate iterations over the parameters array by consolidating the extraction logic in substr.rs.
Jefffrey
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we can take this approach; it doesn't seem very ergonomic to essentially require the signature to be specified twice, once via the original way and then again via the parameters 🤔
…parameter handling and type signature building
…ameter variants - Implemented `from_parameter_variants` method for the `Signature` struct to allow the creation of function signatures that accept multiple parameter configurations. - Added internal methods for extracting parameter names, building type signatures for variants, and consolidating multiple type signatures. - Enhanced documentation with examples on how to use the new method effectively and constraints on parameter name inference and variant requirements.
…d add tests for constructor requirements
…improved performance
|
hi @Jefffrey
How about: // signature for
// substr(str, start_pos) OR substr(str, start_pos, length)
Self {
signature: Signature::from_parameter_variants(
&[
vec![
("str", string.clone()),
("start_pos", int64.clone()),
],
vec![
("str", string.clone()),
("start_pos", int64.clone()),
("length", int64.clone()),
],
],
Volatility::Immutable,
)
|
Which issue does this PR close?
Rationale for this change
When defining function signatures in DataFusion, parameter types (via
TypeSignature) and parameter names (viaSignature::with_parameter_names) are currently specified in separate places. This split makes signatures harder to read, increases boilerplate, and risks names/types drifting out of sync when signatures evolve (for example, when adding optional parameters).This PR introduces an ergonomic constructor that lets call sites define each variant’s parameter names and types together, producing the appropriate
TypeSignatureand inferredparameter_namesin one place.What changes are included in this PR?
Added a small helper enum
ParameterKindto allow the new builder API to accept either:DataTypevalues (forTypeSignature::Exact), orCoercionrules (forTypeSignature::Coercible).Added
Signature::from_parameter_variants:accepts multiple parameter variants (supporting optional/trailing parameters naturally),
infers
parameter_namesfrom the longest variant,builds the appropriate
TypeSignatureautomatically:TypeSignature::Nullaryvia an empty parameter list (vec![]),TypeSignature::ExactviaDataTypeparameters,TypeSignature::CoercibleviaCoercionparameters,TypeSignature::OneOfwhen multiple variants are provided,validates that
DataTypeandCoercionare not mixed within a single variant.For other signature kinds (e.g.
TypeSignature::Variadic,Uniform,Numeric,String,Comparable,Any,ArraySignature,UserDefined), existing dedicated constructors such asSignature::variadic,Signature::uniform,Signature::numeric, etc. should continue to be used.Refactored the Unicode
substrscalar UDF to useSignature::from_parameter_variantsinstead of manually constructingTypeSignature::OneOf(...)plus.with_parameter_names(...).Here are other examples of possible refactoring:
Example 1: Pad Functions with Optional Fill Character
Example 2: Round with Optional Precision
Example 3: Date/Time Extraction with Different Precisions
Example 4: Replace with Optional Occurrence Count
Example 5: Array Functions with Optional Parameters
Example 6: Split with Optional Delimiter and Limit
Example 7: To Timestamp with Optional Format
Example 8: Aggregate Functions with Filtering
Example 9: Regex Functions with Optional Flags
Example 10: Window Functions with Optional Frame
Added unit tests covering:
OneOf,DataTypeandCoercionwithin a variant,Are these changes tested?
Yes.
datafusion/expr-common/src/signature.rsforSignature::from_parameter_variants.substrto use the new API and covered behavior via the signature-building tests.Are there any user-facing changes?
No end-user behavior changes are intended.
If there are any breaking changes to public APIs, please add the
api changelabel.LLM-generated code disclosure
This PR includes LLM-generated code and comments. All LLM-generated content has been manually reviewed and tested.