chore: Refactor `CometScanRule` to improve scan selection and fallback logic#2978

andygrove wants to merge 4 commits intoapache:mainapache/datafusion-comet:mainfrom

andygrove:scan-rule-refineandygrove/datafusion-comet:scan-rule-refineCopy head branch name to clipboard