[fea-rs] Parse Glyphs.app glyph predicates (phase 1) - #2082
Conversation
First half of the fea-rs parsing support for Glyphs.app $[...] predicates (#92): lex `!` as its own Bang token and ident delimiter -- `!` is an NSPredicate operator character (`!=` now, negation once #2052 adds it) -- so name!="x" splits cleanly, and no legal FEA contains `!`. Also add the predicate AST kinds and typed nodes -- predicate, clause, attribute, operator, value, and distinct `and` and `or` connectives -- with syntax-level operator, value and connective enums in the typed layer. Accessors preserve each child token and its range so validation can attach errors to the offending child rather than the whole predicate.
Parse the $[...] predicate surface inside glyph class literals, following the grammar sketched in #2057: clause chains of attribute, operator (keyword or symbolic) and value (quoted or bare), joined by a flat chain of and/or connectives. The grammar accepts the structural surface and leaves the supported subset to validation, in the spirit of the Glyphs number-value grammar; structurally foreign constructs (not, parentheses) are parse errors pointing at #2052. Known divergences from glyphsLib's opaque regex capture are documented at the top of the grammar. Until compile support lands in the follow-up PR, validation rejects every predicate with a pointer to #92, so a predicate-bearing source errors cleanly instead of panicking during class-literal resolution.
cmyr
left a comment
There was a problem hiding this comment.
this looks good! There are a few details around the parsing logic that I have some thoughts on (and have left detailed comments about inline) but big picture this makes sense to me :)
| // The caller only enters on an adjacent `$[`; pin that invariant in | ||
| // debug builds but eat gracefully so a drifted guard cannot panic. | ||
| debug_assert!(parser.matches(0, Kind::Dollar) && parser.matches(1, Kind::LSquare)); | ||
| parser.eat(Kind::Dollar); | ||
| parser.eat(Kind::LSquare); |
There was a problem hiding this comment.
if this is ever not true something is majorly wrong. I'd just do,
assert!(parser.eat(Kind::Dollar));
assert!(parser.eat(Kind::LSquare));| // idents), so the single Bang check covers both spellings. | ||
| if parser.current_token_text().eq_ignore_ascii_case("not") || parser.matches(0, Kind::Bang) { | ||
| parser.err_recover( | ||
| "not predicates are not yet supported (see fontc#2052)", |
There was a problem hiding this comment.
| "not predicates are not yet supported (see fontc#2052)", | |
| "negation (not/!) are not yet supported in predicates (see fontc#2052)", |
| } | ||
|
|
||
| parser.in_node(AstKind::GlyphsPredicateClauseNode, |parser| { | ||
| if !eat_glyphs_predicate_attr(parser) { |
There was a problem hiding this comment.
I would rename to expect_glyphs_predicate_attr (and ditto for the other three fns here) and then move the error reporting into those fns; then this just becomes,
expect_glyphs_predicate_attr(parser)
&& expect_glyphs_predicate_op(parser, recovery)
&& expect_glyphs_predicate_value(parser, recovery)| parser.eat_raw(); | ||
| true |
There was a problem hiding this comment.
I would eat_remap here, to something like GlyphsPredicateOpName? It's very defensive, but it guards against a situation where there is an op name that overlaps with a keyword or something else that doesn't just parse as an ident, which could fall through the cracks later?
| parser.eat_remap(parser.nth(0).kind, AstKind::GlyphsPredicateAttr) | ||
| } | ||
|
|
||
| fn eat_glyphs_predicate_op(parser: &mut Parser, _recovery: TokenSet) -> bool { |
There was a problem hiding this comment.
it would be nice to document the actual list of ops? According to the apple docs, the supported (non-string) operators are just, "=" | "!=" | "<" | ">" | "<=" | ">="; and == isn't supported at all, although it's used heavily in the glyphs.app documentation. I must be missing something...
| } | ||
|
|
||
| #[test] | ||
| fn glyphs_predicate_types_keyword_operators() { |
There was a problem hiding this comment.
if we go ahead and parse all the named operators we can make this a parse test too.
| // `!` is `Bang` (and an ident delimiter, see `is_special`) so that | ||
| // `name!="x"` lexes as `name` `!` `=` `"x"`. This changes the error | ||
| // shape of already-invalid `!`-adjacent input: `hi!` used to lex as | ||
| // one bad Ident (rejected as an invalid glyph name), and now lexes | ||
| // as `hi` (a valid name) plus a stray `!` that a caller rejects | ||
| // downstream. No legal FEA contains `!`, so nothing valid is affected. |
There was a problem hiding this comment.
I don't think this comment is really adding much?
| } | ||
|
|
||
| #[test] | ||
| fn glyphs_predicate_fails_validation_until_compile_support_is_added() { |
There was a problem hiding this comment.
if this is still relevant is it well expressed as a normal compile test (text files).
|
|
||
| /// An operator written in a Glyphs.app glyph predicate. | ||
| #[derive(Clone, Debug)] | ||
| pub enum GlyphsAppPredicateOp { |
There was a problem hiding this comment.
yea it's annoying that our deferred parsing pushes us in this direction, and it's an argument for just doing the parsing ahead of time.
| @@ -0,0 +1,24 @@ | |||
| @missing_value = [$[name == ]]; | |||
There was a problem hiding this comment.
i'd generally prefer to split these different failures into more cases, if there's a reasonable way to group them?
Operators are now parsed eagerly into per-operator token kinds via the suggested eat_adjacent_remap, unknown operator words are parse errors, and values are single tokens except the single-quoted form; the hand-written enums and classification logic in typed.rs become generated token/enum types. The accepted operator set is documented against glyphsLib's. Also: expect_* renames with error reporting moved inside, asserted $[ prefix, the negation error reworded, the validation-stub test expressed as a compile test, the bad fixture split by theme, and unit tests whose inputs the parse-test fixtures already pin dropped.
First of two PRs for #92 (supersedes the #2057 draft): this one teaches the fea-rs lexer and parser the Glyphs.app
$[...]predicate surface; a stacked follow-up PR will expand name-only predicates at compile time.The grammar follows the BNF sketched in #2057 (comment), one node per production:
GlyphsCompoundPredicate = GlyphsPredicate [AND/OR GlyphsCompoundPredicate]->GlyphsPredicateNode, with the and/or connectives as distinctGlyphsPredicateAnd/GlyphsPredicateOrtokensGlyphsPredicate = GlyphsPredicateObject GlyphsPredicateOperation GlyphsPredicateValue->GlyphsPredicateClauseNodeGlyphsPredicateObject = Ident->GlyphsPredicateAttrtokenGlyphsPredicateOperation = Ident | == | > | < | ...->GlyphsPredicateOpNodeGlyphsPredicateValue = String | Number | Ident->GlyphsPredicateValueNodeThe grammar deliberately accepts more than the compiler will initially support: anything predicate-shaped parses (chains of attribute/operator/value clauses joined by and/or connectives, keyword and symbolic operators, quoted and bare values), one whole token at a time, in the spirit of the number-values grammar's "just eat values and operators; we will validate later". Only constructs that don't fit that shape at all (NOT, parentheses, nesting) are parse errors; they point at #2052 (our phase 2 which tracks the parts of the predicate language we don't support yet). Narrowing the well-formed predicates down to the supported subset is validation's job, and in this PR validation is a stub rejecting every predicate ("not yet supported at compile time"), so compiled output is unchanged until the follow-up lands.
There are some minor known divergences from glyphsLib's opaque regex capture (
\$\[([^\]]+)\]), summarized below, but none of these lets a predicate that both toolchains accept select different glyphs; where glyphsLib silently evaluates something other than what was written, fontc errors instead:"or#inside a single-quoted value trips FEA's own string/comment lexing and breaks the parse (impossible in a real glyph-name value);name contains"x",x&&name) are parse errors, though glyphsLib's boundary-free regexes accept some of them (every operator example in the Glyphs docs is spaced);name != 123abcbecomesname != 123, which selects every glyph).NOTE: Most of the diff (especially the second commit) is test fixtures: a parse-tree fixture pinning the accepted surface, and error fixtures for each rejected family.