Make trivia first class member of token - #219
Conversation
|
|
||
| #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] | ||
| pub enum TriviaPieceKind { | ||
| Newline, |
There was a problem hiding this comment.
It is not the job of rowan to define this. Trivias should just have a SyntaxKind. From rowan's point of view, they're tokens attached to tokens. The rest will be handled by rust-analyzer. This will also require fewer changes.
| self.raw.text() | ||
| } | ||
|
|
||
| pub fn text_trimmed(&self) -> &str { |
There was a problem hiding this comment.
I believe text() and text_range() should exclude trivia, and we should have a text_including_trivia() method (and ditto for text_range()). While I don't know for sure, code in r-a today does not handle trivia so I expect this to be easier.
There was a problem hiding this comment.
I also think it'll be easier if trivias will store their text (like regular tokens).
|
|
||
| let start: usize = leading_len.into(); | ||
| let end: usize = (total_len - trailing_len).into(); | ||
| let text = unsafe { std::str::from_utf8_unchecked(self.data.slice()) }; |
There was a problem hiding this comment.
| let text = unsafe { std::str::from_utf8_unchecked(self.data.slice()) }; | |
| let text = self.text(); |
Save the unsafe.
| @@ -0,0 +1,134 @@ | |||
| use std::{ | |||
There was a problem hiding this comment.
I'm not pleased with the amount of public functions here. We're going to remove green trees so we need to minimize their usage. Things that do not need to be public should remain private.
| } | ||
|
|
||
| impl<L: Language> SyntaxToken<L> { | ||
| pub fn new_detached<Leading, Trailing>( |
There was a problem hiding this comment.
Why do we need this? A token cannot be a root, this feels wrong.
| // endregion | ||
|
|
||
| #[derive(PartialEq, Eq, Clone, Hash)] | ||
| pub struct SyntaxTrivia { |
There was a problem hiding this comment.
I don't think we need this type. We can just have fn leading_trivia(&self) -> impl Iterator<Item = SyntaxToken> om SyntaxToken, and ditto for trailing_trivia(). Just store a list (even Box<[GreenToken]>) of leading trivia in a token, and ditto for trailing trivia.
There was a problem hiding this comment.
In particular, using regular tokens for trivias simplifies both the implementation and the changes in r-a.
2 participants
Still making changes to RA