Partial parse/tokenisation?

[hadley]

Not sure if you have this already, but if you do, it's not documented.

Could be own function, or partial argument to existing functions.

[kevinushey]

What would the partial argument do in this case? For tokenization it's at least it's (mostly) stateless so this should be easy. The exception being [ and [[, e.g. in tokenizing

foo[bar[[1]]]
foo[[bar[1]]]

We need to maintain a token stack to disambiguate when to tokenize ] vs ]].

[hadley]

partial would change it from returning an error to returning something with a partial flag.