DEV Community

Discussion on: What is a Lexer, Anyway?

Collapse
 
adam_cyclones profile image
Adam Crockett 🌀

🦄 for me this post is perfectly timed. My first project in rust is alot like this but I can see some potential improvements I can make based on your code.

Collapse
 
cad97 profile image
Christopher Durham

I'm glad it could help! Out of curiosity, what kind of improvements have you spotted?

Collapse
 
adam_cyclones profile image
Adam Crockett 🌀 • Edited

Specifically my tokens are defined in structs Token and stored in a Btree Map where the key is the token and the value is Token struct which I had planned to provide callbacks to handle what is expected after the next token. But I'm struggling to implement (new to rust) I wanted to do something like Marpa parser.

I like how your tokens are just a match statement.