DEV Community

Calin Baenen
Calin Baenen

Posted on

ParseJS. An easy function I made to tokenize things.

ParseJS on GitHub

Want to make a math evaluater? A programming language mockup? A document format? A file reader?
If you want to make any of those, my small one-function library is for you! :D

This is the function signature:

function parse_string(
    str:string,
    toks:Array<string>
):Array<string|symbol>;
Enter fullscreen mode Exit fullscreen mode

How does it work?
It takes a string to scan and a list of tokens. If a token is found; a symbol (Symbol.for( tok )) is inserted, otherwise the character in the string being scanned is inserted -- and the final result is returned.

Here is an example of the function in action!
"Sonic", "Tails", and "Knuckles" are all recognized as tokens, but "Amy" isn't.

The documentation looks like this:

Parses a string and returns a list consisting of one of
length-1-`string` (`char`) or `symbol`, representing the
tokens the scanner finds.
Every time a token is successfully found, a symbol
representing the token is inserted, otherwise
the current character is used instead.
Enter fullscreen mode Exit fullscreen mode

Top comments (0)