Introducing ast-grep: A tool for structural searching and transforming code

Are you tired of manually searching and modifying code? Do regular expressions and text-based tools fall short when it comes to complex syntax and nested structures? Look no further! In this blog post, I'm excited to introduce you to ast-grep, a powerful command-line tool designed to make structural searching and transforming of code a breeze using abstract syntax trees (ASTs).

What is ast-grep?

ast-grep is a command-line tool that leverages the power of abstract syntax trees (ASTs) to search and transform code. ASTs are data structures that represent the syntactic and semantic structure of source code. By using ASTs, ast-grep provides a more precise and reliable way to analyze and manipulate code compared to regular expressions or text-based tools.

With ast-grep, you can define search patterns and rewriting using a source-code-like template, enabling you to perform tasks such as finding code smells, refactoring code, or enforcing coding standards with ease.

Why use ast-grep?

1. Power and Precision: Regular expressions and text-based tools have their limitations when it comes to complex syntax, multiple languages, and nested structures. ast-grep overcomes these limitations by operating on the structured AST representation of the code, enabling more powerful and precise searching and transforming capabilities.

2. Ease of Use: ast-grep is designed to be user-friendly and easy to use. It offers a beautiful command-line interface (CLI) that provides a seamless experience. Additionally, ast-grep comes with rich documentation and examples to help you quickly get up to speed and make the most out of the tool.

3. Installation Options: ast-grep can be installed using popular package managers such as npm, pip, brew, or cargo, depending on your preferred programming language. This flexibility ensures that you can easily integrate ast-grep into your existing development environment.

4. Interactive Editing: ast-grep provides an interactive editing mode that allows you to experiment and fine-tune your search patterns and actions. This feature saves you time and effort by providing instant feedback as you refine your queries.

How to use ast-grep?

Getting started with ast-grep is a breeze. Here's a step-by-step guide to help you get up and running:

1. Installation: Begin by installing ast-grep using your preferred package manager.

# install via npm
npm i @ast-grep/cli -g
# install via cargo
cargo install ast-grep
# install via homebrew
brew install ast-grep

1. Specify the Language: Once ast-grep is installed, you need to specify the language of the code you want to search and transform. You can do this by using the -l option followed by the language identifier or by specifying the file extension of your code.

2. Define Search Patterns: With the language specified, you can now define your search patterns using the --pattern or -p option. The search pattern is written in a source-code-like template format that represents the structure you want to match in the AST.

3. Perform Transformations: If you want to perform transformations on the matched code, you can use the --rewrite or -r option. This allows you to replace the matched code with the desired code.

ast-grep also supports more advanced usage through rule files written in YAML. Rule files enable you to perform even more complicated searches and transformations by defining multiple patterns and actions. You can even use it as a linter to enforce coding standards. By defining search patterns that represent code smells or violations, you can identify and flag problematic code automatically.

Examples of ast-grep

To give you a taste of what ast-grep can do, here are a few examples:

1. Replacing Print Statements in Python: Suppose you want to replace all print statements with logging calls in a Python codebase. You can achieve this with ast-grep using the following command:

sp -l py -p 'print($A)' -r 'logging.info($A)'

1. Removing Unnecessary useState Type in React Codebase: In a React codebase, you might come across unnecessary type annotations in useState calls. ast-grep can help you remove them with the following commands:

sg -p 'useState<number>($A)' -r 'useState($A)'
sg -p 'useState<string>($A)' -r 'useState($A)'
sg -p 'useState<boolean>($A)' -r 'useState($A)'

1. Swapping Arguments in Yew's use_memo Function: To automate the migration of code using the Yew framework, you can swap arguments of the use_memo function using ast-grep with the following commands:

sg -p 'use_memo($CALLBACK,$$$DEPENDENCIES)' --r 'use_memo($$$DEPENDENCIES, $CALLBACK)' -l rs
sg -p 'use_memo($DEPENDENCIES,,$$$CALLBACK)' -r 'use_memo($DEPENDENCIES,$$$CALLBACK)' -l rs

1. Web Crawling with ast-grep: ast-grep is not limited to searching and transforming code. You can also parse code from standard input with ast-grep to perform some scripting jobs. For example, the following command extracts a list of authors from the SciPy 2022 conference website:

curl -s https://schedule2021.scipy.org/2022/conference/ |
  sg --pattern '<div $$$> $$$ <i>$AUTHORS</i> </div>' --lang html --json |
  jq '.[] | .metaVariables | .single.AUTHORS.text'

These examples demonstrate the flexibility and power of ast-grep in different scenarios, from refactoring code to extracting information from web pages.

Where to learn more about ast-grep?

If you're interested in exploring ast-grep further, here are some resources to check out:

• Official Website: Visit the official ast-grep website for comprehensive documentation, tutorials, examples, and deep-dive guides. It serves as a valuable hub of information to help you understand and utilize ast-grep effectively.

• GitHub Repository: The ast-grep GitHub repository hosts the source code, issues, pull requests, and releases of ast-grep. You can explore the repository, star it, fork it, or even contribute to the project if you're interested in getting involved.

Conclusion

In this blog post, we introduced ast-grep, a powerful tool for searching and transforming code using abstract syntax trees (ASTs). We discussed what ast-grep is, why you should consider using it, how to use it effectively, and provided examples of its usage in various scenarios.

ast-grep's ability to handle complex syntax, multiple languages, and nested structures sets it apart from regular expressions and text-based tools. Its ease of installation, user-friendly CLI, and interactive editing mode make it accessible to developers of all levels.

To dive deeper into ast-grep, visit the official website for detailed documentation, tutorials, and examples. If you're interested in contributing to the project or exploring its source code, head over to the GitHub repository.

Give ast-grep a try and empower yourself with a versatile and precise tool for structural searching and transforming of code. Happy coding!