DEV Community

Ben Halpern
Ben Halpern

Posted on

How do you feel about regex?

Regex seems to have a broad array of love and hate. How do you feel about it? Do you seek to use or avoid it as a problem solver, and how much do you understand it?

Discussion (72)

Collapse
link2twenty profile image
Andrew Bone

I like it but I tend to tell people to avoid it if possible just because it makes code hard to read.

If people on the team do use regex I ask them to include a link to regexper.com/ in a comment.

It generates a flow chart like this one.

regex flow chart

// https://regexper.com/#%2F%5E%5Cw%2B%28%5B-%2B.'%5D%5Cw%2B%29*%40%5Cw%2B%28%5B-.%5D%5Cw%2B%29*%5C.%5Cw%2B%28%5B-.%5D%5Cw%2B%29*%24%2F
const emailReg = new RegExp(/^\w+([-+.']\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*$/);
console.log(emailReg.test('user@place.com'))
Enter fullscreen mode Exit fullscreen mode
Collapse
baenencalin profile image
Calin Baenen

Oooh. I like this tool you introduced upon everyone.
Personally, if a regex is confusing me, I'll just pop over to regex101 for a quick test. It's also a great tool.

Collapse
lepinekong profile image
lepinekong • Edited on

I prefer regexr.com because regex101 is not embeddable in figjam document figma.com I use to keep code notes as it supports code blocks.

Other tools I find pretty but buggy for complex regex are :

regexper.com/
jex.im/regulex
extendsclass.com/regex-tester.html

Collapse
priteshusadadiya profile image
Pritesh Usadadiya

I didn't know about one. definitely useful. Thanks for sharing.

Collapse
andypiper profile image
Andy Piper

Now, this is really cool and useful - thank you!!

Collapse
jesusantguerrero profile image
Jesus Guerrero

Wow thanks for sharing that tip.

Collapse
internetmosquito profile image
Alejandro Villamarin

didn't know that service, seems awesome, also worth a try cucumber.io/docs/cucumber/cucumber..., makes regular expressions more human friendly and readable

Collapse
lepinekong profile image
lepinekong • Edited on

Have tried them all I think, that one is nice but it doesn't work for very complex regular expressions (like with some lookaround expressions) whereas it works with regexr.com

Collapse
drewclem profile image
Drew Clements • Edited on

I don't write regex. I look up what I need to when I need it and I never think about it again. 🤣

I've found that the specificity of what I need when I do need it can vary so vastly, and that the library for it is so large, that the best use of my time is figuring out what I need at that specific moment and moving on.

It's not needed often enough to warrant full deep dive into it, for me at least.

Collapse
citizen428 profile image
Michael Kohl

Having used them since my Perl days, I love them.

I think one of the problems is that they are badly taught. Some examples:

  • Regex as available in most programming languages are actually more powerful than the formal concept of a regular language, which is why the term is often used to distinguish them from "regular expressions". PCRE engines are surprisingly powerful, as illustrated e.g. by /^1?$|^(11+?)\1+$/ which can be used to check if a number is prime (it's obviously a pretty silly way to test primality but it shows how much power they pack).
  • Most people do not understand how regex engines actually work, i.e. how they are compiled into automata, how (and when) backtracking works,, why backtracking can lead to unexpected matches, etc. When you treat things as blackboxes they often look scary. This leads to people copy/pasting things off StackOverflow and the cycle continues.
  • For the above reasons there are many bad examples of regular expressions out there which give the whole concept a bad name.
  • For many tasks a relatively simple recursive descent parser would be a better solution.
Collapse
mbarzeev profile image
Matti Bar-Zeev

You write a regex and don't write a comment next to it describing what it does, you and I gonna have a lil' chat ;)

Collapse
waylonwalker profile image
Waylon Walker

I recently had a case where the person I was reviewing added regex, and they did comment what it did well, and honestly the comment looked exactly correct. They were missing tests, and it wasn't until they ran tests that they realized that what they had done was not what they described in the comment.

Collapse
mbarzeev profile image
Matti Bar-Zeev

Hey, tests go without saying, right?
;)

Collapse
zakwillis profile image
zakwillis

As long as it can be tested proven, and injectable (i.e. not hard coded) should be fine? :)

Collapse
jeremyf profile image
Jeremy Friesen

I love using regular expressions. And try to adopt the following pattern:

  1. Write a method for evaluating the regular expression.
  2. Bombard that method with tests, both matching and non-matching cases.
  3. Profit!

Regular expressions can be quite dense to read, so I want to use the tests to highlight my expectations around what I wrote.

Collapse
citizen428 profile image
Michael Kohl

Regular expressions can be quite dense to read

For more complicated regex Ruby's x modifier comes in handy, we should probably use it more.

Collapse
jeremyf profile image
Jeremy Friesen

I haven't used the x modifier before, and just did some looking and I found it initially harder to read. But I could see it as a boon for other folks, and could acclimate to reading regex with comments.

Thread Thread
citizen428 profile image
Michael Kohl

I agree to a certain extent but I believe for sufficiently complicated regex [1] it makes sense.

I also think people underutilize the fact that Ruby allows composing regex (#{}, Regexp.union). Having properly named sub-expressions greatly reduces cognitive overhead.

[1] But not complicated enough to warrant a full parser just yet.

Collapse
billraymond profile image
Bill Raymond

I am what you might call an ocassional developer. As an author, I created an app that helps me convert my work to ePub format.

One of the requirements with ePub is that you output all your files to xHTML. However, the output from my word processing software (Microsoft Word) outputs to some very haphazardly developed HTML that is not xHTML compliant.

I found a whole bunch of libraries that allow me to manipulate the output to xHTML, but in fact they did not do many of the things required to pass basic specifications. Specifically, in xHTML, tags must be lowercase. All of the libraries I worked with at the time made broad assumptions about the HTML structure and worse, did not do the basic conversion from uppercase tags to lowercase tags.

After spending way too much time dealing with this, I paid a developer to figure out the problem. One day later, he delivered three lines of regex code that handled the uppercase to lowercase issue and two other problems I was dealing with. Nearly a month of work and me trying to understand how all these libraries work and sudenly I had working code. My books are thousands of pages long and broken up into dozens of files. The regex code worked great everytime.

Having not known about regex until that moment, I went about using it everywhere. I used it when my XML output was not correct. I used it to fix the file output where special characters should be renamed with escape codes and much more.

I am willing to bet that while my code technically works, all that regex is probably a bad idea. Also, regex is not easy to read, so you really have to document it well.

Overall, I really do not like the structure because of it being hard to read, but will say that without it, I probably would have just tossed my app into the trash bin had I spent two more weeks on something as simple as fixing uppercase and lowercase letters along with a few edge case issues.

The drive to complete my app drove me to use regex in areas I probably should not have used it. For example, I used regex to modify some XML files and since I did not really know XML that well, I simply gave up learning it. Instead, I used the XML output from a library and then modified the file with REGEX. Really, I should have sat down and learned XML a little more.

Regex feels like one of those things we all refer to as monolith applications. It seems like you can do anything and everything with it, but the complexities in how you write proper regex and then create test cases all feels very convoluted. At the same time, there is something very tantalizing about 1-3 lines of regex code that would otherwise require customizing libraries, creating a custom API, or doing something else to solve some basic problems.

I am curious, do we know if there are alternatives that are easier to understand and use?

Collapse
waylonwalker profile image
Waylon Walker

Works great in my terminal and editor. I can search for something till it looks right, then replace it with what i want.

It production code its so hard to see all the edge cases. As a Data Engineer performance is about last on my list of metrics, as most of what I do is run on a schedule, not when someone clicks it, so an extra 5s on a 10 minute run is no big deal. With that said every time I see a regex in a code review I ask can we do this without a regex, even with a huge performance hit, or write tests for every possible edge case we can brainstorm.

Collapse
phantas0s profile image
Matthieu Cneude • Edited on

If I can avoid it in my code, I will. They can grow like crazy overtime, and they become difficult to read and understand.

I love them in my editor (I use Vim btw) or to perform operations in the shell, however. It's super powerful for everything plain text: search, search and replace, repeating an action on specific lines... the list goes on.

Collapse
fjones profile image
FJones

Regex is one of our most useful tools, when used appropriately. But that requires good knowledge of regex (such that you actually understand what you're doing) and good knowledge of the limitations.

If you're touching HTML with regex, chances are you're doing it wrong.
If you need backreferences and readaheads, chances are you're doing it wrong.
If you're only using .*, chances are you're doing it wrong.

I recommend that anyone who wants to understand regex implement a small regex parser for themselves, to get a greater understanding of how it works. Alternatively: Try to rewrite your regex pattern exclusively in terms of |+.() to explain what it actually does. Everything else in the basic regex syntax builds off of that.

Collapse
lexlohr profile image
Alex Lohr

I admit, when I first became really fluent in RegExp, I tended to overuse them for some time. Even now, I have the impulse to solve string-based issues with RegExp, but I have learned to stop and think about if they really are an improvement over other solutions. So you could say I seek to use it and try to avoid it at the same time, mostly because I understand it.

Collapse
rfornal profile image
bob.ts

I love working with Regular Expressions.

I use them frequently in VS Code and have a few articles out here on implementing them for Search-and-Replace.

I had a project where I had to replace an AS400 Custom Script Search Language with a JavaScript version. I quickly learned how slow they are when running hundreds of them per line. It also prompted me to create a new tool for documenting Regular Expressions (github.com/bob-fornal/reggie-docs).

Collapse
conw_y profile image
Jonathan • Edited on

How do you feel about it?

Good. It's a useful tool.

Do you seek to use or avoid it as a problem solver

I treat it similarly to SQL or CSS – use it where appropriate, but try to keep it behind an abstraction if possible. E.g. I would wrap a phone number RegEx in a function such as isValidPhoneNumber.

If it's simple to solve the problem without a RegEx then I'll solve it without the RegEx. But sometimes a RegEx is simpler, e.g. the above example.

how much do you understand it

RegEx language generally – I know the basic concepts well enough, but I always check reference materials and/or use a RegEx tool when implementing one.

Specific RegExs – I almost never re-use a RegEx without first taking it apart and making sure I understand what's going on (same with any code snippet really).

Collapse
cess11 profile image
PNS11

For being terse it's surprisingly hard to read compared to other forms of computer programming line noise, like old Perl that runs on oak barrels and mules or everyday J.

Requiring visualisation tools to be somewhat interpretable outside the trivial case means one can't just skim it and be fairly confident about what it does, unlike the surrounding application code (hopefully).

If I can take the performance hit or development time I'll probably avoid regex if I can, usually it's possible to implement a parser that's easier to understand at a quick glance.

So I mostly use them in CLI settings, like patterns for ripgrep or sed.

Collapse
thumbone profile image
Bernd Wechner

What's the alternative? Just curious. I use REs when I need REs, and when I need one it's because the alternative seems to be to write a whole string parser or some clunky seriously code-verbose combination for finds, splits, replaces, etc that is even harder to read that an regex.

But then I guess I grew up with REs and have no real issue with them. Be nice if advanced RE syntax had evolved a little more standardly and didn't diverge into flavours, which it did but hey.

Collapse
citizen428 profile image
Michael Kohl

What's the alternative?

Grammars and parsers. Overkill for small ad-hoc parsing tasks where regex really shine but advisable for more complex tasks. This is where Raku (formerly Perl 6) really shines, having recursive-descent parsing as a first-class language construct is quite the game changer.

Collapse
thumbone profile image
Bernd Wechner

Indeed. The very point. That REs shine for small ad hoc jobs. Grammars and parsers are from a different context altogether. To epitomise the role of REs consider a job I example that simply wants detect all lines that contain one word somewhere before the other. This is trivially simple in an RE and preciously whence their definition stems (as in I can write that on the command line for a grep or find) and the niche in which their ubiquity reigns. That small need right now for a mildly complex pattern test or field extraction.

Thread Thread
citizen428 profile image
Michael Kohl

Yes, I think we fully agree here, my original response was maybe not clear enough. For the things regex were made for, there's IMO no better solution. But I've also seen too many people implement what effectively become super brittle parsers with them, which is also not the right thing to do.

The main reason I mentioned Raku is that Larry Wall deserves a lot of the credit for modern regex implementations, after all they generally try to be Perl compatible. So it's interesting to see which shortcomings he perceived (Raku regexes are quite different from PCRE) and also that he decided that there is a space for grammars on the language level.

Collapse
moopet profile image
Ben Sinclair

If there's a problem where it makes sense, I love it.
Regular expressions can be written in a clear and easy-to-read way, across multiple lines and with comments. They can condense a lot of logic into something easy to parse by humans, even though their reputation says otherwise. Trying to replicate what they do with a bunch of separate if contains(..) and startsWith(..) and not contains(..) methods is a hack, imo.

Using them for anything where a simple single, named method would suffice is a bad idea.

Collapse
mellen profile image
Matt Ellen • Edited on

A few years ago I bought a book (I can't find right now) about the theory of computation, and it started with finite deterministic automata (FDA), moving on to non-deterministic finite automata (NDA) and how FDA and NDA are mathematically equivalent, then it showed how to turn NDA into regular expressions.

The explanation opened my mind to a new way to look at regex, so I found it a lot easier to understand and write them, and also understand their limitations.

The main issue I'm left with is that regex engines these days have added look-ahead and look-behind which means they are no longer mathematically equivalent to NDAs.

Collapse
zakwillis profile image
zakwillis

I think they are great. They don't work in all situations.

ultrapico.com/expresso.htm is for .Net and a great tool for creating regex expressions, it doesn't support all elements of regex. Am not a super genius on them but they are an essential element of development for me when handling information.

Here is an example of configuration from a translation file from my application which is an awesome DevOps Deployment application am hoping to market in future.

The configuration below is to either find a value or a regex expression in an input file (typically itself an application configuration file) to clean a development configuration file. Sounds complex, but a great way of translating source configuration into something generic for different environmental deployment.

Regexes provides more control, but aren't always the best approach.

inforhino.co.uk/beta/automation-an...

                        {"IsRegex": false, "FindValue" : "C:\\\\InfoRhino\\\\cms\\\\IRWebsite\\\\IRWebsiteCMS\\\\wwwroot\\\\ClientConfiguration\\\\TConfigType.json", "ReplacementValue" : "{TargetServerRoot}\\\\ClientConfiguration\\\\TConfigType.json"}
                        ,
                        {"IsRegex": false, "FindValue" : "C:\\\\InfoRhino\\\\cms\\\\Adverts", "ReplacementValue" : "{TargetServerRoot}\\\\Store\\\\Adverts"}
                        ,
                        {"IsRegex": false, "FindValue" : "C:\\\\InfoRhino\\\\cms\\\\IRWebsite\\\\IRWebsiteCMS\\\\wwwroot", "ReplacementValue" : "{TargetServerRoot}\\\\wwwroot"}
                        ,
                        {"IsRegex": false, "FindValue" : "content\\\\Cards", "ReplacementValue" : "Cards"}
                        ,
                        {"IsRegex": true, "FindValue" : "(?is-nx:(?<=\"CardDomainHeaderCaption\"\\:(\\s{0,1})\")([a-z0-9\\s\\,\\.]+))", "ReplacementValue" : "{CardDomainHeaderCaption}"}
                        ,
                        {"IsRegex": true, "FindValue" : "(?is-nx:(?<=\"CardDescription\"\\:(\\s{0,1})\")([a-z0-9\\s\\,\\.]+))", "ReplacementValue" : "{CardDescription}"}
                        ,                       
                        {"IsRegex": true, "FindValue" : "(?is-nx:(?<=\"CompanyName\"\\:(\\s{0,1})\")([a-z0-9\\s\\,\\.]+))", "ReplacementValue" : "{CompanyName}"}

Enter fullscreen mode Exit fullscreen mode
Collapse
mondash profile image
Matthew Ondash • Edited on

If you're trying to pattern match a string, is there really a better option? If so, I don't know it, but feel free to let me know.

There's definitely a rabbit hole to fall into and a lot of complexity along the way, but I've found regex to be one of the more reusable and powerful tools in my arsenal even though most of the time they seem to be on the simple side rather than complex.

If it ever get's confusing I've always got my goto: regex101.com/

Collapse
mistval profile image
Randall

It's a great tool for solving certain types of problems that would be very hard to solve otherwise. I've seen some very arcane solutions for problems that could have been solved very simply with regex - because the author of the code didn't know how to use regex. Almost everyone learns about regex later than they should - I know I did.

Any non-trivial regex should be clearly commented, though. That's often missing.

Collapse
alohci profile image
Nicholas Stimpson

It's a tool in the toolbox. Like the other tools, my understanding is incomplete, but generally good enough to know when to use it and when to choose a different tool. Can use it sufficiently well to get done what I need to, which normally isn't particularly demanding. While it'd be nice to understand the mathematics behind it, it probably wouldn't actually change how I'd use it.

Collapse
davestewart profile image
Dave Stewart • Edited on

regexone.com/ is BY FAR the best resource to learn RegEx from scratch.

Pretty much everything you would get in this book (but also worth reading!)
amazon.co.uk/Mastering-Regular-Exp...

And actually, core RegEx is fairly straightforward:

  • learn the tokens
  • read left to right
  • say what you see
  • (learn the tokens again)

Good luck!

Collapse
dan_starner profile image
Daniel Starner

I love regex and I'm fairly confident with creating complex regex's from memory, but I understand that many people find it difficult, so I try to either use it sparingly or place a lot of comments around it to help others who may not be so comfortable.

Collapse
lewiscowles1986 profile image
Lewis Cowles

It's generated code, according to a declarative syntax, that has been tested and iterated on by millions of projects over decades... What isn't to love?

Prefer if we can use simpler things, but if it's needed. Then it is needed.

Collapse
dannyengelman profile image
Danny Engelman

youtube.com/watch?v=WDaNJW_jEBo

I've learned to love
more than hate
and
that's the overall thing
of it
all just to try to learn
to love more
because it's easy to hate
but it's hard to love
-- Snoop Dogg

Collapse
syxaxis profile image
George Johnson

Very useful BUT it's almost an entire language in it's own right. I kind of respect that and just skirt around it. I use online regex builders when I need a regex for a filter style func call and that's me and regex done with until next time!

There was an 300 page O'Reilly book sold at one point just on Regex and nothing else! That's how deep the rabbit hole goes on regex. If you can avoid regex, then do so, if you can't then make the expression simple and watertight by testing the heck out it. Regex is greedy and if you don't test it properly you will regret it later.

Collapse
tylerauerbeck profile image
Tyler Auerbeck

My go to example of a love/hate relationship. It’s a super useful tool, but used in the wrong ways can lead to a lot of pain. One thing I always like to do Is leave useful comments around a regex when I have to use it in code. This is good for future me just as much as it is good for other team members who stumble along it in the future.

Collapse
ryencode profile image
Ryan Brown

When I use RegEx's I try to follow these rules (more like a guideline 🏴‍☠️)

  1. Keep the inputs shorter (don't usually regex over whole documents)
  2. Don't do too much in each regex (follow up on matches to further refine if needed)
  3. Don't let the user put in regex (unless you're building a tool for super-advanced-users)
  4. Expect non-matches and handle those
  5. Over-Test, test, test again, automate the tests, and ensure any related issue/bug reported is added as a case to the tests.

My context is usually used in file name parsing as my users are supposed to name their documents in a meaningful, structured way for ingestion into other systems. The exception report (files not match-able) typically runs longer than the source code ;) Turns out people just can't spell common words consistently when it matters. (I'm no exception)

Collapse
qarunqb profile image
Bearded JavaScripter

I like to use Regex and I use it frequently but not for anything too complicated. I generally build all my RegEx from Regex101.

I really understood it at its core during my university Computer Science Program where we talked about DFAs -> NDFAs -> Regular Languages -> Regular Expressions

Collapse
steelwolf180 profile image
Max Ong Zong Bao

Hmmm...I would advise to use it only for search, data scrapping and data validation/cleaning purposes. Beyond that I won't advise anyone to use it unless there's a real reason to use it like to reduce the complexity of your code for the above purposes.

Plus there's tons of websites that allows you to copy and paste regex for specific uses depending on your language of choice or regex engine you are using.

To me it's sort of like Leetcode, readability is a issue unless you had learnt it but beyond that it kind of being too clever with your code.

Collapse
cubikca profile image
Brian Richardson

I started on Perl, so I'm very familiar with regular expressions. That said, they definitely are not that easy to read, so for code, keep it simple and comment what you are doing. It solves a lot of problems, but not necessarily better than more readable approaches. Still, you won't completely avoid them. Rewrite rules, for example, still require a basic understanding of regular expressions, as well as some network device configurations.

Collapse
meatboy profile image
Meat Boy

Pretty good. I remember first time when I saw guy typing long multi-group and nested regex to grep out logs from file. I was amazed. Then I learnt it on my own. Still wikis are helpful, but for most daily cases it's possible to learn.

Collapse
pinotattari profile image
Riccardo Bernardini • Edited on

I love them. I find them really powerful and flexible, especially in script-like languages like Ruby.

Yes, I know, sometimes they look like line noise... That is maybe their major drawback, but it is compensated by their power.

How much do I understand regexp? Well, it is difficult to judge yourself, but I think I know them pretty well, although there are maybe some features (e.g. greedy vs not greedy) that I do not use often and I need to check the manual.

Oh, yes, BTW, with regexp-search-and-replace in emacs you can do miracles...

Collapse
anabella profile image
anabella

I think they're fun to figure out but I only understand the very basics. My own regexps are usually way longer than the stack overflow suggestions. It seems like a fun topic to dive into, although I've never had a real need to do so. Maybe we need a game for learning regexp 🤔

Collapse
afrinc profile image
John Dears • Edited on

Never liked it, I tried to but never.
While writing or reading some, for a very short while, I get this feeling that I understood it now , but next second here and there, gets confused again.

I guess its just one of those skills, where one would *either get it entirely or nothing at all.*
Writing regex especially is tricky for matching IPV4 addresses.
I hope I am not alone :/

Thank god for these parser/convertor tools.

Collapse
mr_destructive profile image
Meet Gor

I usually take reference and modify it as per needs while working in the terminal.
It takes time for me to digest what's going on, after tinkering a litle I feel better and start to move towards the result.
Working with regex feels really messy to me sometimes, but the feeling when that works is second to none :)

Collapse
lepinekong profile image
lepinekong • Edited on

In the past I hate them and just searched copy and paste because it was pretty simple need but as I'm currently developping a Visual Meta Programming tool I'm obliged to really understand what I'm doing like Lookaround and Lookbehind so I'm now starting to like them more :)

Collapse
valeriavg profile image
Valeria

My current stance is that RegExps are hard to read, have performance issues and some dangerous pitfalls (e.g. regular expression denial of service (ReDoS)).

I find it more pleasant to write small custom parsers instead.

Collapse
mcsee profile image
Maxi Contieri
Collapse
ben profile image
Ben Halpern Author
Collapse
rgthree profile image
Regis Gaughan, III

Regex was my second programming “aha moment.” I love regex, all thanks to Apache’s mod_rewrite way back in like 2004.

Collapse
sherrydays profile image
Sherry Day

It's something I use often, but still 100% only copy from Stack Overflow.

Collapse
voyeg3r profile image
Sérgio Araújo

As a vim/nvim user I would say that regex is my bread and butter. I also love using GNU grep, "Global REgular exPression Print", sed and a bunch of other programs that make use of it.

Collapse
joaogouveia profile image
joao-gouveia

Regex is difficult to read and can have unanticipated side effects. I tend to run from it if I can.

Collapse
michaelkith profile image
MICHAELKITH (GitHub username)

Hello Developers

Collapse
scorpit01 profile image
Anish Mahapatra

I'd rather master oop then master regex

Collapse
094459 profile image
Ricardo Sueiras

One of the key building blocks of tech over the past 30 years, but one that you still have to reach for the cheat sheet. Once you have written your first mod_rewrite rules, you never look back...

Collapse
sheikh_ishaan profile image
Ishaan Sheikh

I use it for validations many times, I think it is better (sometimes) instead of using multiple if-else statements. I think the code looks more clear when using regex.

Collapse
epsi profile image
E.R. Nurwijayadi

Does sed count?

Collapse
jonrandy profile image
Jon Randy

Super useful, interesting to write

Collapse
highcenburg profile image
Vicente Reyes