Why your folder structure sucks

YouTube

Inexperienced developers organize their code by technology rather than by feature. This is a mistake. Separating by technology leads to

1. folder straddling
2. hard-to-find files
3. reusability confusion
4. slow load times
5. slow build times
6. huge folders

1. Folder straddling

When do you tend to edit or view all files that use the same technology? It’s usually when you are

• learning the technology
• teaching the technology
• adding/upgrading the technology

This is why every time a new technology is invented, everyone seems to agree that all the files that use it should have a dedicated folder. Learners and teachers both want to quickly become oriented with the new technology and switch easily between examples that use it.

But they eventually grow out of this pattern. Let's look at some examples.

HTML + JavaScript + CSS

In the early days of AngularJS, everyone had separate templates/ and controllers/ folders. Both learners and teachers were focused on AngularJS, which treated HTML, CSS and JavaScript very differently.

This was okay for demos and small projects, but as real-world AngularJS applications grew, developers spent most of their time editing files in completely different folder paths. This was extremely annoying.

Pretty much all at once, everyone in the Angular community decided to refactor to a separation by components instead, where each component had its own folder with all its relevant files, regardless of the fact that multiple technologies were being used.

State

When state management became a primary concern with SPAs, developers felt the need to create a separate folder for managed state. This means in order to work on a single feature, you needed to spend most of your time editing files in completely different folder paths. This was extremely annoying. Unfortunately, people blamed Redux instead of the bad folder structure.

But a global store is an implementation detail, and nothing more. The fact that reducers/states/slices are ultimately collected into a single, global object is only a mechanism for reactivity and Redux Devtools. Any code that assumes it can just grab any state from anywhere in the global store is going to be bug-prone, so each store slice needs to export its own selectors/actions to interface with instead, which entirely abstract away the global store behind them. In the end, you just have a bunch of independent features, like always. Your folder structure should reflect that.

Types

When most Angular developers learned TypeScript in 2016, for a short period of time they wanted to put all their types in a central types/ folder. They have almost completely moved away from that, because if a developer needs to edit a type, they are guaranteed to need to edit other files in their project, which would require editing files in completely different folder paths, which is extremely annoying. React developers adopted TypeScript much later, and many new React projects are still set up with a central “types” folder. This will not last very long, just like it didn’t in the Angular community.

This will always happen

Whenever there is a new technology, there is always a strong influence from tutorials and content creators to separate by technology instead of by feature. If the cost isn’t too high, this naïve strategy will remain a habit for some time.

New tools can be intimidating, but the actual purpose of a project quickly overshadows the technology as the primary concern.

2. Hard-to-find files

When do you look for files? It’s usually when you are

1. becoming familiar with a project
2. debugging an issue

Becoming familiar with a project

If a project is your first encounter with a technology, you will want to see that technology reflected in the folder structure. But you are not going to understand much about what you are looking at, no matter how the folders are structured. The technology-based organization will give a slight sense of familiarity, but it won’t help you understand the actual project in front of you any more than picking an arbitrary file that uses that technology and trying to understand that.

If you want to see examples of a technology, you can just search for files with the extension for that technology, such as .slice.ts. File extensions indicate what technology is being used, and it's the last part of the file name because it's the least important detail.

If you are experienced, you will want the details of the project to be reflected first.

Debugging an issue

When debugging an issue, you start by seeing a broken feature. Problems are connected to features, which require multiple technologies to define, and, therefore, can be messed up by multiple technologies. Horizontally scanning all files by technology is useless for debugging anything but the technology itself. What you want is good developer tools that lead you through the technology layers, from the problem to the cause.

Let's say something is wrong in the UI. You start by inspecting the HTML. Then you use your framework's devtools to go directly to the component. Then you "click to definition" to a state management file, and continue until you find the problem. Now, if your folder structure is organized by technology, you will have jumped between 5 totally different folders. If it's organized by feature, you might not even have had to open more than 1 file, let alone folder. And maybe you even immediately saw the problem when you opened the component file, because every layer defining the feature was already there in front of your eyes.

3. Reusability confusion

If your code is organized by technology rather than by feature, you may be tempted to reuse one slice of that technology layer across multiple features. This seems convenient at first, but it introduces unnecessary complexity. Each feature is an independent concern, and is likely to drive changes across multiple technology layers that don't make sense to have in other features.

I was once a naïve Angular developer arguing for keeping HTML and JavaScript separate. I thought it would be good to have the flexibility to have multiple templates available for a single controller. The controller was responsible for managing state, and the template was responsible for displaying that state. Those are separate concerns, right?

But I have been writing component-only UI code for 8 years now, and never once have I taken advantage of Angular's separation of HTML and JavaScript; I have never used the same template in 2 different components, nor have I used the same component class in 2 different components.

HTML and JavaScript are different technologies, but your primary concern should be vertical slices of functionality that may span multiple technologies. These vertical slices are likely to change independently from other slices.

Reusability comes from having thin vertical slices that can be imported by other thin vertical slices. And these slices should be organized around a single type/interface, because that is the foundation that everything else is built on: When a type changes, everything that references it needs to change too. Ideally, those should be in the same folder. Even if your vertical slice extends all the way up to "global" state or even server-side code, it should be in the same folder as the primary type it uses. And if the feature is small, maybe it should even be in a single file.

4. Slow load times

Developers often prefer horizontal separation because they never have to worry about dependencies. They can just import whatever they want, because layers from above never import from layers below. At the top you have types, which are extremely abstract; then there are utilities; then you have state; then components.

Because of this, circular dependencies are easily avoided when code is separated by technology. So this is the one problem separation by technologies seems to handle very well.

But this is more of a strategy of avoidance than a solution.

A circular dependency is when multiple things have actually become one thing.

The reason we want our build tools to warn us of this is so we don't accidentally create giant things and make our users download more code than necessary. Separation by technology is just an escape hatch that allows us to be complacent about large bundles, as long as they only include code from the same horizontal layer (technology).

Users only need the code for the features on the current page, so we want our code bundles to only include what is needed for each feature. If it's okay to import anything from the technology layers above, there will be no friction to prevent developers from coupling each feature to every other feature. We want our tools to prevent us from creating circular dependencies between features, not technologies.

How do you deal with circular dependencies between features? You create a 3rd feature that the other features import from. Just because it doesn't plainly exist in the designs or in a route, doesn't mean you can't have a folder for it. You should. Every unique type can have its own folder, and you will never have circular dependencies. It doesn't have to have API, state or UI files, but you will have a place for them later.

5. Slow build times

Smaller bundles are great for users, but they are also great for developers, because they mean faster build times. With Nx, for example, running nx affected:build will build only the code related to the feature you've been working on, if libraries are organized by feature. But if you have a types library, for example, and you make a change to just one type inside it, then every library that depends on any type needs to rebuild. This will cascade to a huge amount of code totally unrelated to your changes in a specific feature. In large projects this can extend build times by multiple orders of magnitude.

I may or may not have written that last paragraph while waiting for a build.

6. Huge folders

Imagine you have 5 features, each with a component and a Redux slice.

This is a folder structure organized by technology:

store
    a.slice.ts
    b.slice.ts
    c.slice.ts
    d.slice.ts
    e.slice.ts
components
    A.tsx
    B.tsx
    C.tsx
    D.tsx
    E.tsx

This is a folder structure organized by feature:

a
    a.slice.ts
    A.tsx
b
    b.slice.ts
    B.tsx
c
    c.slice.ts
    C.tsx
d
    d.slice.ts
    D.tsx
e
    e.slice.ts
    E.tsx

When we organize by technology, we have more large folders than when we organize by feature.

However, both have the problem that there is at least 1 folder with 5 things in it. If we keep adding features, the folder will keep getting bigger.

One solution is to group the features into scopes or categories. So let's say A, B and D are related somehow, and we can change the folder structure to this:

some-commonality
    a
        a.slice.ts
        A.tsx
    b
        b.slice.ts
        B.tsx
    d
        d.slice.ts
        D.tsx
c
    c.slice.ts
    C.tsx
e
    e.slice.ts
    E.tsx

If we were organizing by technology, we would have to create this nested structure in 2 different places:

store
    some-commonality
        a.slice.ts
        b.slice.ts
        d.slice.ts
    c.slice.ts
    e.slice.ts
components
    some-commonality
         A.tsx
         B.tsx
         D.tsx
    C.tsx
    E.tsx

That's annoying.

Another way is to nest the store and component folders like this:

some-commonality
    store
        a.slice.ts
        b.slice.ts
        d.slice.ts
   components
        A.tsx
        B.tsx
        D.tsx
store
    c.slice.ts
    e.slice.ts
components
    C.tsx
    E.tsx

This is a little better, but now we're going to be repeating store and components folders a lot, which gets annoying to deal with.

Ultimately, organizing folders by feature is the least annoying strategy.

Organizing by routes

Features usually correspond with routes, but as projects grow, this 1-to-1 relationship has exceptions. Often a single feature needs to appear inside multiple routes.

It is fine to start by organizing files by routes first, since there are some benefits to doing it this way. However, it is important to realize that nesting a folder slightly couples it to its parent feature, and as soon as it needs to be reused, it needs to be moved to a top-level folder where it can be imported into both routes.

Directory trees are inherently limited

Organizing folders by feature is better than organizing by technology, but ultimately there is no great way to organize code in a folder structure.

If you think about the actual code you are writing, what is its relationship with other code? It imports it, right? It gets imported, too. And that creates a dependency graph. And if we try to fit a graph into a directory tree, we are going to have a bad time.

Imagine if we didn't have to deal with this fundamental problem!

Every once in a while developers will try to use symlinks to make the folder structure behave like a graph instead of a tree, but these are irritating to deal with, so this pattern never catches on.

What we need is a solid abstraction on top of files that allows us to manage our assets as a graph instead of a tree. I think Nx Devtools' dependency graph visualization is awesome, and could maybe do the job if it could be used to open the files it's visualizing, maybe as a VSCode extension. Then you could completely ignore the underlying folder structure.

We also need the ability to create relationships that are purely organizational. Sometimes it's helpful to be able to hide code inside a parent folder with a more abstract name. This relationship doesn't come from importing the code, but from an implicit relationship between the code and code that's similar to it. But this relationship should be a graph too, because there are multiple ways to categorize all code.

For example, you could have a file called concat-array-strings.function.ts. Would you put that in a string folder, an array folder, or a utils folder? A utils folder is like organizing by technology (or lack thereof), so it is not ideal in my opinion. But the point is that there are potentially multiple buckets you could put a concat-array-strings.function.ts file in, so why do you have to choose? Each of these potential parent folders is just abstracting out a different property of the code, when in reality it is all of those things.

This might be getting a little too philosophical now. I have many ideas about how code could be organized. But for now, I would just be happy if people would start organizing their folders by types more instead of by technology. Maybe some content creators will see this and realize they have been misleading new developers this whole time.

Conclusion

There is no folder structure that is perfect, but some are definitely better than others. You should realize that what you are concerned with at the beginning of a project may not be your primary concern later on, and you may later regret the decisions you make.

Ever since I started organizing my project folders by feature instead of technology, getting stuff done has been less annoying. If you haven't tried it yet, please do. And again, comment below if you think you have an example that I have not thought of.

And if you agree, please share this with your team!