System Documentation 101

Introduction

Have you ever been in a situation where you needed to get started on an ongoing project or even maintain an older system, and instead of getting right into the work you had to spend hours navigating through a pile of information just to understand where to even begin? Well, I know I have, and tell you what, it wasn’t fun… even worse, wasn’t just once!

We understand the nature of this dynamic world of software development, where lines of code are constantly shaping the digital landscape, it feels like the fast-paced environment often prompts us to skip certain steps to swiftly reach the final goal and meet the almost impossible deadlines set by customer expectations, and often documentation is the first to go. However, the importance of having a robust system design documented cannot be overstated.

Let’s explore the critical role of system design in the development life cycle and delve into the often-underestimated importance of documentation in ensuring the longevity, adaptability, and collaborative success of software projects. However, if you think this does not apply to you or your project because it has such a small scope, like building a microservice, and you’re assuming you don’t really need to come up with a design or a document, the truth is, even if it only stays in your mind and you don’t realize it, you are, in fact, designing the system, so you might as well document it!

What is System Documentation?

In a nutshell, it is the collection of comprehensive documents describing the inner workings of a system, as well as, the design and even business decisions surrounding it. It can be thought of as the blueprint that will guide developers, support staff, stakeholders, and even end-users. Without documentation, you may find yourself and your team navigating uncharted waters and struggling to stay afloat whenever there is a need to perform maintenance, debug issues, or implement new features.

These documents come in all shapes and forms and navigating all these formats we can choose from is overwhelming, on top of that picking the wrong one could get you wasting time on an ineffective document. Selecting the right type will come from experimenting with documentation yourself, and the more you do it the easier it becomes, like most things in life. Nevertheless, I understand not everyone has the time to navigate these waters through trial and error, therefore I hope to bring some clarity on this subject and help you make the right decision from the get-go.

Yet, I will ask you not to be discouraged, even if mid-way through your document you realize it’s not going to work the way you expected! Don’t be afraid to start over, and you will learn that the next document you write, not only will be easier and faster but will also be much more efficient.

Now let’s talk about some of these types and, hopefully, you’ll have enough information to make the best decision for your project.

Text Documents

This is the most common type of document, and probably the easiest one to produce. Most common when documenting specification requirements, system design, user manuals, and technical reports. It is typically created using word processors or plain text editors, it is also the preferred format when writing README files for code repositories.

- Target Audience: Project managers, developers, testers, and sometimes even the end-user, depending on the scope.
- Pros: Flexibility to adapt around the project scope and target audience; allows for thorough explanations; easy to produce and read.
- Cons: When the scope is too large it will become dense and extensive; hard to illustrate complex concepts, relationships, and flows; reliant on common language understanding.

Spreadsheet Documents

Well, these are very straightforward, they are very useful when dealing with data related to the project and sometimes even to track statuses if your team is lacking better tools, they can contain a matrix of decisions, test result reports, checklists, and many other useful structures.

- Target Audience: Project managers, developers, testers, and, sometimes, stakeholders.
- Pros: Familiar format; suitable for calculations and organizing data; easy visualization.
- Cons: Can become too complex for large-scale systems.

Diagrams and Visuals

My favorite form of documentation, by far. Is not as easy to produce for beginners but can be the easiest to read. Encompasses charts, flowcharts, UML diagrams, ER diagrams, wireframes, and visual representations of system architecture.

- Target Audience: Developers, project managers, and stakeholders.
- Pros: Allows you to convey complex information intuitively; facilitates understanding and discussion; is not limited by language barriers.
- Cons: Can quickly become outdated if not maintained as the project grows; requires in-depth knowledge and experience to produce.

Code Comments

A very niche form of documentation, these are notes embedded directly in the source code that explain the functionality. They are very useful for projects on a smaller scope as they would make up for the lack of external documentation, which could increase the velocity of the project without causing maintainability issues.

- Target Audience: Developers and those maintaining the code.
- Pros: Provide a critical understanding of complex code logic and sections; eliminate the necessity of external documentation.
- Cons: Extensive comments can clutter the codebase causing code readability issues; can easily become outdated if the code changes without maintaining the comments; unnecessary on a well-organized and clean codebase.

Presentations

These are very useful as project starters, as they can be produced quickly and can convey the project points concisely and directly, they will also guide the discussions on the project requirements and the direction of the more in-depth documentation.

- Target Audience: Stakeholders, project team members, end-users (depending on content).
- Pros: Provide excellent high-level discussion points; good for conveying summaries; good for engagement due to its visual medium focus.
- Cons: Have limited details; cannot be used on their own as they require inputs from a presenter.

Video Tutorials

Are not the usual kind of documentation, although they do have their benefits, they have a limited scope of uses, normally as onboarding or enablement sessions.

- Target Audience: End-users and new developers.
- Pros: Visual and engaging, good for demonstrating detailed steps; can save the time of senior resources during onboarding.
- Cons: Take a lot of time to produce and update; not good for quick references.

Knowledge Bases (Wikis)

These are platforms that can contain a collection of documents and can be edited collectively. They serve a similar purpose as text documentation, and can also include direct links to other documents. Very useful when sharing publicly accessible documentation.

- Target Audience: Project teams and end-users (public only).
- Pros: Promote collaborative knowledge building and updating; can centralize a multitude of information; allow for fine-tuned control of access to information.
- Cons: Require moderation to ensure information quality.

API Documentation

This is essentially an instruction manual for an Application Programming Interface (API) service. Describes functions, parameters, and expected responses of APIs. Can contain examples, screenshots, tutorials, and anything that may be useful in explaining the API's functionality, it is essential for external integration and understanding API usage.

- Target Audience: Developers using the API.
- Pros: Enables third-party developers to use the API; increases the speed of onboarding of new developers.
- Cons: Requires precision, consistency, and ongoing maintenance.

Prototypes

Early versions or mockups of software with a focus on usability or core functionality. Can be created as a part of the development process, or as a standalone project to test concepts and prove values.

- Target Audience: End-users, stakeholders, and developers for gathering feedback.
- Pros: Provide hands-on user feedback early; help catch UI/UX issues before the full implementation; help validate core functionalities.
- Cons: Require upfront investment of time and resources;

Why is documentation so important?

During the system design phase, critical decisions regarding the architecture, structure, and functionality are made to ensure that, in addition to meeting the desired requirements, the system will also empower developers to anticipate challenges, plan for scalability, and create a framework that is both flexible and adaptable to future changes. Those specifications might give an indication of the complexity you will face throughout the project and while you may possess an impressive brain power to keep on top of everything you do, there is just so much data our minds can hold at a time.

Recording all this information may seem like just an additional task on your already busy schedule, however, bypassing documentation and relying solely on memory to ensure the robustness and consistency of a system is not a sustainable long-term strategy. You will eventually realize that developing strong design and documentation skills offers numerous benefits that make it a valuable investment of effort and time. I will highlight some of the reasons I have found to be the most crucial and compelling, based on my own experience.

Communication

Clear documentation is an essential tool in ensuring effective communication among team members and stakeholders. It helps convey the architectural decisions, design principles, and overall structure of the system. Considering the scale of the tech industry today, where you will most likely be working with people from all over the world, documentation is also an invaluable tool in bridging any language gaps.

Maintainability & Troubleshooting

Legacy code is one of the biggest issues when implementing new features to an existing system, which makes documentation vital for maintenance. It allows developers to understand the existing design, making it easier to identify areas that need updates, enhancements, or bug fixes while ensuring the current features and code are not affected by the changes. Sometimes things do go wrong, and being able to react fast to issues is critical and it is only possible with a well-documented system.

Scalability

A well-documented system design gives fundamental insights into how different components are interconnected, helping you identify possible stress points in the system and plan for future growth and on-demand requirements.

Onboarding & Knowledge Transfer

Working as a team is not always straightforward. There are situations where time constraints require us to collaborate with others to speed up the development process, but this can be a double-edged sword as onboarding can be time-consuming, not only for the new resource joining but also for the project lead who needs to ensure the knowledge transfer. This can be mitigated by having system documentation ensuring effective knowledge transfer and faster onboarding of new developers or consistent continuation of a project whenever replacement of people is necessary.

Quality Assurance & Testing

Ensuring the quality of a system is one of the most important steps in its life cycle. This will give the final product a strong competitive advantage, as a seamless user experience goes a long way for the reputation and trust of the software, while a bad one can cause irreparable damage to it. When planning the required tests to ensure quality, the documentation will allow the Quality Assurance (QA) team to navigate the details and complexities of the system, while executing and reporting on the tests.

To further optimize the testing capabilities of the team, the developers can use the documentation to establish unit tests, these are automated and will ensure the reliability of a system before even getting to the QA team. This is the foundation for Test Driven Development (TDD).

Decision Making & Consistency

System design will capture all the relevant details, specifications, and decisions regarding a particular software, and I can’t stress enough the importance of recording all this information. The resulting documentation will prove highly valuable when revisiting or reevaluating the system architecture, as it provides context for past choices, which will also be indispensable in guiding future improvements and supporting well-informed decision-making.

It will also enable regularity in the development cycle, ensuring a uniform and steady adherence to the decisions and standards for the project, delivering a stable level of performance, no matter what point of the life cycle the project finds itself in, as it will always allow you to come back into the project and pick up exactly where you left off, without the risk of forgetting critical information and the need to reconstruct the system design mentally in order to get started again.

Cost Reduction

All the time cost that exists in software development can be greatly reduced via documentation. While this may seem counterintuitive, as documentation also takes time to be produced, I assure you, the many more hours that you will spend looking for information, onboarding resources, refactoring the system, or fixing mistakes, will far outweigh the time spent on it.

Where to begin?

Okay, so I've convinced you of the importance of having system documentation, or at least I hope I have… But how do we even get this started? Well, there are a few things we need to clarify first: Who is this documentation intended for? Are they technical? Is it for a customer or the internal team?

These definitions will be your starting point for deciding the format of your documentation, although if you are in a leadership position where your job is to talk to customers, understand the requirements, plan the system, and then convey to both the client and the development team that is going to be working on this, chances are you’ll have multiple answers to the questions above and that means you will most likely need more than one type of documentation.

If you end up producing multiple documents you do need to make sure they are consistent throughout the project no matter who your target audience is or the form the documentation will take. The next section will cover how to do just that!

Stick to Best Practices

I’d like to argue that, even though starting documentation from scratch can be daunting, it can also be an opportunity. By following best practices you’ll ensure good quality documentation from the beginning of your project, and by sticking to them you will make sure to maintain those high standards when updating your system and documentation.

And what are these best practices? Well, first and foremost, whatever is agreed with the team you are working with. As they will be the main stakeholders of the documentation it is imperative that everyone is aligned and on board with the best practices to follow when documenting, some companies even have standards for documentation, especially the ones geared toward clients, and that should be your starting point.

However, if you are truly starting from scratch and have no idea what these best practices can be, here are some pointers I can give based on what has worked quite successfully throughout my career.

Terminology

Using clear and consistent terminology and notation will ensure the effective communication of your document. Stay clear of jargon, acronyms, and ambiguity, your objective is to be understood by the readers, and introducing terms they might not be familiar with will only cause confusion. I do understand that sometimes specific terms are required for certain projects but whenever that is the case make sure to define them in your documents’ glossary so your audience knows what it is about. This is particularly important when working with multilanguage teams.

Organization

Keep your documentation very well organized. Make use of headings, sections, subsections, table of contents, index, and glossary as these tools will help navigate the document and contribute to clarity and ease of access to your documentation. A well-organized document will help developers find the relevant pieces of information much quicker, boosting overall efficiency and speeding up the development process.

Format

Finally, make sure you choose a suitable format for the documentation, one that fits the project requirement and the target audience. In my opinion, this is a critical part of the process and, sometimes, a difficult one. As I’ve shown you, there are several documentation types to choose from, each serving its purpose and they can even be combined for a more comprehensive document. So, take your time going through that list to make sure you are going in the best direction for your project.

Keep it Up To Date

An outdated documentation can do more harm than good. While keeping the documentation up to date is arguably the most important best practice, I believe it deserves its own highlighted section. Imagine yourself trying to navigate a city with an outdated map, I bet you can picture yourself going into numerous dead ends, or even worse, getting on the wrong way of a road. Of course, there are signs that can point you in the right direction, but that might not always be the case either. You’ll find that software development is not much different.

I’ve worked with outdated documentation before, while it is very distressing to try and navigate a system code or understand the expected functionality without documentation, having the wrong direction can set you back quite a bit, not to mention you could be actively introducing issues into the system and you might have to deal with messages from clients enquiring why something they asked to be changed now is back the way it was.

This is all easily preventable by ensuring the documentation is up to date for every development cycle, be it during bug fixes or new feature requests. It will not only save you a lot of headaches and budget, but more importantly, time!

What's next?

I know… there is a lot to take in. This is the start of your project and will help you shape it moving forward, so the best thing you can do is just get started. Don’t be afraid to experiment, it is better to have to start a documentation over than to refactor an entire system.

If you are still unsure of how to proceed or what format of documentation will be the best for you, I would like to recommend a visual one, using charts and diagrams can be less overwhelming and a lot more fun! Not to mention they can bring more clarity to the more complex interactions of your system and they are very accessible to various levels of technical knowledge.

A really good way, I’ve found, to practice documenting is to work on an existing system, preferably one that does not have documentation at all. Take a look through it and try to write down what you see, be it the architecture, code structures, services interaction, connected APIs, everything. If this is a project you were asked to maintain, even better! Now you have up-to-date documentation and once you start working on the task at hand, you can even complement the documents with whatever additional findings you may come across.

Conclusion

Thorough system documentation plays a crucial role in the success and longevity of any software project. While it may seem like an extra investment of time and effort upfront, the benefits far outweigh the cost. Having comprehensive documentation facilitates smooth communication, streamlines onboarding, boosts maintainability, and ensures future scalability. No matter the scope of your system, having documentation will pave the way for your project growth, and despite the overwhelming amount of information we can add to a document, choosing the right format for your needs will get you there faster. And, of course, make sure to keep it up-to-date, as it will ensure the maintainability of your system.

Now, if you are new to this and your documentation is lacking a bit at first, as long as you keep working with it and updating it as you go, you will not only improve your skills but also set yourself up for success in the long run. And always remember, your future self (and any other developer encountering your code) will thank you, so I encourage you to get started right away!