Documentation as Code practises

Documentation as Code. Documents are written in a domain-specific language like code, and are managed with reference to software development methods (such as source code management and deployment). It can be edited, previewed, viewed with the help of specific tools, or deployed to the server through a dedicated system. A common architectural pattern of document coding for non-technical personnel is: edit-release-development separation "

In the past month, I am developing a new document system based on Git + Markdown + DSL which is call Ledge. I customized a markdown-based markup language to support the file system of radar charts, bar charts, mind maps and other charts.

Also in the past few years, we have been discussing a variety of coding, infrastructure coding, design coding, requirements coding ... In my article "Cloud development: R & D as Code", I designed a fully coded software development process. And today we will discuss another interesting existence: document as code.

Introduction 1: Architectural Decision Record: Formatting Documents

Three years ago, when I first came into contact with the concept of "architectural decision record", I was attracted by its concept:

• Use lightweight text formatting language to describe major decisions Version along with the code
• Use a specific document format (title, context, decision, status, consequence)

Later, I wrote an ADR tool using Node.js + TypeScript. Now, in most of my open source recommendations, I will use it to manage some technical decisions. Because this document system designed based on this theory is really great, I can query:

• When a technical decision occurs and the architecture changes, the corresponding modifier
• Backtrack all technical decisions and sort out the architecture development process
• All decisions are recorded in the version control system and can be recovered
• Easy to manage and maintain

For a long-term development system, it is really very useful.

Introduction 2: Static Site Generation: Data as Code

Static site generation is a hybrid Web development method that deploys by deploying pre-built static files to allow developers to build server-based websites locally.

When GitHub Pages became the programmer's preferred blog / content / document server, he / she also adopted the technique of static site generation. Static site generation has various advantages:

• Better reliability, safety, stability, usability, etc.
• Version control
• Easy to test
• Easy to practice continuous deployment. Submit to go online
• Flexible and easy to customize

In fact, the most important thing that static site generation does is to code the data in the database. When using a CMS such as Wordpress, we store the data in the database to achieve CRUD for the data. An article becomes a fragment in the database binary file.

Subsequently, the second thing the static site generation tool did was to visualize the text content for people to read. In this way, we have achieved release-development separation.

Introduction 3: Customized markup language: expansion

When codify data, we faced a very big challenge: easy-to-write and readable markup languages ​​(such as markdown) only designed the form of the content, and lacked other information related to the content, such as creation time, author, modification time and many more.

So each static site generator customized its own markdown and added some additional information, such as hexo adopts the form of: year-:month-:day-:title.md to manage the date and title of the article. In this way, there is no need to read the Git information of this article to construct the entire information.

The same is true of GitHub Flavored Markdown, which we are familiar with, extending the markdown data field through a compatibility mode that does not significantly disrupt the content format.

In addition, we can customize charts and mind maps based on markdown data.

Introduction 4: Edit-Release-Development Separation: For Non-Technical Personnel

Designing for non-technical personnel is a major challenge for code documentation. As a programmer, we think the markdown syntax can't be simpler, but it's not the case for non-technical people. He / she needs: an easy-to-use visual programmer. To achieve such a goal, we need to make some changes in the architecture, we can try to use the "edit-release-development separation" model to solve this problem.

That is, we split the process into three steps:

• Editors can use commonly used editors or customized editors
• Developers, writing content display
• When publishing, integrate the two parts of the code

We can still choose to manage content with source code management. Just change the database interface to the Git server interface-of course they are slightly different. However, converting local Git to Git remote is basically the same.

As a result, in the end our cost will be to transform a markdown editor based on Git.

Document as code

Perfect, I once again expressed the central idea in the introduction.

Why do you need to document as code?

The main reasons are: if the documentation is not coded, there is no possibility of refactoring.

The remaining reasons are:

• Binary documents are difficult to manage version. Imagine 2020-02-30.docx and 2020-02-31.docx.
• Can't accurately know who is the editor of the document, everyone may be admin, or Zhang San at the meeting
• Can't find which is the latest document
• The documentation is terrible, but you ca n’t reconstruct the binary
• Vendor binding
• ...

There should be more.

What is Documentation as Code?

Back to the topic:

Documentation as Code. Documents are written in a domain-specific language like code, and are managed with reference to software development methods (such as source code management and deployment). It can be edited, previewed, viewed with the help of specific tools, or deployed to the server through a dedicated system.

It has such characteristics:

• Use markup language to write content. Such as markdown
• Version control can be performed through the version control system. Like git
• Programming experience consistent with programming (except that the content cannot be written for testing)

And an efficient document coding system also has such characteristics:

• Continuous deployment, that is, content can be automatically published after modification.
• Organize content indexes with specific forms. For example, to organize content in the form of knowledge base.
• Specific text format. Such as architecture decision records, static content generation, for use to provide a better user experience
• Supports REST API. To modify the content through the editor
• Can support multiple ways of output. Such as the standard HTML of the website, or Docx, Latex, etc.
• Support editing and proofreading workflow
• Support search
• Multiplayer collaboration

In fact, most of the teams do not need the advanced features mentioned above, and they already have mature solutions.

How to design a documentation as Code system?

In fact, we have marked the elements we need in four primers:

1. Use formatted documents
2. Release the system with the help of static site generation technology
3. Expand capabilities through custom markup languages
4. Implementation of the editor for non-technical personnel

Design a markup language and its extended syntax, and then implement it.

1. Establish key factors

Considering that my colleagues and I have recently implemented such a system, I still endure the pain of my hands and briefly talk about how to make such a system. The main factors we considered are:

• Chart rendering
• Flowchart rendering
• Visual display

Because the chart converted from DSL is easy to modify and can be indexed. Since then, we:

1. Expand these capabilities through Markdown's Code syntax
2. Use markdown tables and lists to provide data
3. Use D3.js to support flowchart drawing
4. Use Echarts for charting
5. Try to use SVG images
6. ...

2. Implement an MVP

We used Angular + GitHub to quickly implement an MVP:

1. We use Git as the database. It can achieve the purpose of multi-person collaboration, and can track all changes
2. We use GitHub Pages as the server. As soon as the document or code is modified, the latest document is deployed.
3. We use marked.js, which allows us to quickly expand the syntax.
4. Using textarea combined with markdown to make a simple editor.

Subsequently, we carried out a quick iteration on this basis.

3. Extended syntax

We used the markdown code as the DSL of the chart, extending some of the syntax:

1. echarts. Render Echarts charts directly
2. mindmap. Markdown List becomes mind map
3. radar. Markdown List to radar chart
4. process-table. Chart with process
5. process-step. Another chart with process
6. pyramid. Pyramid graphics
7. quadrant. Four-quadrant graph
8. class. Directly call CSS class
9. graphviz. Use Dot to render pictures
10. mermaid. Use mermaid visualization
11. webcomponents. Call the WebComponents component
12. toolset. Call Toolset related components
    • slider. Trade-off slider
    • line-chart. Chart Therefore, for users, only need to write the following code:

\`\`\`radar
  -Quality maturity evaluation model
   -Built-in quality: 3-> 4
   -Optimize business value: 2-> 2
   -Uniform quality, visualization: 1-> 5
   -Full participation: 3-> 4
   -Fast delivery: 4-> 5
   -Test as an asset: 2-> 3
   -Quick feedback: 5-> 5

config: {"legend": ["Current", "Future"]}
    \`\`\`

You can generate the corresponding chart:

We also input JSON through config and do some lazy processing (don't exhaust yourself).

3.1 Rewrite the markdown renderer

The biggest challenge we encountered in this process was that as we continued to expand the markdown syntax, the relevant code has become a big ball of mud. Therefore, we have to rewrite this part of the code:

1. Resolve token with the help of marked.js lexer
2. Generate new token based on token modification
3. Traverse the newly generated token and render the element
4. Combine virtual scrolling to solve performance problems

It has been open sourced on GitHub and the corresponding npm package has been released: @ledge-framework/render.

4. Publish this project

We have published this documented system on GitHub, and you can participate in its use and development.

GitHub: https://github.com/phodal/ledge

Project homepage: https://devops.phodal.com/

Summary

Then, you become a Markdown engineer, D3.js designer, Echart configuration administrator.