GSoC 2019 (6 Part Series)
I'm back, as promised! Time to talk about what I have been doing all this time. Three weeks have passed since the coding period started and some work has been done since then:
This was my first module! I tried to do a basic tool to check if some files stopped being generated. It needs improvement and it's not finished yet, but my mentor recommended me to leave this project aside for a while.
Mini doc repository issue #2529
As you can see in the doc directory of Perl6 documentation repo, there are 383 pod files approximately. It takes quite a long time (~20 minutes) to process them all and build the doc site. This leads to one problem: if you want to make any change to the build process, first you need to follow these easy steps:
- Code the change.
- Build the site waiting 20 minutes.
- Yep in this step you are still waiting.
- See that you forgot a semicolon, come back to step 1.
Quite frustrating, isn't it? So, in order to change this, as discussed in issue #2529, a mini doc repository would be quite helpful. So I create it here, but, once the environment was set up and I started to search for the correct self-contained subset of pod6 files, I realized I did not know what needed to be in it! So to discover what was necessary I did the following:
Spin off Perl6::Documentable Issue #1937
In order to know what we need, first we need to know what it's being used and how. So, how are pod files being processed and where?
First they are read and parsed, then we get a data structure of
Pod::* objects in an array. These objects contain the text written in the pod files, but to generate indexes and have a search system, an additional layer is necessary. That's where
Perl6::Registry (source code) take part. A
Documentable object is whatever piece of a pod file that is documented, but, what's being documented? Several things:
- Pod files: every pod file in the doc directory is represented by a
Documentableobject with the following attributes:
$name: By default is set to the filename. If the pod is representing a type, then it's set to the last word of the filename. Otherwise, it's set to the content of
$url: This is set to
$kindis the name of the folder where the pod file is allocated and
$linkcan be set to the value you want adding
link<your-link>just after of
=begin pod. If you don't, then it's set to the filename.
$pod: array containing
Pod::*objects with the data.
$summary: value of
$kind: is the name of the folder where the pod file is allocated (
$categories: If it's a type, set to the value obtained from
Perl6::Typegraph. If not, then
$subkindis set to
$categoriesis not set.
pod-is-complete: Set to
truebecause they represent an entire pod file.
When all of this is set, the new
Documentable object is added to
Perl6::Registry, which is that exactly, a registry where all
Documentable objects are stored to be used later.
Next thing that needs to be done is search for definitions to be indexed. This a key part of the process because this definitions will be used to create TOCs, a search file for the site and intra-links. Let's start explaining what is a definition: a definition is a header, more precisely, a
Pod::Heading object. They are something like this:
=head2 Some text (the heading level does not matter). But as you can suppose, not all headings are valid definitions. There are two different types of valid ones: ambiguous and unambiguous. You can check more info about them here. Every new found definition is stored in a
Documentable object and added to the registry.
But there's more: references. A reference is a
X<> element. All and each one of
X<> elements are processed as references, stored in a
Documentable object and added to the registry as well.
Well, all this work is made in
htmlify.p6 by three different funtions:
If you take a look at that functions, you will see that global variables are involved. Moreover, that code could be parallel (currently is parallel but the number of threads is set to 1). And, as you may know, global variables and parallelism, need monitors or some alike exclusion mechanism. It looks like monitors are not working as expected (by that reason the numbers of threads is set to 1). To fix that, I have moved all the logic described here to the
Perl6::Documentable class, where there is not any global variable so no monitors are needed. You can check the new module here.
Perl6::Typegraph Issue #2573
I spinned this module to test it and document it. In addition, we want to get rid of the
type-graph.txt file and get all the information necessary through introspection. This file should be added to that module too.
This module is going to be used to read every pod file in the documentation, so I have been using it and finding some bugs.
Three days ago was #esLibre19! A congress about open source where I hosted a devroom about Perl and Perl6!
This week I want to focus in several things:
- Publish and polished
- Spin off
Pod::Convenience, which will be renamed to
Pod::Utilities. In addition, all scattered functions related to this topic found in the repo, will be moved there to keep consistency.
- Discuss where to move logic that reads and initializes the pod and the registry (it's not trivial because in the next weeks a cache system will be made and that logic will use it).
Perl6::RegistryToDocmodule, where the rest of the logic of
htmilify.p6will be moved.
I have not explained several things, but if you go to the repository of each module, you will find a documentation explaining all the necessary!
And that's all for this week! See you next week, hopefully :D.