Hobby Project: Pull All Your Dev.to Articles

This concludes the test drive for the api. I needed to make some final adjustments.

https://gitlab.com/jessephillips/devarticlator/-/compare/v0.0.8...v0.1.0

The purpose of this session was to navigate to the last page and store all users articles. And while this work was easy I started realizing how broken my filtering for new articles was. The returned articles are in order of publication, I knew this, the implications is that article ID has no guaranteed order. This is why I included the date pulled, I had just thought I found a shortcut.

The unpublished articles create their own set of challenges. These articles are more likely to be changing and Dev provides a different slug than what the published article will be.

At this point I'm not solving this problem. Implications

• unpublished articles are always written
• an unpublished article which gets published will result in a new article file

One of the more challenging parts of this was keeping the isNewerArticle function pure and nothrow. It would have been easy to just perform the string conversion on the spot, but since I marked the method nothrow this action was denied.

Remember the original statement I said was my code wasn't better for adding attributes. Well now I am paying for it and I think it is for the better.

Placing the conversation in the isNewerArticlefunction causes a delay in reporting an issue. I'm now required to validate and setup my data before beginning to process it.

auto data = res.readJson().get!(Json[]);
            data.each!(x => x["published_timestamp"] = x["published_timestamp"].get!string.empty 
            ?  SysTime.init.toISOExtString : x["published_timestamp"].get!string);

When writing the unittests for the range got a little bit out of hand.

See nested functions actually come out as delegates, so I created an alias to a lambda so the compiler can determine context does not require a delegate. But like most writings, I had forgotten static nested functions aren't delegates.

alias fakeArticles = (uint page) @safe {
        return () @trusted {
        static iteration = 0;
        if(iteration++ < 2)
        return generate!fakeArticleMe
            .map!(x => cast(immutable(ArticleMe))x.deserializeJson!(ArticleMe))
            .take(3)
            .array;
        return generate!fakeArticleMe
            .map!(x => cast(immutable(ArticleMe))x.deserializeJson!(ArticleMe))
            .take(0)
            .array;
        }();
    };

I also used some unsafe calls so I wrapped the body in a trusted lambda.

I've included a readme now that a have a reasonably usable application for pulling articles. I've also registered with D's code registry.

https://code.dlang.org/packages/devarticlator

My unittests still don't cover the unpublished articles.