rhymes for The DEV Team

Posted on Nov 1, 2019 • Edited on Jan 23, 2020

State of the API, October/November 2019

#changelog

DEV strives towards being open, collaborative, and a positive force in the larger ecosystem.

In addition to the platform's code being open source, we also provide an API.

Anyone can generate an API key from their own user's account and start playing with it.

Some endpoints, like the list of the recently published articles, are public - https://dev.to/api/articles - and require no authentication.

Currently the API has three authentication mechanisms: API keys, OAuth2 (in private testing and limited only to some endpoints) and the HTTP session (only on those endpoints accessed by dev.to's frontend).

API keys will allow you to access both public and private endpoints.
OAuth2 is intended for third party apps that need to request data on behalf of an authenticated user.

History

The API was initially developed in the pre open source days to serve internal needs and was slowly opened to the public, adding API keys support to serve private endpoints.

Current status

The API v0 is in beta. We are committed to not breaking any of the existing interfaces or responses, which means that any possible quirks related to those are there to stay until the next version.

Though we don't currently have a deprecation policy in place for v0, we do know that we don't want to remove or change endpoints, remove fields or change their type, or do anything else that would automatically break clients.

We have several endpoints related to resources that you might recognize: articles, comments, chat channels, videos, podcasts, tags, listings, analytics (in private testing) and of course, users.

The API is edge cached, which means that it can occasionally respond with stale content. That said, the advantage to this approach is that the source of the data is near you. Some endpoints are additionally cached server side — usually those that require a lot of computation.

Areas of improvement

The API v0 could use improvement on the following fronts: fixing bugs, adding missing resources/features, adding needed fields to existing resources, improving documentation and better caching.

Known bugs

There are a few known bugs related to the API, here's a notable one:

API: can't get comments belonging to an article #2250

rhymes posted on Mar 31, 2019

Describe the bug

I can't seem to be able to retrieve articles comments from the DEV API. It works on my local installation but not using the live API.

Given the article API response https://dev.to/api/articles/95907 - https://dev.to/kathyra_/what-security-through-obscurity-is-and-why-it-s-evil-47d5 - I tried to retrieve its comments using https://dev.to/api/comments?a_id=95907 but the server returns HTTP 404.

At a first glance it should work, according to the code:

https://github.com/thepracticaldev/dev.to/blob/54ccef47669759b060370c1cde8f527ce6d39333/app/controllers/api/v0/comments_controller.rb#L15-L18

The odd thing is that such call works correctly on a local installation.

I'm probably doing something wrong on my end.

To Reproduce Steps to reproduce the behavior:

Go to https://dev.to/api/comments?a_id=95907
See error

Expected behavior

I'd expect this endpoint to return all the comments of the article with id 95907 which should be the following: https://dev.to/kathyra_/what-security-through-obscurity-is-and-why-it-s-evil-47d5

Screenshots

View on GitHub

Open feature requests

A few outstanding feature requests, for example:

Filtering articles by tag
Adding series to the API: we can currently filter by the ID of the series, but we lack CRUD operations on those
Adding CORS support to the API: ideally all public endpoints should have CORS enabled, allowing frontend code to use the API directly with JavaScript
Adding video upload to the API: right now video upload is only available through an async process via the website's UI

Documentation

The API docs are written in YAML, following the OpenAPI 3.0 spec. From there a script generates automatically the HTML and it gets updated at each deploy.

What is lacking now is actual documentation on most of the endpoints, I've opened a "meta" issue listing all in detail:

API docs: add OpenAPI documentation to undocumented routes #4474

rhymes posted on Oct 17, 2019

Is your feature request related to a problem? Please describe.

The API docs are incomplete. Right now they only document endpoints under /articles and /webhooks

Describe the solution you'd like

Many different endpoints, resources and responses are missing, the goal is to arrive to a documentation as complete as possible.

The documentation file is https://github.com/thepracticaldev/dev.to/blob/master/docs/api_v0.yml

There are a couple of requirements if we want to complete this documentation effort: knowledge of OpenAPI and coordination.

To get acquainted with OpenAPI I suggest two resources: the official OpenAPI 3 spec and the OpenAPI guide by Swagger.io. The first one contains all the info, the second one is helpful when in doubt about how to do things.

Also read the Contributing to the API spec docs page, in there you can find information about Visual Studio Code extensions that can help and how to serve the API docs locally for testing (remember to refresh the browser page at each change :)).

To coordinate I'm going to leave here the list of missing endpoints and then I'll explain where to find their code and their tests:

Endpoints to document

[x] GET /api/comments api/v0/comments#index
[x] GET /api/comments/:id api/v0/comments#show
[x] GET /api/followers/users api/v0/followers#users
[x] GET /api/listings api/v0/classified_listings#index
[x] POST /api/listings api/v0/classified_listings#create
[x] GET /api/listings/:id api/v0/classified_listings#show
[x] PUT /api/listings/:id api/v0/classified_listings#update
[x] GET /api/listings/category/:category api/v0/classified_listings#index
[x] GET /api/podcast_episodes api/v0/podcast_episodes#index
[x] GET /api/tags api/v0/tags#index
[x] GET /api/users/:id api/v0/users#show
[x] GET /api/users/me api/v0/users#me
[x] GET /api/videos api/v0/videos#index

For example, you can find the code for GET /api/comments inside the file https://github.com/thepracticaldev/dev.to/blob/master/app/controllers/api/v0/comments_controller.rb in the action index. Its test would be in https://github.com/thepracticaldev/dev.to/blob/master/spec/requests/api/v0/comments_spec.rb

I also suggest to scan the API doc file to understand how is it structured and play around with curl or other tools like Postwoman, so you can see how the responses are, for example: curl https://dev.to/api/comments/gi9b (jq is a great companion if you use the command line)

Don't forget to bump the version in your PR.

Please declare which endpoint(s) you're going to document, we have to be extra careful to avoid having conflicts in the documentation effort.

Additional context

A couple of tips: you'll probably find bugs or inconsistencies while documenting, please open separate issues for those, keeping in mind that the policy we have is:

inconsistencies that don't break the API can be fixed, please open issues and separate PRs for those, PRs tied to this ticket are only about documentation
any inconsistencies that you think should be known by the API docs reader can be documented
possible caching bugs, or other bugs, are hiding around the API, please open separate issues for those as well. There are already some documented cases of API bugs here: https://github.com/thepracticaldev/dev.to/issues?q=is%3Aissue+is%3Aopen+label%3A%22area%3A+api-v0%22

Don't forget to read the contributing guide and please use the following schema for your PRs: USERNAME/DESCRIPTION-4474 (4474 is the ID of this very issue, it makes the reviewer's job easier to know which branch belongs to whom and to which issues refers to)

We could also integrate a validator like speccy to make sure that the API doc file gets validated at each commit.

View on GitHub

Changelog

We plan to keep the community updated with a changelog post from now on, for every change related to the API.

Top comments (10)

Evaldas Buinauskas • Nov 1 '19

Curious.

How are /articles/me and /articles/me/published different? They seem to have exactly same descriptions : /

Also /articles/me/all seems awkward, shouldn't this have been /articles/me? :)

rhymes • Nov 1 '19 • Edited

Yeah, that was weird :D

The rationale is: /me defaults to published articles, drafts are an explicit permission with the OAuth2 authentication we were testing, hence the default.

If you want me/all and me/unpublished with OAuth2 you need to ask explicit permission to the user. me/published probably wasn't necessary in hindsight, since it's exactly like /me without further qualifiers

Evaldas Buinauskas • Nov 2 '19 • Edited

Feels slightly awkward, still : ) I always expect (perhaps just me) the last sub-resource to tell me what it's going to bring back, so in this case /articles/me looks like it's going to bring back information about myself.

IMHO it could be more REST-full and more elegant if it was:

/me             # Brings information back about authenticated user.
/me/articles    # To bring back information about my own articles.

And then it could have a query parameter with an enum value to bring back reports of certain status, e.g. status = [draft,published,unpublished]

And then have it as follows:

/me/articles                   # can default to all, or published
/me/articles?status=published
/me/articles?status=draft
/me/articles?status=unpublished

or accept multiple statuses:

/me/articles?status=draft&status=published

At least that's how I would model it. Other than that - it's looking good.

Great job!

Ben Halpern • Nov 2 '19

This is the conversation we should definitely have when we go to break ground on v1.

v0 is definitely gonna be a mishmash of whatever it takes to offer a technically sound API of any kind 😄

Evaldas Buinauskas • Nov 2 '19

That is totally fine, it's better to start with anything that works instead of aiming for perfection :)

rhymes • Nov 2 '19

Yeah, the next version will get closer to that, after all mistakes are a way to learn!

Ben Halpern • Nov 1 '19

Community contribution here has been amazing. By this time next year we will have a vast and sophisticated API which will allow folks to do practically anything 😄

Shiraaz Moollatjie • Nov 2 '19 • Edited

Nice update! What I also found is that if you you're writing a series, the series name is not being returned on get articles.

Another update is that there's an unofficial go API wrapper too! Check out devtogo on github.

Let's build some great apps together 😍

rhymes • Nov 2 '19

Nice update! What I also found is that if you you're writing a series, the series name is not being returned on get articles.

Yeah, that's a work in progress. We have an an issue open about this:

API: Return properties regarding series in GET request data #3669

cdvillard commented on Aug 09, 2019

Is your feature request related to a problem? Please describe.

When creating articles using the API, it's possible to assign the article to a series. However, when querying for articles from the API, no information regarding series is made available in either the objects returned /api/articles/?=${queryString} or the data returned from /api/articles/${id}.

Describe the solution you'd like

Ideally, a series property would be returned along with the rest of the data for each article from either endpoint with the title or some other identifier of the series to which the article belongs.

Describe alternatives you've considered

Additional context

View on GitHub

this lead to a PR in which we added the collection/series ID to the response and the ability to filter by it: