DEV Community

Cover image for Creating a Laravel Artisan command to import posts from DEV via the command line
Sourcegraph profile image Erika Heidi
Erika Heidi for Sourcegraph

Posted on • Edited on

Creating a Laravel Artisan command to import posts from DEV via the command line

#php #laravel #tutorial #beginners

In a previous part of our series, we saw how to install and configure TailwindCSS 3.0 within a Laravel project and how to set up a basic blog layout for our demo application. With that base template in place, the fourth part of the series is focused on the back-end side of the application.

In this post, you'll learn how to create an Artisan command in Laravel to import posts from DEV using their API and how to save the content as markdown files in your local application storage folder. This recap is also available as a video that you can watch on YouTube.

Preparation

Before moving along, make sure you have followed all steps in the first tutorial of the series in order to bootstrap your Laravel application and development environment. This will require you to have Docker, Curl, and Git installed on your system.

If you haven't yet, bring your environment up with:

./vendor/bin/sail up -d
Enter fullscreen mode Exit fullscreen mode

This command will start up your environment and keep it running in the background. Your Laravel application should now be available at http://localhost from your browser.

This tutorial assumes that you have an alias to ./vendor/bin/sail on the root of the application folder, so that you can run Sail with ./sail. You can create such an alias with the following commands, executed from the application root folder:

ln -s ./vendor/bin/sail sail
chmod +x sail
Enter fullscreen mode Exit fullscreen mode

1. Creating a new Artisan command

Start by creating a new Artisan command using the make:command utility. This will create a new file with boilerplate code for an Artisan command, saved at app/Console/Commands/.

./sail artisan make:command ImportPosts
Enter fullscreen mode Exit fullscreen mode

Open the newly generated file app/Console/Commands/ImportPosts.php in your code editor, and you'll see content like this:

<?php

namespace App\Console\Commands;

use Illuminate\Console\Command;

class ImportPosts extends Command
{
   /**
    * The name and signature of the console command.
    *
    * @var string
    */
   protected $signature = 'command:name';

   /**
    * The console command description.
    *
    * @var string
    */
   protected $description = 'Command description';

   /**
    * Create a new command instance.
    *
    * @return void
    */
   public function __construct()
   {
       parent::__construct();
   }

   /**
    * Execute the console command.
    *
    * @return int
    */
   public function handle()
   {
       return 0;
   }
}
Enter fullscreen mode Exit fullscreen mode

Start by changing the command:name for the command signature you'd like to use. In this demo we'll use import:dev.

The handle() method is where the command action happens; this is what will be executed when you run artisan import:dev from the command line.

2. Obtaining latest posts from the DEV API

Luckily for us, DEV has an API that we can query to obtain the latest posts from a user, without the need to use an authorization token, since that is all public data. The endpoint we want to query is:

https://dev.to/api/articles?username=DEV_USERNAME
Enter fullscreen mode Exit fullscreen mode

To make the request, we'll use minicli/curly, which is a small PHP library based on curl. Require that dependency with:

./sail composer require minicli/curly
Enter fullscreen mode Exit fullscreen mode

Once you have the dependency installed, add a use directive at the top of the file so that you can easily reference Minicli\Curly\Client in your code. We'll also use Laravel's Carbon/Carbon library to parse the article's publish date, so you should include that within your use directives as well. Lastly, include the Illuminate\Support\Facades\Storage class with another use directive, since we'll be using this facade class to check and save files within the local file storage system.

This is how the first lines of your app/Console/Commands/ImportDev.php file should look like:

<?php

namespace App\Console\Commands;

use Illuminate\Console\Command;
use Minicli\Curly\Client;
use Illuminate\Support\Facades\Storage;
use Carbon\Carbon;
Enter fullscreen mode Exit fullscreen mode

Next, update your handle() method with the following code:

public function handle()
{
   $crawler = new Client();
   $headers = [
       'User-Agent: curly 0.1',
   ];

   $articles_response = $crawler->get('https://dev.to/api/articles?username=DEV_USERNAME&per_page=10', $headers);

   if ($articles_response['code'] !== 200) {
       $this->error('Error while contacting the dev.to API.');

       return Command::FAILURE;
   }

   $articles = json_decode($articles_response['body'], true);
   foreach ($articles as $article) {
       $article_slug = $article['slug'];
       $published = new Carbon($article['published_at']);
       $filename = $published->format('YmdH') . '-' . $article_slug .'.md';

       if (Storage::disk('local')->exists($filename)) {
           continue;
       }

       $endpoint = sprintf('https://dev.to/api/articles/%s', $article['id']);
       $article_query = $crawler->get($endpoint, $headers);

       if ($article_query['code'] == 200) {
           $article_full = json_decode($article_query['body'], true);

           Storage::disk('local')->put($filename, $article_full['body_markdown']);

           $this->info("Article saved: " . $filename);
       }
   }

   return Command::SUCCESS;
}
Enter fullscreen mode Exit fullscreen mode

This code will make a request to the DEV API to obtain a list with the latest 10 articles from a user. To get the full contents of the article, including the article's metadata defined in the front matter, we'll make a second request once we check that the article hasn't been imported before. Then, we'll use the Storage utility to save the body_markdown of each article as a markdown file in the application's storage folder. The files are saved using a combination of the article's publish date and the article slug as filename, with the .md extension.

Don't forget to change DEV_USERNAME to your own username on DEV.to.

Save the file when you're finished.

3. Test the command to import posts from DEV

With the command ready, you can now run it with:

./sail artisan import:dev
Enter fullscreen mode Exit fullscreen mode 
Article saved: 2021121515-setting-up-tailwindcss-30-on-a-laravel-project-1cb8.md
Article saved: 2021121013-creating-a-new-laravel-application-with-sail-and-docker-no-php-required-4c2n.md
Article saved: 2021120318-how-to-set-up-elgatos-stream-deck-on-ubuntu-linux-2110-pdh.md
Article saved: 2021112011-dynamic-twitter-header-images-with-dynacover-github-action-35nb.md
Article saved: 2021111818-how-to-build-github-actions-in-php-with-minicli-and-docker-1k6m.md
Article saved: 2021111618-automatically-update-your-contributors-file-with-this-github-action-workflow-d98.md
Article saved: 2021102717-10-sourcegraph-search-tricks-for-open-source-contributors-and-maintainers-44n9.md
Article saved: 2021092715-3d-design-creating-printable-solid-shapes-from-svg-files-with-inkscape-and-freecad-266e.md
Article saved: 2021090615-3d-design-with-freecad-part-2-working-with-sketcher-part-design-workbenches-3leo.md
Article saved: 2021090614-an-introduction-to-3d-design-with-freecad-part-1-navigation-3gjo.md
Enter fullscreen mode Exit fullscreen mode

You should see the titles of your latest posts showing up in the output of the command. Once the command is finished running, have a look at your storage/app folder and you'll see your posts as individual markdown files named after each post's publish date and slug.

ls storage/app
Enter fullscreen mode Exit fullscreen mode 
2021090614-an-introduction-to-3d-design-with-freecad-part-1-navigation-3gjo.md
2021090615-3d-design-with-freecad-part-2-working-with-sketcher-part-design-workbenches-3leo.md
2021092715-3d-design-creating-printable-solid-shapes-from-svg-files-with-inkscape-and-freecad-266e.md
2021102717-10-sourcegraph-search-tricks-for-open-source-contributors-and-maintainers-44n9.md
2021111618-automatically-update-your-contributors-file-with-this-github-action-workflow-d98.md
2021111818-how-to-build-github-actions-in-php-with-minicli-and-docker-1k6m.md
2021112011-dynamic-twitter-header-images-with-dynacover-github-action-35nb.md
2021120318-how-to-set-up-elgatos-stream-deck-on-ubuntu-linux-2110-pdh.md
2021121013-creating-a-new-laravel-application-with-sail-and-docker-no-php-required-4c2n.md
2021121515-setting-up-tailwindcss-30-on-a-laravel-project-1cb8.md
public
Enter fullscreen mode Exit fullscreen mode

Conclusion

In this article, which is part 3 out of 4 in our Getting started with Laravel series, we've seen how to create an Artisan command and how to import posts from a user of DEV.to using the DEV API and the minicli/curly library. The command uses the underlying storage system from Laravel to store the posts as individual markdown files in your storage/app folder.

In the next part of this series, we'll finish this demo application by updating the index controller and listing the posts on the main application's page.

You can follow Sourcegraph on Twitch to be notified when we go live. Follow our YouTube channel for the full recap videos of our livestreams.

Top comments (5)

Collapse
 
arielmejiadev profile image
Ariel Mejia •

Why do you prefer curly over HTTP new wrapper? there is some additional benefit?

Collapse
 
erikaheidi profile image
Erika Heidi •

No particular reason, it's a simple library I built that doesn't have any other dependencies and I'm very used to it.

Collapse
 
arielmejiadev profile image
Ariel Mejia •

Thanks for reply, it makes sense

Collapse
 
msamgan profile image
Mohammed Samgan Khan •

nice work here erika.

Collapse
 
erikaheidi profile image
Erika Heidi •

Thank you! <3

Read next

imsushant12 profile image

Deployment of a Node.js Application on AWS

Sushant Gaurav -

codewithsadee profile image

How to Build AI Chat App Using React, Tailwind and Framer motion

Sadee -

pauq profile image

Scraping New Telegram Channels

pauq -

koolkamalkishor profile image

Evaluating Medical Retrieval-Augmented Generation (RAG) with NVIDIA AI Endpoints and Ragas

KAMAL KISHOR -