You can keep reading here or jump to my blog to get the full experience, including the wonderful pink, blue and white palette.
The scripting spree is not ended yet! To write the last few posts here I've fruitfully used "Scaffolding a Blog Post", "Tweeting a Blog Post via command line" and "Crossposting to DevTo via command line". Now it's time to see a script that was written for a work-related task.
In particular, we wanted to move a few of our project kanban boards from Kanbanery to Trello. Since I wrote the script after reading the awesome "Optics by Example", the code uses optics.
Let's see how the script works first:
$ ./kan2tre.hs --help
Usage: kan2tre.hs [--in CSV_FILE] TRELLO_BOARD_ID TRELLO_API_KEY
TRELLO_API_TOKEN
Moves all tickets exported from Kanbanery in CSV_FILE to the Trello board
identified by TRELLO_BOARD_ID. Kanbanery exports a CSV with the following
schema: Title,Owner email,Task type,Estimate,Priority,Description,Column
name,Creator email,Created at,Subtasks,Comments Some records exported by
Kanbanery could be malformed. When encountered they are printed to stdout so
that you can manually add them. Also, any failed request to the Trello API is
printed to stdout. Subtasks and comments are strings separated by semicolon.
Unfortunately, there's no way to distinguish between a separarator and a
semicolon in a comment / subtask. Thus, this script does not attempt to split
comments / subtasks. In other words, comments and subtasks are always going to
be one string each. This scripts ignores Owner email, Estimate, Priority,
Creator email and Created at.
Available options:
--in CSV_FILE Path to csv file with exported tickets from
Kanbanery (default: "./kanbanery.csv")
TRELLO_BOARD_ID Trello board id. To get this visit
https://trello.com/b/ID/reports.json where ID is the
one you see in the URL of your board. For example, in
the following URL the ID is 'lPbIpQIl'
https://trello.com/b/lPbIpQIl/habitatmap
TRELLO_API_KEY Trello api key
TRELLO_API_TOKEN Trello api token
-h,--help Show this help text
Here's code:
#!/usr/bin/env stack
{- stack
script
--resolver nightly-2019-12-21
--package wreq
--package optparse-applicative
--package aeson
--package bytestring
--package lens
--package filepath
--package time
--package cassava
--package text
--package split
--package lens-aeson
--package containers
--package mtl
--package unbounded-delays
--package transformers
--package optparse-applicative
--package http-client
-}
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE FlexibleContexts #-}
import Network.Wreq
import Options.Applicative
import Data.Semigroup ((<>))
import Data.Aeson hiding ((.:))
import GHC.Generics
import Data.ByteString
import Data.ByteString.Char8
import Control.Lens hiding ((.=))
import System.FilePath.Posix
import Data.Foldable
import Data.Time
import Data.Time.Format.ISO8601
import qualified Data.ByteString.Lazy as BL
import Data.Csv
import Data.Text
import Data.List.Split
import Data.List
import Data.Aeson.Lens
import Data.Maybe
import Data.Map.Strict
import Control.Monad.State.Strict
import Data.Time.Clock.POSIX
import Control.Concurrent.Thread.Delay
import Control.Monad.Trans.Reader
import Control.Exception
import Network.HTTP.Client
import Data.Text.Lens
data KanbaneryTicket =
KanbaneryTicket
{ _kanbaneryTicketTitle :: String
, _kanbaneryTicketOwnerEmail :: String
, _kanbaneryTicketTaskType :: String
, _kanbaneryTicketEstimate :: Maybe Float
, _kanbaneryTicketPriority :: Maybe Float
, _kanbaneryTicketDescription :: String
, _kanbaneryTicketColumnName :: String
, _kanbaneryTicketCreatorEmail :: String
, _kanbaneryTicketCreatedAt :: NominalDiffTime
, _kanbaneryTicketSubtasks :: String
, _kanbaneryTicketComments :: String
} deriving (Show,Eq,Ord)
makeFields ''KanbaneryTicket
-- ^ Generate accessors for each field of the record.
-- That means we can later do stuff like
-- `kanbaneryTicket ^. title` to "view" the title
-- without having to worry about duplicate record fields.
data KanbaneryTicketParseResult
= Ok KanbaneryTicket
| Malformed String
deriving (Show)
makePrisms ''KanbaneryTicketParseResult
-- ^ Generate a Prism for each constructor of a data type.
-- See later uses of `_Ok` or `_Malformed`.
instance FromNamedRecord KanbaneryTicketParseResult where
-- ^ Tell cassava how to convert a CSV record to KanbaneryTicketParseResult.
parseNamedRecord x
| Data.Foldable.length x == 11 = Ok <$> parsed
-- ^ When the CSV record has all the fields then..
-- ^ ..it's wellformed..
| otherwise = pure . Malformed $ show x
-- ^ ..otherwise..
-- ^ ..it's malformed.
where
parsed =
KanbaneryTicket <$>
(x .: "Title") <*>
(x .: "Owner email") <*>
(x .: "Task type") <*>
(x .: "Estimate") <*>
(x .: "Priority") <*>
(x .: "Description") <*>
(x .: "Column name") <*>
(x .: "Creator email") <*>
fmap (utcTimeToPOSIXSeconds . zonedTimeToUTC) (x .: "Created at") <*>
(x .: "Subtasks") <*>
(x .: "Comments")
instance FromField ZonedTime where
parseField s = do
mzt <- iso8601ParseM <$> parseField s
Data.Maybe.maybe mempty pure mzt
data Opts =
Opts
{ _optsCsvFile :: String
, _optsTrelloBoardId :: String
, _optsTrelloApiKey :: String
, _optsTrelloToken :: String
} deriving (Show)
makeFields ''Opts
data Env
= Env
{ _envTrelloBoardId :: String
, _envTrelloApiKey :: String
, _envTrelloToken :: String
} deriving (Show)
makeFields ''Env
type App a = ReaderT Env IO a
-- ^ Our application runs in an environment where it can..
-- ^ ..read `Env` (i.e. board id, api key and Trello token)..
-- ^ ..perform IO..
-- ^ ..return a result of type a.
main :: IO ()
main = do
o <- execParser opts
-- ^ Get command line input. More on this in the previous posts.
let env = Env (o ^. trelloBoardId) (o ^. trelloApiKey) (o ^. trelloToken)
-- ^ Wrap command line input into `Env`.
Control.Monad.Trans.Reader.runReaderT (run $ o ^. csvFile) env
-- ^ Apply `env` to `run`.
where
opts = info (parser <**> helper)
( fullDesc
<> progDesc "Moves all tickets exported from Kanbanery in CSV_FILE to the Trello board identified by TRELLO_BOARD_ID. \
\ \
\Kanbanery exports a CSV with the following schema: \
\Title,Owner email,Task type,Estimate,Priority,Description,Column name,Creator email,Created at,Subtasks,Comments \
\ \
\Some records exported by Kanbanery could be malformed. When encountered they are printed to stdout \
\so that you can manually add them. \
\ \
\Also, any failed request to the Trello API is printed to stdout. \
\ \
\Subtasks and comments are strings separated by semicolon. \
\Unfortunately, there's no way to distinguish between a separarator and a semicolon in a comment / subtask. \
\Thus, this script does not attempt to split comments / subtasks. In other words, comments and subtasks \
\are always going to be one string each. \
\ \
\This scripts ignores Owner email, Estimate, Priority, Creator email and Created at."
)
parser :: Options.Applicative.Parser Opts
parser = Opts
<$> strOption
( long "in"
<> metavar "CSV_FILE"
<> help "Path to csv file with exported tickets from Kanbanery"
<> value "./kanbanery.csv"
<> showDefault
)
<*> Options.Applicative.argument str
( metavar "TRELLO_BOARD_ID"
<> help "Trello board id. To get this visit https://trello.com/b/ID/reports.json \
\where ID is the one you see in the URL of your board. For example, in the \
\following URL the ID is 'lPbIpQIl' https://trello.com/b/lPbIpQIl/habitatmap"
)
<*> Options.Applicative.argument str
( metavar "TRELLO_API_KEY"
<> help "Trello api key"
)
<*> Options.Applicative.argument str
( metavar "TRELLO_API_TOKEN"
<> help "Trello api token"
)
run :: String -> App ()
run csvFile = do
csv <- liftIO $ Data.ByteString.Lazy.readFile csvFile
-- ^ Read the CSV file.
-- ^ We need to liftIO because we are in the `App` monad transformer with IO as the base.
case decodeByName csv of
Left err -> liftIO . error . show $ err
-- ^ If the content of the file cannot be decoded by cassava then exit.
Right (_,records) -> do
let malformed = records ^.. folded . _Malformed
-- ^ Take all the `Malformed` records and..
liftIO $ Prelude.putStrLn $ "Found " <> (show . Data.Foldable.length $ malformed) <> " malformed records."
traverse_ (liftIO . Prelude.putStrLn . (<>) "MALFORMED: ") malformed
-- ^ ..print some info about them so that they can be manually transferred to Trello.
let kanbaneryTickets = records ^.. folded . _Ok
-- ^ Take all the wellformed (i.e. `Ok`) records and..
liftIO $ Prelude.putStrLn $ "Creating " <> (show . Data.Foldable.length $ kanbaneryTickets) <> " tickets."
liftIO $ Prelude.putStrLn "Creating lists."
columnNamesByListIds <- createLists kanbaneryTickets
-- ^ ..create in Trello the same columns that are on Kanbanery..
liftIO $ Prelude.putStrLn "" >> Prelude.putStrLn "Creating cards."
kanbaneryTicketsByCardId <- createCards columnNamesByListIds kanbaneryTickets
-- ^ ..create in Trello the tickets that are on Kanbanery..
liftIO $ Prelude.putStrLn "" >> Prelude.putStrLn "Creating labels."
labelsByIds <- createLabels kanbaneryTickets
-- ^ ..create in Trello the labels that are on Kanbanery..
liftIO $ Prelude.putStrLn "" >> Prelude.putStrLn "Adding labels to tickets."
traverse_ (addLabelsReq labelsByIds kanbaneryTicketsByCardId) kanbaneryTickets
-- ^ ..associate labels to tickets in Trello..
liftIO $ Prelude.putStrLn "" >> Prelude.putStrLn "Adding subtasks to tickets."
traverse_ (addSubtasksReq kanbaneryTicketsByCardId) kanbaneryTickets
-- ^ ..add subtasks that were in Kanbanery tickets to Trello tickets..
liftIO $ Prelude.putStrLn "" >> Prelude.putStrLn "Adding creator email and creation date to tickets."
traverse_ (addCreationReq kanbaneryTicketsByCardId) kanbaneryTickets
-- ^ ..for each ticket add the Kanbanery creation date as a comment..
liftIO $ Prelude.putStrLn "" >> Prelude.putStrLn "Adding comments to tickets."
traverse_ (addCommentsReq kanbaneryTicketsByCardId) kanbaneryTickets
-- ^ ..add comments that were in Kanbanery tickets to Trello tickets.
liftIO $ Prelude.putStrLn "" >> Prelude.putStrLn "Done!"
-- THE REST OF THE SCRIPT WILL BE HERE NEXT WEEK ;)
Get the latest content via email from me personally. Reply with your thoughts. Let's learn from each other. Subscribe to my PinkLetter!
Top comments (0)