Polywork Poster in Go

#go

A couple of days ago I finally got access to Polywork. Which, for those who don't know, is like a new-aged Linkedin. It allows you to create a timeline of your work and accomplishments. The idea is that people will be able to contact you through the site to offer you work or opportunities.Take a look at meta.shindakun.dev for a simple example timeline.

One of the things that struck me was that there is no public API at the moment. Which bummed me out as I didn't want to start entering blog posts by hand. So, I decided to poke around a bit and see what the flow is for creating a new post. With the objected being crafting some Go code to create posts.

This would dovetail nicely into our data connector series.

Getting Started

First, we need to look to see how the site lets us put up a new post. This can be done via the network tab in Chromes Dev Tools.

We can see that clicking on the "Post" button brings up a form that we can use to enter our details. Looking at the details of the edit network call shows the entire Turbo Frame that makes up. new in this case seems to just redirect to edit.

Perfect! This will allow us to grab the details we need from the form. We need both the update ID and the authenticity token. Though at present it seems the authenticity token isn't actually required when posting a new update. 🤷‍♂️ I won't show the complete posting flow via the website since it includes cookies but you get the idea.

Our Plan

So here is what we are going to do:

make a request to https://www.polywork.com/shindakun/highlights/new/
extract the update ID and authenticity token
create a multipart form with information we want to post
POST to https://www.polywork.com/shindakun/highlights/{{update_id}}

Enjoy this post?
How about buying me a coffee?

Getting Polywork to Go

As always you can find the full code listing at the bottom of this post. Also, this code is pretty sloppy and a bit brittle since it is just a proof of concept.

package main

import (
    "bytes"
    "fmt"
    "io"
    "log"
    "mime/multipart"
    "net/http"
    "regexp"
    "strings"

    "github.com/andybalholm/cascadia"
    "golang.org/x/net/html"
)

const cookie = "_polywork_session=...snip..."
const username = "shindakun"

We start up by setting our cookie value. This is necessary to be able to POST to the website. I simply grabbed this value from the network tab in Chrome. We could start with a "magic" link login but this seemed easier. We'll also set our username.

func main() {
    req, err := http.NewRequest(http.MethodGet, "https://www.polywork.com/"+username+"/highlights/new/", nil)
    req.Header.Add("Cookie", cookie)
    req.Header.Add("Turbo-Frame", "navbar-post-highlight")
    if err != nil {
        log.Fatal(err)
    }

    res, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    defer res.Body.Close()

If you've worked with Go before this probably looks very familiar. What we are doing is using http.NewRequest to request to /{{username}}/highlights/new. Note how we are setting the cookie directly in the header and setting a header for the Turbo-Frame.

Calling http.DefaultClient.Do(req) makes the request to the remote server and if successful will return that data in res.

    doc, err := html.Parse(res.Body)
    if err != nil {
        log.Fatal(err)
    }

    // inputs := cascadia.MustCompile("input").MatchAll(doc)

    buttons := cascadia.MustCompile("button").MatchAll(doc)

    re := regexp.MustCompile(`\/shindakun\/highlights\/(.*)\/save_draft`)
    r := re.FindStringSubmatch(buttons[1].Attr[5].Val)

Here we're going to get a bit tricky. The first thing we want to do is use the parse our incoming form as HTML. This then allows us to use the cascadia library and CSS selectors to grab all the buttons. We don't really need all of them but for this first version what the heck.

Using the regular expression \/shindakun\/highlights\/(.*)\/save_draft we extract the ID for the post we are working with.

This is the main section of the code that is very brittle since I'm not doing anything to validate we are looking at the right button key and instead just hard coding the values that I saw during testing.

    // _method: patch
    // authenticity_token: ...snip...
    // draft_highlight[end_date]: 01-01-2021
    // draft_highlight[content]: <div>Testing</div>
    // draft_highlight[activity_ids][]: 76199
    // commit: Post

During testing, I made a copy of the form data POSTed from Chrome for reference. We'll use this as our base form data. As you'll see below not every field is required. I think I can leave out the _method field but have not tried as of yet. ~~I mentioned before but you should also note that I'm not using the authenticity_token token as it seems it is not required.~~ Update: the latest site update now properly requires the token.

    body := &bytes.Buffer{}
    writer := multipart.NewWriter(body)

    fw, _ := writer.CreateFormField("_method")
    _, err = io.Copy(fw, strings.NewReader("patch"))
    if err != nil {
        log.Fatal(err)
    }

    // fw, _ = writer.CreateFormField("authenticity_token")
    // _, err = io.Copy(fw, strings.NewReader(inputs[1].Attr[2].Val))
    // if err != nil {
    // log.Fatal(err)
    // }

    fw, _ = writer.CreateFormField("draft_highlight[content]")
    _, err = io.Copy(fw, strings.NewReader("<div>Posting via Golang</div> :) <a href=\"https://shindakun.dev\">shindakun</a>"))
    if err != nil {
        log.Fatal(err)
    }

    fw, _ = writer.CreateFormField("commit")
    _, err = io.Copy(fw, strings.NewReader("Post"))
    if err != nil {
        log.Fatal(err)
    }

    writer.Close()

It's a bit awkward to create a form in Go but hey, it works. We make a form field with writer.CreateFormField and then io.Copy our data into that field. Right now our payload is hardcoded. We could easily adapt my data connector post to automatically add my Ghost blog posts to Polywork.

    post, err := http.NewRequest("POST", "https://www.polywork.com/shindakun/highlights/"+r[1], bytes.NewReader(body.Bytes()))
    if err != nil {
        log.Fatal(err)
    }
    post.Header.Set("Content-Type", writer.FormDataContentType())
    post.Header.Add("Cookie", cookie)
    rsp, _ := http.DefaultClient.Do(post)
    if rsp.StatusCode != http.StatusOK {
        log.Printf("Request failed with response code: %d", rsp.StatusCode)
    }
}

And now we do the actual form submit. And that's all there is to it.

What's Next?

Next, I think I am going to adapt this code to use with the data connector we are working on. I'll probably try to make it pretty generic so I can use it as its own module.

Code Listing

package main

import (
    "bytes"
    "fmt"
    "io"
    "log"
    "mime/multipart"
    "net/http"
    "regexp"
    "strings"

    "github.com/andybalholm/cascadia"
    "golang.org/x/net/html"
)

const cookie = "_polywork_session=...snip..."
const username = "shindakun"

func main() {
    req, err := http.NewRequest(http.MethodGet, "https://www.polywork.com/"+username+"/highlights/new/", nil)
    req.Header.Add("Cookie", cookie)
    req.Header.Add("Turbo-Frame", "navbar-post-highlight")
    if err != nil {
        log.Fatal(err)
    }

    res, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    defer res.Body.Close()

    doc, err := html.Parse(res.Body)
    if err != nil {
        log.Fatal(err)
    }

    // inputs := cascadia.MustCompile("input").MatchAll(doc)

    buttons := cascadia.MustCompile("button").MatchAll(doc)

    re := regexp.MustCompile(`\/shindakun\/highlights\/(.*)\/save_draft`)
    r := re.FindStringSubmatch(buttons[1].Attr[5].Val)

    // _method: patch
    // authenticity_token: ...snip...
    // draft_highlight[end_date]: 01-01-2021
    // draft_highlight[content]: <div>Testing</div>
    // draft_highlight[activity_ids][]: 76199
    // commit: Post

    body := &bytes.Buffer{}
    writer := multipart.NewWriter(body)
    fw, _ := writer.CreateFormField("_method")
    _, err = io.Copy(fw, strings.NewReader("patch"))
    if err != nil {
        log.Fatal(err)
    }
    // fw, _ = writer.CreateFormField("authenticity_token")
    // _, err = io.Copy(fw, strings.NewReader(inputs[1].Attr[2].Val))
    // if err != nil {
    // log.Fatal(err)
    // }
    fw, _ = writer.CreateFormField("draft_highlight[content]")
    _, err = io.Copy(fw, strings.NewReader("<div>Posting via Golang</div> :) <a href=\"https://shindakun.dev\">shindakun</a>"))
    if err != nil {
        log.Fatal(err)
    }
    fw, _ = writer.CreateFormField("commit")
    _, err = io.Copy(fw, strings.NewReader("Post"))
    if err != nil {
        log.Fatal(err)
    }
    writer.Close()

    post, err := http.NewRequest("POST", "https://www.polywork.com/shindakun/highlights/"+r[1], bytes.NewReader(body.Bytes()))
    if err != nil {
        log.Fatal(err)
    }
    post.Header.Set("Content-Type", writer.FormDataContentType())
    post.Header.Add("Cookie", cookie)
    rsp, _ := http.DefaultClient.Do(post)
    if rsp.StatusCode != http.StatusOK {
        log.Printf("Request failed with response code: %d", rsp.StatusCode)
    }
}