This blog is a translation of its Arabic version on my website.
As a Facebook user when I see any tech tweet on twitter I share it with my FB community. so I thought having a twitter account that takes a screenshot from a tweet can be a nice idea.
Stack
Go-lang: you can use any lang as chromedp supporting several languages but I prefer Go to get a strong knowledge.
Go-lang Twitter API: to receive tweets through
StreamAPIand post tweets throughTwitter APIChromedp: Chrome Debugging Protocol. it allows you to use the chrome browser and interact with it to simulate real user behavior with code. also it supports headless mode -no GUI- so it can run on CI/CD pipelines to do e2e tests or any automation you need like scrapping websites and other cool stuff.
Redis: an InMemory database we will use to store processed tweets so we don't respond to the same tweet more than once
Source Code
https://github.com/ahmedash95/shareaspic
Scenario would be like:
- create a twitter account @shareAsPic
- people can mention him in a reply to any tweet with a message like @shareAsPic share this please
- use twitter stream API to listen for all these tweets
- use Chromedp to open that tweet in the browser and take a screenshot in - a -headless- mode reply to the mention with the screenshot
Application flow
1 - Users always post tweets.
2 - These tweets land on Twitter's servers which it has a lot of systems. each system has it's own mission with that tweet (notifications, timeline, streaming API, etc..)
3 - We will use the StreamAPI to listen only to replies contain @ShareAsPic our twitter app name to process it
the below example shows how to prepare the stream params to listen only for our app's mentions
params := &twitter.StreamFilterParams{
Track: []string{"@ShareAsPic"},
StallWarnings: twitter.Bool(true),
}
4 - At this step our app starts to receive tweets data twitter sending to us through StreamAPI
// Full struct https://github.com/dghubble/go-twitter/blob/master/twitter/statuses.go#L13
type Tweet struct {
CreatedAt string `json:"created_at"`
CurrentUserRetweet *TweetIdentifier `json:"current_user_retweet"`
ID int64 `json:"id"`
IDStr string `json:"id_str"`
InReplyToScreenName string `json:"in_reply_to_screen_name"`
InReplyToStatusID int64 `json:"in_reply_to_status_id"`
InReplyToStatusIDStr string `json:"in_reply_to_status_id_str"`
InReplyToUserID int64 `json:"in_reply_to_user_id"`
InReplyToUserIDStr string `json:"in_reply_to_user_id_str"`
Text string `json:"text"`
User *User `json:"user"`
}
from the above Struct we send InReplyToScreenName and InReplyToStatusID to chromedp to open tweet URL on twitter and take the screenshot
5 - here we realize the power of headless browsers as chromedp simulates real user behavior on a real browser and makes interactions.
for us, it opens https://twitter.com/{UserName}/status/{TweetID} and using one of the chromedp functions takes a screenshot of that tweet using element selector that contains tweet body
func TweetScreenShot(username string, tweetId string) (string, error) {
chromedpContext, cancelCtxt := chromedp.NewContext(context.Background()) // create new tab
defer cancelCtxt()
// capture screenShot of an element
fname := fmt.Sprintf("%s-%s.png", username, tweetId)
url := fmt.Sprintf("https://twitter.com/%s/status/%s", username, tweetId)
var buf []byte
if err := chromedp.Run(chromedpContext, elementScreenshot(url, `document.querySelector("#permalink-overlay-dialog > div.PermalinkOverlay-content > div > div > div.permalink.light-inline-actions.stream-uncapped.original-permalink-page > div.permalink-inner.permalink-tweet-container > div")`, &buf)); err != nil {
return "", err
}
fmt.Printf("write pic to path %s\n", fmt.Sprintf("%s/%s", PIC_STORAGE_PATH, fname))
if err := ioutil.WriteFile(fmt.Sprintf("%s/%s", PIC_STORAGE_PATH, fname), buf, 0755); err != nil {
return "", err
}
return fname, nil
}
// elementScreenshot takes a screenshot of a specific element.
func elementScreenshot(urlstr, sel string, res *[]byte) chromedp.Tasks {
return chromedp.Tasks{
chromedp.Navigate(urlstr),
chromedp.WaitVisible(sel, chromedp.ByJSPath),
chromedp.Sleep(time.Second * 3),
chromedp.Screenshot(sel, res, chromedp.NodeVisible, chromedp.ByJSPath),
}
}
6 - then we reply to the user with the screenshot attached
filename, err := TweetScreenShot(tweet.InReplyToScreenName, tweet.InReplyToStatusIDStr)
if err != nil {
logAndPrint(fmt.Sprintf("Faild to take a screenshot of the tweet, %s", err.Error()))
return
}
filePath := fmt.Sprintf("%s%s", PIC_STORAGE_PATH, filename)
logAndPrint("upload photo")
mediaId, _ := TwitterUploadClient.Upload(filePath)
logAndPrint(fmt.Sprintf("photo has been uploaded: %d", mediaId))
statusUpdate := &twitter.StatusUpdateParams{
Status: "",
InReplyToStatusID: tweet.ID,
PossiblySensitive: nil,
Lat: nil,
Long: nil,
PlaceID: "",
DisplayCoordinates: nil,
TrimUser: nil,
MediaIds: []int64{mediaId},
TweetMode: "",
}
_, _, err2 := client.Statuses.Update(fmt.Sprintf("Hello @%s , Here u are", tweet.User.ScreenName), statusUpdate)
if err2 != nil {
logAndPrint(fmt.Sprintf("Faild to reply pic tweet, %s", err2.Error()))
}
logAndPrint(fmt.Sprintf("replied to: %s\n", tweet.IDStr))
and the result of the whole flow is like
Conclusions
The app itself has no goal but it was just a simple idea to implement and see how can we automate browser interactions with Chromedp. it adds huge value for E2E tests especially for SPA. or scrapping websites content.
If you have any comments or ideas to share about chromedp. I'm more than interested to discuss and learn.
Top comments (0)