Fetching Open Graph Data from URLs to show a thumbnail preview
This article was originally posted on the Web Highlights blog. You can find it here.
Recently, I improved the design of my Web Highlights app's dashboard by showing a link preview for each highlighted page. Here is what the new design looks like:
In this article, I want to share how to create such a link preview component and make web pages appear with a thumbnail.
If you follow my articles, you will probably know that I am a big fan of Web Components. So, of course, I built this component with Web Components as well.
For several reasons:
I can use the component in the Vue.js web app as well as in the Web Component-based Chrome Extension.
The architecture is better encapsulated
Anyone can reuse the component
Suppose you are not yet convinced. Embedding the application is as easy as including these few lines of code in your web application:
You can find a demo here. Here is also a CodePen showing how easy it is to use the link preview anywhere.
For the component's design, I followed a plain one as Medium does. The preview card contains a title, description, link, and preview image. Here is what the thumbnail preview component looks like:
Furthermore, the component shows some loading indicators when fetching the open graph data from our API, which we will come to later.
I will not go into detail about the CSS implementation in this article. But you can find the Github repository here. Feel free to check it out.
Building the Web Component
To build this web component I created the custom element webhighlights-link-preview. To make development easier, I am using the Lit library from Google.
We create our custom element by using Lit's customElement decorator and provide some reactive properties to make our element customizable by the client:
A client must provide a url for which we will fetch the open graph data from our API. Furthermore, one can provide an alternative apiUrl URL if you want to use a different API.
Not every website provides open graph meta data tags so that a client can provide some fallback data in case we can not find a title, description, or image for the given URL.
Furthermore, we need a reactive property that stores the fetched metadata for the corresponding URL:
The OpenGraphMetaData interface defines the object we expect to get back from the API. It looks like this:
So, if you wanted to create your own API, you would need to make your server response adapt to this OpenGraphMetaData interface and provide the URL of the endpoint within the components apiUrl property.
Furthermore, we need to know whether the component should show the text-loading indicators. In this case, we want to show it as long as our metaData property is undefined. To check this in the template, we create a simple getter:
Now, in the template, we can conditionally add some CSS classes to make our component show a loading state depending on whether the metaData is still being fetched from the server or has already been loaded:
Fetching Open Graph Data
To get all the needed data for our preview, we must somehow fetch the Open Graph metadata tags from the corresponding URLs. If you are not familiar with the Open Graph protocol, you should check out this article first:
There is the possibility to get the necessary data using APIs like OpenGraph.io. Using APIs like this, we wouldn't need to create a server to get the data. Unfortunately, most of those APIs have a limited amount of requests. OpenGraph.io, e.g., limits it to 100 requests.
As I don't want to pay anything to get Open Graph Data, I created a simple Netlify function that fetches Open Graph data using the open-source openGraphScraper library. This library is a simple node module for scraping Open Graph for any website.
In production, I am using the normal server environment for my Web Highlights application as I also wanted to provide some server-side caching to decrease the loading time. But, for this example, a simple Netlify function should be fine.
We can get the Open Graph data for any URL by importing the ogs object from the library:
And then fetch the data like this:
Afterward, we adapt the received SuccessResult interface to our expected OpenGraphMetaData interface by using an adapter function:
Here is what the Netlify handler looks like:
Now, in our frontend, we can get the needed metadata by fetching it from our endpoint:
Final Thoughts
Thanks for reading this article. I hope you could follow along to build your own link-preview component or reuse the one we created here. Notice that I can not guarantee that the provided Netlify API will be available forever. So, if you are planning to use this component in production, make sure to provide your own endpoint.
I am always happy to answer questions and am open to criticism. Feel free to contact me at any time! Get in touch with me via LinkedIn, follow me on Twitter, Medium, or subscribe to get my stories via email.
About the Author
I am a Software Engineering Analyst at Accenture Song. What drives me the most is my urge to create something that has the potential to be helpful and life-changing for others.
For example, are you tired of browsing your history to find the information you saw a few days ago? My Web Highlights Chrome Extension got you covered and will boost your productivity by organising your research in a structured and efficient way. Just like you do on books and articles, highlight text on any web page or PDF. Your highlights sync directly to the web app on web-highlights.com, where you can find them from anywhere.
Top comments (0)