DEV Community

loading...

Easily visualise entity relationships from Wikipedia with Wiki Topic Grapher! ๐Ÿ‘พ

Charly Wargnier
Thrilled to make Data Science & SEO cross paths. Python ๐Ÿ & PowerBI lover. ML dabbler. Obsessed w. automation.
Originally published at charlywargnier.com ใƒป2 min read

Leverage the power of Google Cloud Natural Language to retrieve entity relationships from Wikipedia URLs or topics!

  • Get interactive graphs of connected entities
  • Export results with entity types and salience to CSV!
  • Export results with entity types and entity salience to CSV!

๐Ÿ’ป Try the app: https://lnkd.in/dkfuhed
๐Ÿงต Twitter thread: https://lnkd.in/dji2a4s

Use cases

Many cool use cases!

  • Research a topic then get entity associations that exist from that seed topic
  • Map out related entities with your product, service or brand
  • Find how well you've covered a specific topic on your website Differentiate your pages!

This is not just an SEO tool! You can map any types of relationship from a given Wikipedia seed URL or topic. E.g someone from the community already mapped out relationships in structural molecular biology! ๐Ÿค“๐Ÿง 

Stack

About the stack, it's 100% Python! ๐Ÿ๐Ÿ”ฅ

  • @GCPcloud Natural Language API
  • PyWikibot
  • Networkx PyVis
  • @streamlit
  • Streamlit Components -> streamlit.io/components

โš’๏ธ Still To-Doโ€™s

๐Ÿ’ฐ Add a budget estimator to get a sense of @GCPcloud costs!
๐ŸŒAdd a multilingual option (currently English only)
๐Ÿ“ˆAdd on-the-fly physics controls to the network graph
๐Ÿ’ฏAdd Google KG #API to add more data (scores, etc.) (ht @loukilaymen )

That code currently lays in a private repo. I should be able to make it public soon for you to re-use it in your own apps and creations! I just need to clean it a tad, remove some sensitive bits, etc.

๐Ÿ™Œ Shout-outs

Kudos to @jroakes for the original script. Buy that man a ๐Ÿบ for his sterling contributions! ->
paypal.com/paypalme/codesโ€ฆ

Kudos also to fellow @streamlit Creators:

@napoles3d who told me about the PyVis lib! ๐Ÿ”ฅ
@andfanilo /@christianklose3 for their precious advice! ๐Ÿ™

๐Ÿ’ฒ Beware on costs!

It can get expensive quickly with that Google Natural Language API!

Monitor your costs via the GCP console regularly and/or put quotas to tame that G beast! I'm planning to add a budget estimator pre-API calls. Should come handy.

Feedback and support

Wiki Topic Grapher's still in Beta, with possible rough edges! Head-off to my Gitter page for bug reports, questions, or suggestions.

This app is free. If it's useful to you, you can buy me a โ˜• to support my work! ๐Ÿ™

โ–ถ๏ธ buymeacoffee.com/cwar05

That's all, folks. Enjoy!

Charly

Discussion (0)