DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Cover image for Build a graph database with Amazon Neptune Part 3 - Query and Visualize Graphs
Wendy Wong for AWS Community Builders

Posted on • Updated on

Build a graph database with Amazon Neptune Part 3 - Query and Visualize Graphs

Introduction

From previous blog Amazon CloudFormation was used in a production environment to create a Neptune cluster. The second blog, provided a brief overview of graph data modelling which will be explored here.

Learning Outcomes

  • You will learn how to use Graph notebooks to query and visualize your data

High-level architecture

The following solution overview was created by me to explore the ETL process to transform data and the end-to-end process of creating a well-architected data analytics framework to query and visualize data.

solution

Use Case - Social Network Analysis

Popular use cases for network analysis includes fraud detection, customer journeys and 'friend of friends'.

In this blog we explore social network analysis using AWS Community Builder's Directory for builders located in Asia Pacific and Japan (APJ).

Amazon Neptune Pricing - Free Tier

If you have never used Amazon Neptune before, your organization may be able to access 750 hours of usage for the first 30 days with EC2 instances db.t3.medium or db.t4g.medium.

Select your region and you may manually provision the suggested instances.

Note: If you use Amazon CloudFormation template to help you provision many AWS resources, the default EC2 instance in my case was r5.2xlarge.

You may read information on pricing here

Graph Notebooks

You may access the AWS Graph Notebooks in jupyter notebooks attached to Sagemaker from the Neptune DB instance that you may incorporate into your own project.

Step 1: Navigate to the Amazon Neptune console and then on the left hand-side click Notebooks to launch a notebook that is attached to a Neptune db instance.

notebook

Step 2: Select the radio-dial on Neptune DB instance and click Open Notebook

open

Step 3: You will see a folder called 'Neptune' under the Jupyter notebook homepage, double-click this folder.

jupyter

Step 4: Select '01-Getting-Started'

get start

Step 5: Click on the second option Using-Gremlin_to_Access_the_Graph.ipynb

aws notebooks

Data Modelling - Amazon Neptune

We will explore Gremlin statements in the Neptune data model in greater detail with four components:

  • subject (S) - relationship of two vertices (ie.nodes)
  • predicate (P) - edge label
  • object (O) - target vertex (i.e. node)
  • graph (G)

Creating Gremlin Vertex Label Statements

vertex statement

Creating Gremlin Edge Statements

edge

Neptune Workbench

You may explore how to query and visualize graphs with sample data and graph notebooks from the Amazon database blog here

You can verify the version of the graph notebook and also the health of the Neptune cluster:

versionn

You may write a single statement including vertices and edge statements to create a small social network graph with gremlin:

single statement

edge code

Data Visualization

I created an interactive network graph using Flourish to highlight interesting insights from the AWS Community Builders located in Asia Pacific and Japan, just hover your mouse over the data points.

Link: https://public.flourish.studio/visualisation/11069877/

flourish

The dataset created from Amazon Athena was set up in Flourish with Source and Target

flourish data

Most of our AWS Community Builders are involved in Containers as at 12 August 2022.

Bar

Clean Up Resources - Delete Neptune DB Instance

Best practice is to delete the Amazon Neptune database instance if is no longer required to avoid unexpected or 'surprise' charges at the end of the month. You may follow the steps here:

  1. Step 1: Navigate to the Amazon Neptune console, click on Databases on the left-hand menu.

You may follow the Amazon documentation for instructions here

You may take a final snaphot before you delete the Neptune DB cluster.

Select the radio-dial and select your Neptune DB instance and click delete

delete Neptune

Clean Up Resources - Detach an EBS volume before you delete an EC2 instance

Since I used PuTTy to SSH into a Windows EC2 instance, I have to detach the EBS Volume before I delete the Windows EC2 instance

Amazon documentation for instructions are provided here

Clean Up Resources - Delete EC2 Instance

Delete your EC2 instances from the Amazon EC2 console after you have completed your project.

delete neptune

References

Additional Resources

Register for Australia's biggest data engineering conference DataEngBytes:

  • Melbourne: 27 September 2022
  • Sydney: 29 September 2022

Register here: https://dataengconf.com.au/

Data Eng Bytes

AWS Innovate Online Conference APJ - 23 & 25 August 2022

AWS Innovate

Register at this link

πŸš€ Hot off the press: Introducing Amazon Neptune Global Database

From 27 July 2022, AWS Neptune Global Database can build graph relationships in multiple regions including US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Ireland), Europe (London), and Asia Pacific (Tokyo) Regions.

You can read about this announcement from the AWS Database Blog written by author Navtanay Sinha here.

Until the next lesson, happy learning ! 😁

Top comments (2)

Collapse
 
naubit profile image
Al - Naubit

That was a nice read! Liked, bookmarked and followed, keep the good work!

Collapse
 
abc_wendsss profile image
Wendy Wong

Thanks Al for your feedback, hope you learnt something new :)

Build Anything...


Use any Linode offering to create something for the DEV x Linode Hackathon 2022. A variety of prizes are up for grabs, inculding $1,000 USD. πŸ‘€

β†’ Join the Hackathon <-