From previous blog Amazon CloudFormation was used in a production environment to create a Neptune cluster. The second blog, provided a brief overview of graph data modelling which will be explored here.
- You will learn how to use Graph notebooks to query and visualize your data
The following solution overview was created by me to explore the ETL process to transform data and the end-to-end process of creating a well-architected data analytics framework to query and visualize data.
Popular use cases for network analysis includes fraud detection, customer journeys and 'friend of friends'.
In this blog we explore social network analysis using AWS Community Builder's Directory for builders located in Asia Pacific and Japan (APJ).
If you have never used Amazon Neptune before, your organization may be able to access 750 hours of usage for the first 30 days with EC2 instances db.t3.medium or db.t4g.medium.
Select your region and you may manually provision the suggested instances.
Note: If you use Amazon CloudFormation template to help you provision many AWS resources, the default EC2 instance in my case was r5.2xlarge.
You may read information on pricing here
You may access the AWS Graph Notebooks in jupyter notebooks attached to Sagemaker from the Neptune DB instance that you may incorporate into your own project.
Step 1: Navigate to the Amazon Neptune console and then on the left hand-side click Notebooks to launch a notebook that is attached to a Neptune db instance.
Step 2: Select the radio-dial on Neptune DB instance and click Open Notebook
Step 3: You will see a folder called 'Neptune' under the Jupyter notebook homepage, double-click this folder.
Step 4: Select '01-Getting-Started'
Step 5: Click on the second option Using-Gremlin_to_Access_the_Graph.ipynb
We will explore Gremlin statements in the Neptune data model in greater detail with four components:
- subject (S) - relationship of two vertices (ie.nodes)
- predicate (P) - edge label
- object (O) - target vertex (i.e. node)
- graph (G)
Creating Gremlin Vertex Label Statements
Creating Gremlin Edge Statements
You may explore how to query and visualize graphs with sample data and graph notebooks from the Amazon database blog here
You can verify the version of the graph notebook and also the health of the Neptune cluster:
You may write a single statement including vertices and edge statements to create a small social network graph with gremlin:
I created an interactive network graph using Flourish to highlight interesting insights from the AWS Community Builders located in Asia Pacific and Japan, just hover your mouse over the data points.
The dataset created from Amazon Athena was set up in Flourish with Source and Target
Most of our AWS Community Builders are involved in Containers as at 12 August 2022.
Best practice is to delete the Amazon Neptune database instance if is no longer required to avoid unexpected or 'surprise' charges at the end of the month. You may follow the steps here:
- Step 1: Navigate to the Amazon Neptune console, click on Databases on the left-hand menu.
You may follow the Amazon documentation for instructions here
You may take a final snaphot before you delete the Neptune DB cluster.
Select the radio-dial and select your Neptune DB instance and click delete
Since I used PuTTy to SSH into a Windows EC2 instance, I have to detach the EBS Volume before I delete the Windows EC2 instance
Amazon documentation for instructions are provided here
Delete your EC2 instances from the Amazon EC2 console after you have completed your project.
- Melbourne: 27 September 2022
- Sydney: 29 September 2022
Register here: https://dataengconf.com.au/
Register at this link
From 27 July 2022, AWS Neptune Global Database can build graph relationships in multiple regions including US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Ireland), Europe (London), and Asia Pacific (Tokyo) Regions.
You can read about this announcement from the AWS Database Blog written by author Navtanay Sinha here.
Until the next lesson, happy learning ! 😁