DEV Community

Mohamed Mokhtar
Mohamed Mokhtar

Posted on • Updated on

Under the hood of CREATE on AGE cypher

Hola

Greetings, continuing the series of under the hood of Apache AGE, last article was introductory to create_graph which is the first entry of any graph operation we are going to play with, so what is next is having nodes and something to work with inside that graph so we need to understand what is under the hood of the CREATE
Before going into that I would like to recommend the following articles which are good to understand

Overview

Apache AGE is a layer that is being extended on top of PostgreSQL through the nature of the PostgreSQL's enabled extensions and one of them which are being executed after the parsing stage of PostgreSQL we have a Parse Tree from parser and the Query tree from the Analyzer and Re-writer stage so Apache AGE is being executed at the Query tree level, it takes Query Tree and returns tuned Query Tree,

What is that tuning exactly?

From the side of AGE it starts parsing the query and looking for every function call at the query tree once it finds a function call, it checks if it is a cypher or not, if it is a cypher it starts converting it into SUB-QUERY

Our mission today is going through CREATE statement and getting know what is the equivalent SUB-QUERY it is being converted to during the conversion stage of AGE.

Let's get started

Start by looking into transform a cypher_clause function which is responsible for that


Query *transform_cypher_clause(cypher_parsestate *cpstate,
                               cypher_clause *clause)
Enter fullscreen mode Exit fullscreen mode

It looks into the cypher clauses after being parsed into cypher nodes and transforms each node base on its type,
the type we are looking into is CREATE, so at some point in the function we have that

    else if (is_ag_node(self, cypher_create))
    {
        result = transform_cypher_create(cpstate, clause);
    }
Enter fullscreen mode Exit fullscreen mode

That's our target is learning what is happening inside transform_cypher_create

Diving into that


static Query *transform_cypher_create(cypher_parsestate *cpstate,
                                      cypher_clause *clause)
{
    ParseState *pstate = (ParseState *)cpstate;
    cypher_create *self = (cypher_create *)clause->self;
    cypher_create_target_nodes *target_nodes;
    Const *null_const;
    List *transformed_pattern;
    FuncExpr *func_expr;
    Query *query;
    TargetEntry *tle;

    target_nodes = make_ag_node(cypher_create_target_nodes);
    target_nodes->flags = CYPHER_CLAUSE_FLAG_NONE;
    target_nodes->graph_oid = cpstate->graph_oid;

    query = makeNode(Query);
    query->commandType = CMD_SELECT;
    query->targetList = NIL;

    if (clause->prev)
    {
        handle_prev_clause(cpstate, query, clause->prev, true);

        target_nodes->flags |= CYPHER_CLAUSE_FLAG_PREVIOUS_CLAUSE;
    }

    null_const = makeNullConst(AGTYPEOID, -1, InvalidOid);
    tle = makeTargetEntry((Expr *)null_const, pstate->p_next_resno++,
                          AGE_VARNAME_CREATE_NULL_VALUE, false);
    query->targetList = lappend(query->targetList, tle);

    /*
     * Create the Const Node to hold the pattern. skip the parse node,
     * because we would not be able to control how our pointer to the
     * internal type is copied.
     */
    transformed_pattern = transform_cypher_create_pattern(cpstate, query,
                                                          self->pattern);

    target_nodes->paths = transformed_pattern;
    if (!clause->next)
    {
        target_nodes->flags |= CYPHER_CLAUSE_FLAG_TERMINAL;
    }


    func_expr = make_clause_func_expr(CREATE_CLAUSE_FUNCTION_NAME,
                                      (Node *)target_nodes);

    // Create the target entry
    tle = makeTargetEntry((Expr *)func_expr, pstate->p_next_resno++,
                          AGE_VARNAME_CREATE_CLAUSE, false);
    query->targetList = lappend(query->targetList, tle);

    query->rtable = pstate->p_rtable;
    query->rteperminfos = pstate->p_rteperminfos;
    query->jointree = makeFromExpr(pstate->p_joinlist, NULL);

    return query;
}
Enter fullscreen mode Exit fullscreen mode

Function breakdown

  • Extending the nodes of PostgreSQL using DEFINE_NODE_METHODS through DEFINE_NODE_METHODS(cypher_create_target_nodes) And cypher_create is as the following:
typedef struct cypher_create_target_nodes
{
    ExtensibleNode extensible;
    List *paths;
    uint32 flags;
    uint32 graph_oid;
} cypher_create_target_nodes;
Enter fullscreen mode Exit fullscreen mode

So that we can say that we are having a node of type cypher_create is attached to the PostgreSQL nodes and We can have a Node from that type

  • Assigning the target nodes' graphoid to the ParseState's graphoid and flags to NONE|NULLPTR

  • Make a Query node of type CMD SELECT, used in handling previous clauses

    query = makeNode(Query);
    query->commandType = CMD_SELECT;
    query->targetList = NIL;

    if (clause->prev)
    {
        handle_prev_clause(cpstate, query, clause->prev, true);

        target_nodes->flags |= CYPHER_CLAUSE_FLAG_PREVIOUS_CLAUSE;
    }
Enter fullscreen mode Exit fullscreen mode
  • Create the Const Node to hold the pattern. skip the parse node, because we would not be able to control how our pointer to the internal type is copied. [From code comments]
    transformed_pattern = transform_cypher_create_pattern(cpstate, query, self->pattern);

target_nodes->paths = transformed_pattern;
    if (!clause->next)
    {
        target_nodes->flags |= CYPHER_CLAUSE_FLAG_TERMINAL;
    }
Enter fullscreen mode Exit fullscreen mode
  • Creates the function expression that represents the clause. Adds the extensible node that represents the metadata that the clause needs to handle the clause in the execution phase. [From code comments], That's mean that we convert that to intermediate function expression that represents it in the final query, i.e. the final query is split up into subquries and those subquries are function calls
  func_expr = make_clause_func_expr(CREATE_CLAUSE_FUNCTION_NAME,
                                      (Node *)target_nodes);

    // Create the target entry
    tle = makeTargetEntry((Expr *)func_expr, pstate->p_next_resno++,
                          AGE_VARNAME_CREATE_CLAUSE, false);
    query->targetList = lappend(query->targetList, tle);
Enter fullscreen mode Exit fullscreen mode
  • Finish up the query and return it
    query->rtable = pstate->p_rtable;
    query->rteperminfos = pstate->p_rteperminfos;
    query->jointree = makeFromExpr(pstate->p_joinlist, NULL);

    return query;
Enter fullscreen mode Exit fullscreen mode

More explanation will be provided on part 2 about what is going on the backend in terms of the database

All functions are there:

References and resources

Top comments (0)