DEV Community

Thomas.G
Thomas.G

Posted on

Consuming Loki logs with Grafana API and Node.js

Hello 👋

In my last article we were discussing how me and my team had set up and built dashboards with Grafana and Loki 😎.

Today we're going to discuss how you can leverage your logs for third-party tools using Grafana's API and Node.js 😍.

💡 The idea that starts it all

Several months ago, I started thinking about how to use our logs to build a personalized CLI experience.

After a bit of searching, I quickly found that Grafana had an API for queryings Loki logs. I immediately went crazy at the thought of what could be done with it 🔥.

🐢🚀 Node.js SDK

Over the weekend, I set to create an open source Node.js SDK.



import { GrafanaLoki } from "@myunisoft/loki";

const api = new GrafanaLoki({
  // Note: if not provided, it will load process.env.GRAFANA_API_TOKEN
  apiToken: "...",
  remoteApiURL: "https://name.loki.com"
});

const logs = await api.queryRange(
  `{app="serviceName", env="production"}`,
  {
    start: "1d",
    limit: 200
  }
);
console.log(logs);


Enter fullscreen mode Exit fullscreen mode

👀 Notice the support of duration for options like start and end.

It already supports Stream and Matrix and several other APIs:

  • GET /loki/api/v1/labels
  • GET /loki/api/v1/label/:name/values
  • GET /loki/api/v1/series

We've also added datasource APIs (which may be required depending on what you'r building).

It also include a LogParser inspired by Loki pattern.



// can be provided as an option to queryRange
const parser = new LogParser<{
  method: string, endpoint: string, statusCode: number
}>("<method:httpMethod> <endpoint> <statusCode:httpStatusCode>");

const logs = await api.queryRange(
  `... LogQL here ...`, { parser }
);
for (const logLine of logs) {
  console.log(logLine.method);
  console.log(logLine.endpoint);
}


Enter fullscreen mode Exit fullscreen mode

It's still far from perfect, don't hesitate to contribute 💪.

GitHub logo MyUnisoft / loki

Node.js Loki SDK

Loki

Node.js Grafana API SDK (Loki, Datasources ..)

npm version license ossf scorecard github ci workflow size

🚧 Requirements

🚀 Getting Started

This package is available in the Node Package Repository and can be easily installed with npm or yarn

$ npm i @myunisoft/loki
# or
$ yarn add @myunisoft/loki
Enter fullscreen mode Exit fullscreen mode

📚 Usage

import { GrafanaApi } from "@myunisoft/loki";
import { LogQL, StreamSelector } from "@sigyn/logql";
const api = new GrafanaApi({
  // Note: if not provided, it will load process.env.GRAFANA_API_TOKEN
  apiToken: "...",
  remoteApiURL: "https://name.loki.com"
});

const ql = new LogQL(
  new StreamSelector({ app: "serviceName", env: "production" })
);
const logs = await api.Loki.queryRange(
  ql, // or string `{app="serviceName", env="production"}`
  {
    start: "1d",
    limit: 200
  }
);
console
Enter fullscreen mode Exit fullscreen mode

🔎 LogQL builder

We've also built a utility package to construct log queries, inspired by the WHATWG URL/URLSearchParams API.

The package lets you build any LogQL programmatically from scratch



import { LogQL } from "@sigyn/logql";

const logql = new LogQL("foo");
logql.streamSelector.set("env", "prod");
logql.lineEq("bar");
logql.lineNotEq("baz");

// {env=\"prod\"} |= `foo` |= `bar` != `baz`
console.log(logql.toString());


Enter fullscreen mode Exit fullscreen mode

But also able to parse raw string 😇



const logql = new LogQL(
  "{app=\"foo\", env=\"preprod\"} |= `foo` != `bar`"
);

console.log([...logql.streamSelector.entries()]);
console.log(logql.lineFilters.lineContains());
console.log(logql.lineFilters.lineDoesNotContain());


Enter fullscreen mode Exit fullscreen mode

You can use subclasses in your tools at any time, for example to retrieve labels from a LogQL 😮



import { StreamSelector } from "@sigyn/logql";

const logql = '...';
const labels = new StreamSelector(logql).kv();

Enter fullscreen mode Exit fullscreen mode




📜 Credits

I'm not alone and didn't do all the work 👯. Thanks to Pierre Demailly and Sofian Doual for their contribution/support.

🙏 Conclusion

These two tools have enabled us to initiate the development of new tools. One of them is an alerting agent for Loki named Sigyn, to whom I'll be dedicating an article in the near future.

Having APIs really opens up a lot of opportunities and it'll be interesting to see what me and my team are able to produce in the future with them.

Best Regards,
Thomas

Top comments (0)