Hello 👋
In my last article we were discussing how me and my team had set up and built dashboards with Grafana and Loki 😎.
Logs monitoring with Loki, Node.js and Fastify.js
Thomas.G ・ Jun 12 '23
Today we're going to discuss how you can leverage your logs for third-party tools using Grafana's API and Node.js 😍.
💡 The idea that starts it all
Several months ago, I started thinking about how to use our logs to build a personalized CLI experience.
After a bit of searching, I quickly found that Grafana had an API for queryings Loki logs. I immediately went crazy at the thought of what could be done with it 🔥.
🐢🚀 Node.js SDK
Over the weekend, I set to create an open source Node.js SDK.
import { GrafanaLoki } from "@myunisoft/loki";
const api = new GrafanaLoki({
// Note: if not provided, it will load process.env.GRAFANA_API_TOKEN
apiToken: "...",
remoteApiURL: "https://name.loki.com"
});
const logs = await api.queryRange(
`{app="serviceName", env="production"}`,
{
start: "1d",
limit: 200
}
);
console.log(logs);
👀 Notice the support of duration for options like start and end.
It already supports Stream and Matrix and several other APIs:
- GET /loki/api/v1/labels
- GET /loki/api/v1/label/:name/values
- GET /loki/api/v1/series
We've also added datasource APIs (which may be required depending on what you'r building).
It also include a LogParser inspired by Loki pattern.
// can be provided as an option to queryRange
const parser = new LogParser<{
method: string, endpoint: string, statusCode: number
}>("<method:httpMethod> <endpoint> <statusCode:httpStatusCode>");
const logs = await api.queryRange(
`... LogQL here ...`, { parser }
);
for (const logLine of logs) {
console.log(logLine.method);
console.log(logLine.endpoint);
}
It's still far from perfect, don't hesitate to contribute 💪.
Loki
Node.js Grafana API SDK (Loki, Datasources ..)
🚧 Requirements
- Node.js version 18 or higher
- A reverse-proxy to safely expose Loki to internet (if required).
🚀 Getting Started
This package is available in the Node Package Repository and can be easily installed with npm or yarn
$ npm i @myunisoft/loki
# or
$ yarn add @myunisoft/loki
📚 Usage
import { GrafanaApi } from "@myunisoft/loki";
import { LogQL, StreamSelector } from "@sigyn/logql";
const api = new GrafanaApi({
// Note: if not provided, it will load process.env.GRAFANA_API_TOKEN
apiToken: "...",
remoteApiURL: "https://name.loki.com"
});
const ql = new LogQL(
new StreamSelector({ app: "serviceName", env: "production" })
);
const logs = await api.Loki.queryRange(
ql, // or string `{app="serviceName", env="production"}`
{
start: "1d",
limit: 200
}
);
console
…🔎 LogQL builder
We've also built a utility package to construct log queries, inspired by the WHATWG URL/URLSearchParams API.
The package lets you build any LogQL programmatically from scratch
import { LogQL } from "@sigyn/logql";
const logql = new LogQL("foo");
logql.streamSelector.set("env", "prod");
logql.lineEq("bar");
logql.lineNotEq("baz");
// {env=\"prod\"} |= `foo` |= `bar` != `baz`
console.log(logql.toString());
But also able to parse raw string 😇
const logql = new LogQL(
"{app=\"foo\", env=\"preprod\"} |= `foo` != `bar`"
);
console.log([...logql.streamSelector.entries()]);
console.log(logql.lineFilters.lineContains());
console.log(logql.lineFilters.lineDoesNotContain());
You can use subclasses in your tools at any time, for example to retrieve labels from a LogQL 😮
import { StreamSelector } from "@sigyn/logql";
const logql = '...';
const labels = new StreamSelector(logql).kv();
📜 Credits
I'm not alone and didn't do all the work 👯. Thanks to Pierre Demailly and Sofian Doual for their contribution/support.
🙏 Conclusion
These two tools have enabled us to initiate the development of new tools. One of them is an alerting agent for Loki named Sigyn, to whom I'll be dedicating an article in the near future.
Having APIs really opens up a lot of opportunities and it'll be interesting to see what me and my team are able to produce in the future with them.
Best Regards,
Thomas
Top comments (0)