https://github.com/seanlaff/simple-streaming-datasource

JavaScript
End-to-end Grafana streaming datasource example

Simple Streaming Datasource

Inspired by the live-streaming talk at Grafanacon 2019, this is a living example of streaming within a datasource including a reference server that streams random data.

This example relies on unfinalized behavior within Grafana, and may break

This example fixes some of the shortcomings mentioned in the talk.

• Panel repaints are now throttled via RXJS throttling. You can tune the frequency to your use case.
• Streams are cancelled/restarted when queries change.

Motivation

Streaming data is a great way to reduce pressure on your metric backend/network. Rather than using a 5s dashboard refresh (which requests duplicate points over and over again), stream new data as its avaiable!

Running the example

Install as you would any grafana datasource. Then, run go run server.go. This will spawn a server at http://localhost:8080 that this datasource reads data from. Also included is a demo dashboard.

By default, the server will stream data at 50ms and the datasource plugin will throttle repaints to 100ms.

Limitations

• Clicking the dashboard refresh, changing timerange, and some other report interactions do not cause the panel to "refresh". Here's a dicusssion about it https://github.com/grafana/grafana/issues/15760
• Backend server doesn't support providing a start/end time (it always starts streaming from time.Now()) for simplicity.
• Plugin makes no attempt to order the data. If its streamed in unordered, some backwards lines will be drawn.
• Adding more than one query in the datasource is currently unsupported, but could be easily added with some more state management. You'll likely want to multiplex all queries through the same stream (rather than open a stream per query) if your datasource can support it.