Any plans for a visual Logstash pipeline editor?


(Val Crettaz) #1

I was wondering if there were any plans of providing a visual pipeline editor for Logstash at some point?
After writing tons of Logstash configuration files in many different projects over the past few years, I started feeling the need to be able to create Logstash pipelines in a more efficient and sexy way than writing boring flat config files. Since almost everything tends to be visual these days, why not going through some kind of flow editor to build Logstash pipelines.

I looked around but didn’t find much going in that direction… so I started building a raw prototype based on the Node-RED IoT flow editor. The main idea was merely to benefit from Node-RED’s excellent environment and see how it could help assemble Logstash input/filter/output plugins together into a full-fledge LS pipeline.

Ultimately, I would like to port this to a Kibana app, which I think would make sense and integrate Logstash even more tightly into the Elastic Stack ecosystem.
Logstash pipelines designed in such a Kibana app could either be exported to a Logstash config file as we know them and/or be stored in a special ES index, where each LS input/filter/output plugin/node would be modeled as a document (vertex) and each of those would have input/output edges to other documents/nodes part of the pipeline... basically into some kind of directed graph (sounds familiar?).

So back to my original question: does anyone of you know if there are any similar efforts already going on either internally at Elastic or in the community?


(Mark Walkom) #2

It's something we'd like to be able to add in the future, especially once we add APIs to allow dynamic config management.

However I haven't seen anything similar, so thanks for sharing and good luck :slight_smile:


(Val Crettaz) #3

Thank you very much for your input Mark, much appreciated!


(Val Crettaz) #4

Glad to see that the Logstash team has been working on this :wink:




(Andrew Cholakian) #5

Thanks @val :slight_smile: We're hoping to blog about it once the code is further along.


(system) #6