Debugging questions + advice request

Elastic Stack Logstash
1

Hi,

I've started using Logstash coming from different pipeline products and I find it hard to understand what's wrong with my pipelines when it doesn't behave as expected.

For example, I've copied one simple JDBC input to Elastc Output and i get no documents. This is fine. Probably something is mis-configured or other issue.

The problem is, I don't get any logs, even when debug, it's at minimum and I have no indication if the username is wrong, or the query syntax is wrong, or even if the input tried to be executed.

I also don't know how to see how one event looks at a particular logstash, is there a LogEvent filter which can log it somehow?

Any advice from experienced logstash user on how you debug your pipelines from start to finish? Thanks

2

The problem is, I don't get any logs, even when debug, it's at minimum and I have no indication if the username is wrong, or the query syntax is wrong, or even if the input tried to be executed.

With debug logging enabled you should get something, but if the problem is very early there might not be much to log.

I also don't know how to see how one event looks at a particular logstash, is there a LogEvent filter which can log it somehow?

I don't understand what a "LogEvent filter" would do, but you can e.g. use a file our a stdout output to dump the events to something other than Elasticsearch.

3

Say I have a complex pipeline with 20 ~ filters and something is not as expected at one point while filtering. I would like to know what my event looks live before I'm trying to split it, etc.

4

Okay. I haven't heard of someone making such a filter but it should be trivial. In the meantime you could use a ruby filter with a line or two of custom ruby code for dumping the event.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.