Filebeat Monitor app (Visualize or Tail)

[Saleem]

Sorry i'm new to the elasticstack, but bear with me

I'm trying to setup ES-Kibana-Filebeat for Java application logs processing

The specific log files i'm trying to visualize aren't one of the log types with modules

They look like this(This is just an example):

2021-8-19 18:20:43.124 L Starting JMX Shell Server
Thread: localhost-startStop-1:

2021-8-19 18:20:43.124 L Starting Thread system
Thread: localhost-startStop-1:

2021-8-19 18:23:12.135 U [.APPNAME] added driver <etc...>
Thread: "APPNAME" startup:

Now what i've done is use Filebeat's mutliline patttern and indexed them in ES, but when i try to visualize the messages i can't

How do i monitor these kinds of logs using ES-Kibana-Filebeat?
Can they be visualized or tailed? and how?

Is using filestream a good way to tail them? As these logs will be written by the app constantly and i need to visualize them or tail them for monitoring

Any help is appreciated, i would be happy to provide any other details

[warkolm (Mark Walkom)]

Welcome to our community!

@Saleem

Now what i've done is use Filebeat's mutliline patttern and indexed them in ES, but when i try to visualize the messages i can't

Can you provide more information about this, you can't why exactly?

[stephenb (Stephen Brown)]

@Saleem

I suggest you look at this

Log monitoring | Observability Guide [7.14] | Elastic

And here's our equivalent of tailing a log file it's log streaming and it's pretty cool

Tail log files | Observability Guide [7.14] | Elastic

[Saleem]

Thank you!

Well when i got to kibana and try to create a visualization, the message field is not there so i can't pick it.

I read about it and apparently it's not aggregatable , what i want is just to monitor this specific kind of logfiles, and display them or tail them to monitor the application. It might be a simple solution but i haven't figure it out yet.

[Saleem]

Alright i'll check them out thank you so much

[stephenb (Stephen Brown)]

If you go under the log streaming settings you can pick the fields that you want to show.

Here I added a field I wanted to see

Then in the actual log stream app you can filter using a KQL. So then you'll only see logs that meet that criteria I it display the fields you want

Also you can basically do the same thing and Discover and just set a refresh rate of 10 seconds.