Custom logs not appearing in visualization

Hi all,

I created custom logs in NGINX to log for http.response.time and http.response.upstream.time.

I updated the grok filter on the NGINX module pipeline for Filebeat.

I am receiving the data on the Filebeat-* index and everything seems to work fine.

But when I got to create a TSVB visualization I am not able to find the above fields. Even though I am using the right index and I can see the data in the Discovery tab the fields are not coming up int he field box.

Is there something I am missing that needs to be done for the visualizations to work? I would like to get a dashboard of the response times for each requests.

UPDATE:

I looked in the index pattern and noticed that it's identiyfing the field as a string. Even though I have the following in my filter:
%{NUMBER:http.response.time} %{NUMBER:nginx.response.upstream.time:}"

I add :long to the end of those because the other patterns had and it resulted in an error, so it is picking it up as a string for some strange reason.

I ended up solving this issue by adding:
%{NUMBER:http.response.time:float} %{NUMBER:nginx.response.upstream.time:float}.

Do note that the reason it didn't work for me is because the fields I created were already strings. You can change number to strings but not the other way around. So I created a new field and added :float.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.