# Issue with Extracting "trace\_id" from Logs in Elasticsearch Using Filebeat Pipeline **URL:** \ **Category:** Kibana\ **Created:** [September 26, 2024, 1:13pm UTC](https://discuss.elastic.co/t/issue-with-extracting-trace-id-from-logs-in-elasticsearch-using-filebeat-pipeline/367185 "2024-09-26T13:13:55Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![helloworld466](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/helloworld466/32/121265_2.png) [@helloworld466](https://discuss.elastic.co/u/helloworld466)\ **Post date:** [September 26, 2024, 1:13pm UTC](https://discuss.elastic.co/t/issue-with-extracting-trace-id-from-logs-in-elasticsearch-using-filebeat-pipeline/367185/1 "2024-09-26T13:13:55Z")
I am currently pushing logs to the Elasticsearch from Filebeat. I’m trying to extract the `trace_id` from the `message` field and store the value in a separate field. The `message` field has the value: `2024-09-23 10:00:915 [http-nio-9100-exec-1] [INFO] com.ZZZ.YYY.service.UserService [XXXX-80c4-437c-b7ae-XXXXX,YYYYYY-4e90-474e-ba30-YYYYY] [ZZZ-services,post/MMM/process] : Getting user by uuid YYYYYY-4e90-474e-ba30-YYYYY` I created an ingest pipeline with a Grok pattern to capture the `trace_id`, but it doesn’t appear in Kibana. I have added the pipeline name in the filebeat configuration as well. Grok pattern: `%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:minute}:%{NUMBER:milliseconds} \[%{DATA:thread}\] \[%{LOGLEVEL:loglevel}\s\] %{JAVACLASS:class} \[%{UUID:trace_id},%{UUID:other_id}\] \[%{DATA:service}\] : %{GREEDYDATA:message}` When I tested the ingest pipeline using a document, it was able to extract is successfully. My question is why is the `trace_id` field not appearing in Kibana? Is there any additional configurations to be done? Thanks in advance. ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [September 26, 2024, 2:28pm UTC](https://discuss.elastic.co/t/issue-with-extracting-trace-id-from-logs-in-elasticsearch-using-filebeat-pipeline/367185/2 "2024-09-26T14:28:19Z")
The grok pattern cannot work because: - missing \s+ after loglevel - UUID must be in this range [A-Fa-f0-9]{8}-(?:[A-Fa-f0-9]{4}-){3}[A-Fa-f0-9]{12}, XXXXX is not allowed. Use DATA or define your own pattern `%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:minute}:%{NUMBER:milliseconds} \[%{DATA:thread}\] \[%{LOGLEVEL:loglevel}\s*\] %{JAVACLASS:class} \[%{DATA:trace_id},%{DATA:other_id}\] \[%{DATA:service}\] : %{GREEDYDATA:message}` ---
**Author:** ![helloworld466](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/helloworld466/32/121265_2.png) [@helloworld466](https://discuss.elastic.co/u/helloworld466)\ **Post date:** [September 27, 2024, 5:21am UTC](https://discuss.elastic.co/t/issue-with-extracting-trace-id-from-logs-in-elasticsearch-using-filebeat-pipeline/367185/3 "2024-09-27T05:21:27Z")
Thanks @Rios . I have updated the Grok pattern and even verified it with a sample document. But still I am not able to see the field created. Is there anything else to be done? New index has to be created for the changes to reflect? ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [September 27, 2024, 6:38am UTC](https://discuss.elastic.co/t/issue-with-extracting-trace-id-from-logs-in-elasticsearch-using-filebeat-pipeline/367185/4 "2024-09-27T06:38:04Z")
Not sure which version ELK do you have, for lower than 8.x you have to manually refresh index patterns which is now named: the data view.7 If is a test env, drop index and recreate data view. Another possibility is that your messages have optioned fields, check in ruby debug. ---
**Author:** ![ashishtiwari1993](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashishtiwari1993/32/135241_2.png) [@ashishtiwari1993](https://discuss.elastic.co/u/ashishtiwari1993)\ **Post date:** [September 27, 2024, 7:21am UTC](https://discuss.elastic.co/t/issue-with-extracting-trace-id-from-logs-in-elasticsearch-using-filebeat-pipeline/367185/5 "2024-09-27T07:21:52Z")
Could you try to have a look on Data view? Or just try to create a new data view, It should visible. ---
**Author:** ![helloworld466](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/helloworld466/32/121265_2.png) [@helloworld466](https://discuss.elastic.co/u/helloworld466)\ **Post date:** [September 30, 2024, 6:03am UTC](https://discuss.elastic.co/t/issue-with-extracting-trace-id-from-logs-in-elasticsearch-using-filebeat-pipeline/367185/6 "2024-09-30T06:03:03Z")
I am using ES version `8.5.1`. I removed the data view and created a new one. Still not able to find the `trace_id` field.