# Search time field extraction

**URL:** <https://discuss.elastic.co/t/search-time-field-extraction/15982>\
**Category:** Elasticsearch\
**Created:** [February 24, 2014, 5:01am UTC](https://discuss.elastic.co/t/search-time-field-extraction/15982 "2014-02-24T05:01:14Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ES\_Newbie1](https://avatars.discourse-cdn.com/v4/letter/e/f07891/32.png) [@ES\_Newbie1](https://discuss.elastic.co/u/ES_Newbie1)\
**Post date:** [February 24, 2014, 5:01am UTC](https://discuss.elastic.co/t/search-time-field-extraction/15982/1 "2014-02-24T05:01:14Z")

</div>

I am a newbie to ES/Kibana/logstash and still getting my feet wet.

I could be missing something obvious but I haven't come across a way so far  
to extract a field at search time i.e., without setting up a  
grok{}/grep{}/similar filter in logstash. The grok{} filter is great for  
an event format that we know the structure but it isn't always practical to  
setup filters for all interested fields ahead of the time.

As an example, one of the events I am working on has the following format:

Trigger: LOGON Start Time: 2014-02-24T04:42:28 End Time:

> 2014-02-24T04:42:31 Duration: 2918ms.

I wanted to graph on the Duration value i.e., 2918 and would like to  
extract it. I setup a grok{} filter and was able to extract it but given  
that the extraction is index-time, only future events gets the field  
indexed.

I am wondering if someone knows a way/have a suggestion on how to:

1. extract a field of interest at search time
2. refresh an index when a new grok{} filter is added and a new field  
extracted

Appreciate your help!

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/27edad95-a6f3-4729-ba05-6171a1ed6920%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/27edad95-a6f3-4729-ba05-6171a1ed6920%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Binh\_Ly](https://avatars.discourse-cdn.com/v4/letter/b/ce7236/32.png) [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)\
**Post date:** [February 24, 2014, 10:27pm UTC](https://discuss.elastic.co/t/search-time-field-extraction/15982/2 "2014-02-24T22:27:39Z")

</div>

Technically, you can probably do this with a little scripting and the  
script\_fields functionality:

[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-script-fields.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-script-fields.html)

However, Kibana does not expose this at the moment so for now, you'll need  
to produce the exact field values at index time. And yes it will not apply  
to already indexed data as you discovered - you will need to delete those  
old indexes and then re-run the logs again using your new LS config file if  
you want the values from the old logs.

Btw, incase you've not seen this, there is a nifty grok debugger here  
(makes grokking and testing a lot quicker):

[http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/)

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8f8ba448-6330-4b4f-9410-14454845b6b7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8f8ba448-6330-4b4f-9410-14454845b6b7%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:47am UTC](https://discuss.elastic.co/t/search-time-field-extraction/15982/3 "2017-07-06T01:47:40Z")

</div>