# Help Looking/ indexing/ extracting data

**URL:** <https://discuss.elastic.co/t/help-looking-indexing-extracting-data/82179>\
**Category:** Kibana\
**Created:** [April 12, 2017, 2:51pm UTC](https://discuss.elastic.co/t/help-looking-indexing-extracting-data/82179 "2017-04-12T14:51:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![adityajain19](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adityajain19/32/23171_2.png) [@adityajain19](https://discuss.elastic.co/u/adityajain19)\
**Post date:** [April 12, 2017, 2:51pm UTC](https://discuss.elastic.co/t/help-looking-indexing-extracting-data/82179/1 "2017-04-12T14:51:26Z")

</div>

Hello everyone,

we have data as following:

{  
"\_index": "logstash-2017.04.12",  
"\_type": "fluentd",  
"\_id": "AVtiVXDVEWKEV5H6MuHL",  
"\_score": null,  
"\_source": {  
"eais\_prod": "1 2017-04-12T13:23:30.275Z ip-4-0-6-78 - messageId-socket-a - Logs = ,,2017-04-12,13:23:29.922,0-37fcf132-1f83-11e7-9fb6-064267958b48,[something.host.io](http://something.host.io),operation,200,OK,'9B24D5, 'DS',,INTERNET,,,,,,PROD"  
"@timestamp": "2017-04-12T09:23:30-04:00"  
},  
"fields": {  
"@timestamp": [  
1492003410000  
]  
},  
"highlight": {  
"eais\_prod": [  
"1 2017-04-12T13:23:30.275Z ip-4-0-6-78 - messageId-socket-a - Logs = ,,2017-04-12,13:23:29.922,0-37fcf132-1f83-11e7-9fb6-064267958b48,[something.host.io](http://something.host.io),operation,200,OK,'9B24D5, 'DS',,INTERNET,,,,,,PROD"  
]  
},  
"sort": [  
1492003410000  
]  
}

we need to split this line  
,,2017-04-12,13:23:29.922,0-37fcf132-1f83-11e7-9fb6-064267958b48,[something.host.io](http://something.host.io),operation,200,OK,'9B24D5, 'DS',,INTERNET,124,,,,,PROD"

and do sum or aggregate function on 124 index in the above CSV

we will have multiple data in that format and we want to get lines that have INTERNET and then next field and do a sum on that.

Regards  
Aditya

---

<div class="post-metadata">

**Author:** ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)\
**Post date:** [April 12, 2017, 10:23pm UTC](https://discuss.elastic.co/t/help-looking-indexing-extracting-data/82179/2 "2017-04-12T22:23:39Z")

</div>

Can you show us what you have when you go to the index pattern in Kibana \> Management \> Index Patterns. It would us to understand how your data was parsed into fields.

Something like this (you can paste a screenshot in this forum);

 ![](https://us1.discourse-cdn.com/elastic/original/3X/c/3/c3f966c3f4d8dcb3ff2312d881822fba07012a19.png)

What version of Kibana are you using?

It looks like you're loading the data with Logstash. It looks like you probably need to parse the data in Logstash before it goes into Elasticsearch.

Thanks,  
Lee

---

<div class="post-metadata">

**Author:** ![adityajain19](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adityajain19/32/23171_2.png) [@adityajain19](https://discuss.elastic.co/u/adityajain19)\
**Post date:** [April 13, 2017, 1:00am UTC](https://discuss.elastic.co/t/help-looking-indexing-extracting-data/82179/3 "2017-04-13T01:00:39Z")

</div>

Thanks Lee for the updates  
I am very new to EK we use Fluentd instead of Logstash.

our input from application to FluentD is like

Logs = ,,2017-04-12,13:23:29.922,0-37fcf132-1f83-11e7-9fb6-064267958b48,[something.host.io](http://something.host.io),operation,200,OK,'9B24D5, 'DS',,INTERNET,,,,,,PROD

we send that from FluenD to Elastic i think we are using default index.

Logstash\*

once we come here we just grep but now we want to improve. the version of kibana is 5.2.2 and i think elastic is also latest. we also have installed X-pack.

 ![](https://us1.discourse-cdn.com/elastic/original/3X/3/8/38a7c13d48c4cd91ac31dcb1be0562fdc8ea40a3.png)

Regards  
Aditya

---

<div class="post-metadata">

**Author:** ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)\
**Post date:** [April 14, 2017, 1:34pm UTC](https://discuss.elastic.co/t/help-looking-indexing-extracting-data/82179/4 "2017-04-14T13:34:05Z")

</div>

Hi Aditya,

I'm afraid I don't know FluentD. You should get that incoming data parsed into fields instead of everything going into "eais\_prod" (at least that's what it looks like from here). I'm not sure how to do that with FluentD. Here's an example of where they're talking about creating a mapping [https://github.com/uken/fluent-plugin-elasticsearch/issues/33](https://github.com/uken/fluent-plugin-elasticsearch/issues/33)

This question might find someone that can help you more if you post in the Elasticsearch or Logstash forums.

Regards,  
Lee

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 12, 2017, 1:35pm UTC](https://discuss.elastic.co/t/help-looking-indexing-extracting-data/82179/5 "2017-05-12T13:35:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
