How to map between extracted fields

Elastic Stack Logstash
1

Hello, I am extracting info from logfiles, but I want to map them together for aggregations, here's a sample logfile:

2017-01-01 07:53:44 [monitor_utils.py] INFO: Crawled iteration for merchant ariika started
2017-01-01 07:53:44 [utils.py] INFO: UpdateCrawlIteration._start_crawl_iteration function took 0.127 s
2017-01-01 07:57:22 [statscollectors.py] INFO: Dumping Scrapy stats:
{'item_scraped_count': 22,
 'invalid_items_count': 84}

I am extracting the merchant name from the first line ariika and items_scraped_counts, invalid_items_count from the last two lines, I have different logfiles for each merchant, and I want to know items scraped count per logfile for each merchant using Kibana.

How to filter between one merchant and another in my case?

2

Should I save each merchant logfiles in a separate elasticseach _type? I am using one index for all logfiles.

3

Have you looked using the aggregate filter?

Should I save each merchant logfiles in a separate elasticseach _type?

Probably not, but it depends.

4

No, I didn't, will take a look.
Thanks

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.