# Where can we apply business logic in ELK? For instance, I don't want logs just the way they are rather some more metadata to each row. (assuming logs are in csv) **URL:** \ **Category:** Logstash\ **Tags:** elastic-stack-monitoring, elastic-stack-alerting\ **Created:** [August 4, 2020, 6:32pm UTC](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765 "2020-08-04T18:32:05Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![AditiKhalatkar](https://avatars.discourse-cdn.com/v4/letter/a/a183cd/32.png) [@AditiKhalatkar](https://discuss.elastic.co/u/AditiKhalatkar)\ **Post date:** [August 4, 2020, 6:32pm UTC](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765/1 "2020-08-04T18:32:05Z")
Elastic search is mostly for querying data. However, I am stuck with the part where I want to ingest some logs but probably with business logic. So that the data is transformed by the time it reaches Elastic search. Can we do this while ingesting it using Logstash? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 12, 2020, 8:16am UTC](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765/2 "2020-08-12T08:16:10Z")
Hey, there are several solutions to this, logstash is one of them. Elasticsearch features a so-called [ingest node](https://www.elastic.co/guide/en/elasticsearch/reference/7.8/ingest.html) allowing you to configure a pipeline, that is a set of steps/processor which change your JSON before it is being indexed. That might help you as well! --Alex ---
**Author:** ![AditiKhalatkar](https://avatars.discourse-cdn.com/v4/letter/a/a183cd/32.png) [@AditiKhalatkar](https://discuss.elastic.co/u/AditiKhalatkar)\ **Post date:** [August 12, 2020, 6:30pm UTC](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765/3 "2020-08-12T18:30:57Z")
Thanks @spinscale. Actually I just realized I am getting data in xls format which is not supported by Logstash. Either I can write a python script to convert xls to json, ask Logstash to apply business logic or I can apply business logic in python script (too much memory usage, not ideal) and push to Elasticsearch. Which one do you think is suitable for 12 MB data? I think both approach would work similar way. ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 13, 2020, 7:38am UTC](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765/4 "2020-08-13T07:38:54Z")
if you have to write a tool that converts the excel data anyway, maybe do everything in one take then? 12MB of data does not sounds too much, so I guess you're fine there... If memory is a concern, do not load the whole file into memory, but read chunks, send them to Elasticsearch, then read the next... ---
**Author:** ![AditiKhalatkar](https://avatars.discourse-cdn.com/v4/letter/a/a183cd/32.png) [@AditiKhalatkar](https://discuss.elastic.co/u/AditiKhalatkar)\ **Post date:** [August 14, 2020, 1:26am UTC](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765/5 "2020-08-14T01:26:44Z")
Thank you! This helps. 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 11, 2020, 1:26am UTC](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765/6 "2020-09-11T01:26:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.