# Split logs per customers **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [August 8, 2022, 4:16pm UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658 "2022-08-08T16:16:25Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![YannKistenmacker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yannkistenmacker/32/109407_2.png) [@YannKistenmacker](https://discuss.elastic.co/u/YannKistenmacker)\ **Post date:** [August 8, 2022, 4:16pm UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/1 "2022-08-08T16:16:25Z")
Hello everyone, I'm new elasticsearch user and i'm in charge to develop a system to monitoring our customers logs. So starting by this point i have to monitoring Fortinet Firewall Logs and i've already did with filebeat sending directly to elasticsearch. I have one index with 3 or 4 customers sending logs simultaneoustly and i need to split them to see their log size by customer. Someone can help me, please ? Or maybe a way to query via console ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 8, 2022, 9:22pm UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/2 "2022-08-08T21:22:51Z")
Welcome to our community! 😃 What version of Elasticsearch are you using? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [August 8, 2022, 10:35pm UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/3 "2022-08-08T22:35:51Z")
Hi @YannKistenmacker Also Welcome to the community. It all comes down to how precise you need to be. It's very simple to just create a percentage of the total logs by customer as a count by customer terms query or Lens visualization and then that's their percentage of the total storage / cost.. It can be that simple if you want. Firewall logs tend to even out in size across the landscape. So if that's good enough, no need to even break up the index. It'll save you a lot of other trouble 🙂 If you need to know the exact bytes ( which in the end may give you some tiny percentage of precision better) than you will have to break up into separate indices. ---
**Author:** ![YannKistenmacker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yannkistenmacker/32/109407_2.png) [@YannKistenmacker](https://discuss.elastic.co/u/YannKistenmacker)\ **Post date:** [August 9, 2022, 11:25am UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/4 "2022-08-09T11:25:11Z")
Version: 8.3.3 ---
**Author:** ![YannKistenmacker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yannkistenmacker/32/109407_2.png) [@YannKistenmacker](https://discuss.elastic.co/u/YannKistenmacker)\ **Post date:** [August 9, 2022, 11:40am UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/5 "2022-08-09T11:40:36Z")
Based on your answer i don't need to get the exact amount of data, a percentage's sufficient for me. Can you show me how can i get this percentage without split indices ? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [August 9, 2022, 9:32pm UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/6 "2022-08-09T21:32:36Z")
Hi @YannKistenmacker Couple questions how are you collecting the logs? Are you parsing them? What fields do you have that have the customer field? I mean it could be as simple as a Pie Chart Kibana -\> Visualization -\> Lens -\> Pie Chart This is a simple example This is my count of records by App.Name yours would be by your customer field. ![Screen Shot 2022-08-09 at 3.31.38 PM](https://us1.discourse-cdn.com/elastic/original/3X/3/b/3b46a6015dea32f341a182867b4ec5ce450feedf.png) ---
**Author:** ![YannKistenmacker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yannkistenmacker/32/109407_2.png) [@YannKistenmacker](https://discuss.elastic.co/u/YannKistenmacker)\ **Post date:** [August 10, 2022, 11:55am UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/7 "2022-08-10T11:55:07Z")
My infraestructure: Filebeat sends log directly to elasticsearch and parsing the data with "Fortinet Module". I have one field that brings info about customers, in this case "observer.name" field where i can see the name of the customer firewall present in all log entries. I already did a pie with percentage like you, but i would like to know if there is a way to put it in KB/MB/GB form instead percentage. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [August 10, 2022, 10:14pm UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/8 "2022-08-10T22:14:26Z")
Ok Yes but it is not super clean... and requires a manual steps In general your will calculate the avg bytes / log then create a visualization that can show overall storage So First run this command in Kibana - Dev Tools the `bytes=b` means it will show storage in bytes this is sorted by largest but you can also just `s=index` to sort by index `GET /_cat/indices/*?v&s=pri.store.size:desc&bytes=b` Example My Results ```auto health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open .ds-filebeat-8.2.3-2022.08.06-000146 e1us_WnaRJa26jlYejAxNA 3 1 11447320 0 12206153071 6103214925 green open .ds-filebeat-8.2.3-2022.08.07-000149 Z51Vz3VWRRCrmoXJVAajPg 3 1 11433071 0 12194502328 6096678843 ``` So then do the math... the `store.size` is the size of the primary plus what ever replicas... You can use either but the total storage is `store.size` so for me Avg Bytes / Doc = 12206153071 bytes / 11447320 Docs = 1066 Bytes / Doc for prim+replicas (~500bytes/log primary is very typical) So then lets make a Lens Table I am basing my on App Name but you can use Customer This is a table but same for Pie Chart etc..etc.. **NOTE :** This is for the Time period of the Time Picker so keep that in mind ![Screen Shot 2022-08-10 at 4.08.22 PM](https://us1.discourse-cdn.com/elastic/original/3X/0/1/013779da14517d948e426bd9e2c2a95ca59b579a.jpeg) Then in the Rows the App Name ![Screen Shot 2022-08-10 at 4.09.13 PM](https://us1.discourse-cdn.com/elastic/original/3X/4/8/488e5e9ec3ae8adfdd26e5a13491dc5bac53dcd3.png) Then in The Metric choose formula it will even automatically format for Bytes simply `count() * 1066` (my avg bytes per log) ![Screen Shot 2022-08-10 at 4.12.16 PM](https://us1.discourse-cdn.com/elastic/original/3X/9/0/901894ab23e35ab5bd5b366ad49886993f535d1d.png) Hope that helps ---
**Author:** ![YannKistenmacker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yannkistenmacker/32/109407_2.png) [@YannKistenmacker](https://discuss.elastic.co/u/YannKistenmacker)\ **Post date:** [August 11, 2022, 11:42am UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/9 "2022-08-11T11:42:25Z")
Okk! I'm going to try to apply this solution today, but looking at it, it seems to me that it will work that way. I will return when i apply these steps. ---
**Author:** ![YannKistenmacker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yannkistenmacker/32/109407_2.png) [@YannKistenmacker](https://discuss.elastic.co/u/YannKistenmacker)\ **Post date:** [August 11, 2022, 12:46pm UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/10 "2022-08-11T12:46:34Z")
@stephenb I would like to thank you for all the support, this worked perfectly for me!!! Thank you!!! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 8, 2022, 2:47pm UTC](https://discuss.elastic.co/t/split-logs-per-customers/311658/11 "2022-09-08T14:47:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.