# How to read logs in subfolder of main folder filebeat
**URL:** https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888
**Category:** Beats
**Tags:** filebeat
**Created:** [January 25, 2023, 7:09am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888 "2023-01-25T07:09:07Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@devdev7711](https://discuss.elastic.co/u/devdev7711)
#### Post date: [January 25, 2023, 7:09am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/1 "2023-01-25T07:09:07Z")
HI,
I have one main folder inside this I have 2 folder I am able to read the data of both folder in filebeat but how can be show which log is belong to which subfolder.
---
### Author:  [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 25, 2023, 7:36am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/2 "2023-01-25T07:36:25Z")
@devdev7711 when the logs are ingested via filebeat, there's a field called `log.file.path` added as a metadata. That property will tell you the absolute location of the log file from where this event was read and ingested.
This field is essentially defined by ECS which every component of the Elastic Stack conforms to: [ECS fields | Filebeat Reference [8.6] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-ecs.html#_log_4)
---
### Author:  [@devdev7711](https://discuss.elastic.co/u/devdev7711)
#### Post date: [January 25, 2023, 9:11am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/3 "2023-01-25T09:11:38Z")
"log.file.path": [
"C:\ProgramData\FD\SES\Logs\user1\101.log" 10 count
"log.file.path": [
"C:\ProgramData\FD\SES\Logs\user1\102.log" 20 count
"log.file.path": [
"C:\ProgramData\FD\SES\Logs\user2\101.log" 10 count
"log.file.path": [
"C:\ProgramData\FD\SES\Logs\user1\102.log" 50 count
```
I am getting the log file like above
in above example i have 30 logs for user1 and 60 logs for user 2
i want to aggregate and show as per user wise currently it is shwoing 4 graph
how can we specify aggrgation group as per user
```
---
### Author:  [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 25, 2023, 9:26am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/4 "2023-01-25T09:26:00Z")
Can you please be clear what exactly you are trying to achieve ? As per initial description, it seemed you were only interesting in knowing about which log entry comes from which file. But now, the description is more about visualizing the data based on different field altogether.
---
### Author:  [@devdev7711](https://discuss.elastic.co/u/devdev7711)
#### Post date: [January 25, 2023, 9:27am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/5 "2023-01-25T09:27:29Z")
if you log file path i am getting data as per user folder and i want to in pie like user1 count user2 cound user3 count and many more. how can we make aggregation of specific for that.
I am making one pie graph and want to show like user1 30 count and user 2 60 count
---
### Author:  [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 25, 2023, 1:12pm UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/6 "2023-01-25T13:12:50Z")
Ok, I quite don't understand why log file path was your concern when you want a pie chart visualization based on some field.
About pie chart, I believe you cannot have `min_doc_count` functionality there without some additional Kibana plugins like Vega.
---
### Author:  [@devdev7711](https://discuss.elastic.co/u/devdev7711)
#### Post date: [January 25, 2023, 4:27pm UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/7 "2023-01-25T16:27:48Z")
Our requirement to show the log count as per the user which user have how many log count this is our requirement. IN our project we are getting log data within user name folder.
---
### Author:  [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 25, 2023, 4:37pm UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/8 "2023-01-25T16:37:29Z")
OK, for pie chart, use metric as `count` and split buckets on `user` field.
---
### Author:  [@devdev7711](https://discuss.elastic.co/u/devdev7711)
#### Post date: [January 25, 2023, 4:40pm UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/9 "2023-01-25T16:40:20Z")
I do not have user value in data we need to extract from the log.file.path value so we can get user1 user2 how can we extract this.
I am getting data in this way.
"log.file.path": [
"C:\ProgramData\UNO\SES\Logs\vs00776071\UnoSES25012023.log"
],
"log.offset": [
0
],
"message": [
"25-Jan-2023 14:12:56 INFO : BOT license verification successful."
],
"suricata.eve.timestamp": [
"2023-01-25T08:43:04.381Z"
Can we create new field and store username like user1 as per the data is coming
---
### Author:  [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 25, 2023, 5:18pm UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/10 "2023-01-25T17:18:50Z")
For a smaller use case, you can use `add_field` processor of filebeat and based on log file path, set the value for userID. But, imagine you have 1000s of users and logs are being shipped for all of them. This approach will become cumbersome and unmanageable.
One option would be to store the log file for each user under `/user${ID}` directory and using userID field can then be added from logstash based on log.file.path substring (probably `gsub` processor is required, but not sure).
Another way would be to create an ingest pipeline using script, get the substring of `log.file.path` to store only user${ID} as per approach # 1. Add a new field in \_ctx with the value retrieved and create pie chart based on that field.
---
### Author:  [@devdev7711](https://discuss.elastic.co/u/devdev7711)
#### Post date: [January 26, 2023, 4:59am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/11 "2023-01-26T04:59:40Z")
is it possible to extract userid from logpath location value and store in new field
somewhere I heard about tokenizer.
---
### Author:  [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 26, 2023, 7:28am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/12 "2023-01-26T07:28:55Z")
Yes it's possible using tokenizer [Dissect strings | Filebeat Reference [8.6] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/dissect.html)
Or, you can use ingest pipeline to grab the substring of log file path and add another field using script processor: [Script processor | Elasticsearch Guide [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/script-processor.html)
---
### Author:  [@devdev7711](https://discuss.elastic.co/u/devdev7711)
#### Post date: [January 26, 2023, 9:47am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/13 "2023-01-26T09:47:54Z")
for dissect string which file and which configuration we need to make change which setting exactly what we need to do it for can you please suggest
---
### Author:  [@devdev7711](https://discuss.elastic.co/u/devdev7711)
#### Post date: [January 29, 2023, 4:27am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/14 "2023-01-29T04:27:12Z")
@Ayush_Mathur How can we implement as per your suggestion??
Can you please suggest step by step?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 26, 2023, 4:27am UTC](https://discuss.elastic.co/t/how-to-read-logs-in-subfolder-of-main-folder-filebeat/323888/15 "2023-02-26T04:27:22Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.