# JSON parsing creates a field for each user **URL:** \ **Category:** Kibana\ **Created:** [August 5, 2018, 5:09pm UTC](https://discuss.elastic.co/t/json-parsing-creates-a-field-for-each-user/143067 "2018-08-05T17:09:17Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![ansorren](https://avatars.discourse-cdn.com/v4/letter/a/bc8723/32.png) [@ansorren](https://discuss.elastic.co/u/ansorren)\ **Post date:** [August 5, 2018, 5:09pm UTC](https://discuss.elastic.co/t/json-parsing-creates-a-field-for-each-user/143067/1 "2018-08-05T17:09:17Z")
Apologies if it's a silly question but I googled around and I couldn't find a solution, I don't have much experience with Elasticsearch. I have an ES 5.5 stack deployed over the AWS ES service, and I have some JSON logs that I would like to parse. My problem is that ES adds a users.user1 field, so now I have hundreds of those and I cannot select them manually when creating a dashboard because they're way too many. { "\_index": "cwl-2018.08.05", "\_type": "/aws/lambda/tagtest", "\_id": "34197741497112809064849409673362766816549106281078849536", "\_version": 1, "\_score": 1, "\_source": { "users": { "user1": [ "tag1", "tag2", "tag3" ] }, "@id": "34197741497112809064849409673362766816549106281078849536", "@timestamp": "2018-08-05T14:36:04.585Z", "@message": "{"users": {"user1": ["tag1", "tag2", "tag3",]}}\n", "@owner": "024533801171", "@log\_group": "/aws/lambda/tagtest", "@log\_stream": "2018/08/05/[$LATEST]5f7a5a39fcbc4e08b0c011bbf7d5112d" }, "fields": { "@timestamp": [ 1533479764585 ] } } What I would need is a CSV with three columns: the usernames, the number of times these usernames were found in the logs, and the tags associated with each username (also, an username can have a tag1 in one log and a tag2 in another log, I would need both tags) Anyone knows how can I achieve this in Kibana? Thank you, ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 6, 2018, 4:42am UTC](https://discuss.elastic.co/t/json-parsing-creates-a-field-for-each-user/143067/2 "2018-08-06T04:42:55Z")
Does each log have only one Username inside the users array? You will not be able to get the o/p you desire (CSV) directly, till you probably normalize your logs using some ETL like Logstash. ---
**Author:** ![ansorren](https://avatars.discourse-cdn.com/v4/letter/a/bc8723/32.png) [@ansorren](https://discuss.elastic.co/u/ansorren)\ **Post date:** [August 6, 2018, 5:21am UTC](https://discuss.elastic.co/t/json-parsing-creates-a-field-for-each-user/143067/3 "2018-08-06T05:21:27Z")
Thank you for the reply, no, currently the number of users in each log can vary, but if that's a problem since I control the application I might be able to modify the JSON to send a single user per log. Any idea if I could accomplish it using any kind of aggregation? The log looks like it's being parsed and the JSON is RFC compliant, so I thought that there was a way using the Kibana data table dashboards and then export as CSV. ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 6, 2018, 5:36am UTC](https://discuss.elastic.co/t/json-parsing-creates-a-field-for-each-user/143067/4 "2018-08-06T05:36:59Z")
No. Actually the value items are coming as keys. Therefore, aggregation will not be possible. If the fields or logs were in the following format, then it would be properly indexed: ```auto { "users" : "john", "tags": ["tag1", "tag2"] } ``` ---
**Author:** ![ansorren](https://avatars.discourse-cdn.com/v4/letter/a/bc8723/32.png) [@ansorren](https://discuss.elastic.co/u/ansorren)\ **Post date:** [August 7, 2018, 4:54am UTC](https://discuss.elastic.co/t/json-parsing-creates-a-field-for-each-user/143067/5 "2018-08-07T04:54:23Z")
Thank you for your help. I changed the log format of the application to match your format and I can see that the users are now being parsed as values. I just have one doubt on how to proceed, I'm configuring my aggregations as follow: ![35](https://us1.discourse-cdn.com/elastic/original/3X/a/0/a00b82868bbc52fe63b1a9c54e065620ba7ea7b4.png) ![43](https://us1.discourse-cdn.com/elastic/original/3X/e/a/eaa3e27cb4a3c8d14bca20bc226240f4c8acdcc4.png) This gives me several lines of the tag and the corresponding user, but apparently it doesn't report the users without tags, such as: { "users" : "john", "tags" : } Any idea how should I configure the aggregation in this case to display also the users not associated with any tag? Thank you again for your help, I really appreciate it ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [August 7, 2018, 10:19am UTC](https://discuss.elastic.co/t/json-parsing-creates-a-field-for-each-user/143067/6 "2018-08-07T10:19:24Z")
> [@ansorren](#): > > Any idea how should I configure the aggregation in this case to display also the users not associated with any tag? Thank you again for your help, I really appreciate it Create a simple data table based on either unique count or count depending on requirement. Display the columns needed and then use a does not exist filter for the tag field. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 4, 2018, 10:19am UTC](https://discuss.elastic.co/t/json-parsing-creates-a-field-for-each-user/143067/7 "2018-09-04T10:19:26Z")
Closed on September 4, 2018, 10:19am UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.