# Logstash :Config file to parse nested json

**URL:** <https://discuss.elastic.co/t/logstash-config-file-to-parse-nested-json/281528>\
**Category:** Logstash\
**Created:** [August 16, 2021, 10:41am UTC](https://discuss.elastic.co/t/logstash-config-file-to-parse-nested-json/281528 "2021-08-16T10:41:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ippo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ippo/32/93183_2.png) [@ippo](https://discuss.elastic.co/u/ippo)\
**Post date:** [August 16, 2021, 10:41am UTC](https://discuss.elastic.co/t/logstash-config-file-to-parse-nested-json/281528/1 "2021-08-16T10:41:48Z")

</div>

hello i 'm new to elk and i seeking help for my mission. i want to export the data from the log of json file below so that i have visualizations related to the number of successful builds . im trying to remove the fields that i dont need for this . you will find below my configuration file . when i run it i m blocked on this line

 ![MicrosoftTeams-image](https://us1.discourse-cdn.com/elastic/original/3X/4/7/477db2714f391e5dd85962d5e8a2f362bad9def4.png)

Also im not sure if this line is correct or i need to remove the fields through looping all the builds for each job

```auto
 remove_field => ["url","color","[builds][url]","[builds][details][artifacts]","[builds][details][building]","[builds][details][displayName]","[builds][details][estimatedDuration]","[builds][details][executor]" ]

```

the json log:

```auto
[{
    "name": "ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION",
    "url": "http:///ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/",
    "color": "blue",
    "builds": [{
        "number": 49,
        "url": "http://""""/ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/49/",
        "details": {
            "artifacts": [],
            "building": false,
            "displayName": "#49",
            "duration": 30836522,
            "estimatedDuration": 30436797,
            "executor": null,
            "fullDisplayName": "ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION #49",
            "id": "49",
            "keepLog": false,
            "number": 49,
            "queueId": 1182034,
            "result": "SUCCESS",
            "timestamp": 1628477169237,
            "url": "http:///ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/49/",
            "builtOn": "",
            "culprits": []
        }
    }, {
        "number": 48,
        "url": "http:///ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/48/",
        "details": {
            "artifacts": [],
            "building": false,
            "displayName": "#48",
            "duration": 29625208,
            "estimatedDuration": 30436797,
            "executor": null,
            "fullDisplayName": "ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION #48",
            "id": "48",
            "keepLog": false,
            "number": 48,
            "queueId": 1146447,
            "result": "SUCCESS",
            "timestamp": 1627872360241,
            "url": "http://""""""/ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/48/",
            "builtOn": "",
            "culprits": []
        }
    }, {
        "number": 47,
        "url": "http://""""/ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/47/",
        "details": {
            "artifacts": [],
            "building": false,
            "displayName": "#47",
            "duration": 30848660,
            "estimatedDuration": 30436797,
            "executor": null,
            "fullDisplayName": "ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION #47",
            "id": "47",
            "keepLog": false,
            "number": 47,
            "queueId": 1107340,
            "result": "SUCCESS",
            "timestamp": 1627267567286,
            "url": "http://""""/ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/47/",
            "builtOn": "",
            "culprits": []
        }
    }]
}]

```

config file

```auto
input {
  file {
    
    path => "C:/Users/SOAR07211/Desktop/demo_jobs_ordonnanceur3.json"
	start_position => "beginning"
	codec => json
    
	}
  }

filter {
mutate{ 
        remove_field => ["url","color","[builds][url]","[builds][details][artifacts]","[builds][details][building]","[builds][details][displayName]","[builds][details][estimatedDuration]","[builds][details][executor]" ]
	}
json {
         source => "message" 

    
    }
	}
  
 

output {
   elasticsearch {
   hosts => "localhost:9200"
   index => "index_demo5_fichier_json"  
  }
  
}

```

any help is welcome

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [August 16, 2021, 11:26am UTC](https://discuss.elastic.co/t/logstash-config-file-to-parse-nested-json/281528/2 "2021-08-16T11:26:13Z")

</div>

When reading a file Logstash tracks what has been read or not in order to only pull in new data using [sincedb\_path setting](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_tracking_of_current_position_in_watched_files). Add this to your input and give it a shot.

`sincedb_path => '/dev/null'`

What I think is going on is this file has already been read and some point and Logstash won't read those records again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 13, 2021, 11:26am UTC](https://discuss.elastic.co/t/logstash-config-file-to-parse-nested-json/281528/3 "2021-09-13T11:26:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.