Logstash :Config file to parse nested json

[ippo]

hello i 'm new to elk and i seeking help for my mission. i want to export the data from the log of json file below so that i have visualizations related to the number of successful builds . im trying to remove the fields that i dont need for this . you will find below my configuration file . when i run it i m blocked on this line

Also im not sure if this line is correct or i need to remove the fields through looping all the builds for each job

 remove_field => ["url","color","[builds][url]","[builds][details][artifacts]","[builds][details][building]","[builds][details][displayName]","[builds][details][estimatedDuration]","[builds][details][executor]" ]

the json log:

[{
    "name": "ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION",
    "url": "http:///ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/",
    "color": "blue",
    "builds": [{
        "number": 49,
        "url": "http://""""/ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/49/",
        "details": {
            "artifacts": [],
            "building": false,
            "displayName": "#49",
            "duration": 30836522,
            "estimatedDuration": 30436797,
            "executor": null,
            "fullDisplayName": "ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION #49",
            "id": "49",
            "keepLog": false,
            "number": 49,
            "queueId": 1182034,
            "result": "SUCCESS",
            "timestamp": 1628477169237,
            "url": "http:///ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/49/",
            "builtOn": "",
            "culprits": []
        }
    }, {
        "number": 48,
        "url": "http:///ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/48/",
        "details": {
            "artifacts": [],
            "building": false,
            "displayName": "#48",
            "duration": 29625208,
            "estimatedDuration": 30436797,
            "executor": null,
            "fullDisplayName": "ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION #48",
            "id": "48",
            "keepLog": false,
            "number": 48,
            "queueId": 1146447,
            "result": "SUCCESS",
            "timestamp": 1627872360241,
            "url": "http://""""""/ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/48/",
            "builtOn": "",
            "culprits": []
        }
    }, {
        "number": 47,
        "url": "http://""""/ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/47/",
        "details": {
            "artifacts": [],
            "building": false,
            "displayName": "#47",
            "duration": 30848660,
            "estimatedDuration": 30436797,
            "executor": null,
            "fullDisplayName": "ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION #47",
            "id": "47",
            "keepLog": false,
            "number": 47,
            "queueId": 1107340,
            "result": "SUCCESS",
            "timestamp": 1627267567286,
            "url": "http://""""/ARCHIVAGE_EUROSTADE-JENKINS_CONFIGURATION/47/",
            "builtOn": "",
            "culprits": []
        }
    }]
}]

config file

input {
  file {
    
    path => "C:/Users/SOAR07211/Desktop/demo_jobs_ordonnanceur3.json"
	start_position => "beginning"
	codec => json
    
	}
  }


filter {
mutate{ 
        remove_field => ["url","color","[builds][url]","[builds][details][artifacts]","[builds][details][building]","[builds][details][displayName]","[builds][details][estimatedDuration]","[builds][details][executor]" ]
	}
json {
         source => "message" 

    
    }
	}
  
 

output {
   elasticsearch {
   hosts => "localhost:9200"
   index => "index_demo5_fichier_json"  
  }
  
}

any help is welcome

[aaron-nimocks]

When reading a file Logstash tracks what has been read or not in order to only pull in new data using sincedb_path setting. Add this to your input and give it a shot.

sincedb_path => '/dev/null'

What I think is going on is this file has already been read and some point and Logstash won't read those records again.