I am attempting to pull data from CiscoAMP, I am able to connect and grab the data with http_poller, but I am having a hard time understanding how to break out the subfields.
Products:
Logstash (5.2.0)
Elastic Cloud
All the data I want gets put into a single field called 'data'. I would like to break this field out further into other fields. The raw json looks like this:
{
"version": "v1.1.0",
"metadata": {
"links": {
"self": "https://api.amp.cisco.com/v1/events?limit=2",
"next": "https://api.amp.cisco.com/v1/events?limit=2\u0026offset=2"
},
"results": {
"total": 19127,
"current_item_count": 2,
"index": 0,
"items_per_page": 2
}
},
"data": [
{
"id": 1489425771450000746,
"timestamp": 1489425771,
"timestamp_nanoseconds": 450000000,
"date": "2017-03-13T17:22:51+00:00",
"event_type": "Vulnerable Application Detected",
"event_type_id": 1107296279,
"group_guids": [
"5caccfca-ac7f-42dc-b39c-9d1f3f717676"
],
"computer": {
"connector_guid": "35d760f2-0dca-4cee-811f-5fafd7ac1b94",
"hostname": "GSG-01636.gsg.grantstreet.com",
"active": true,
"links": {
"computer": "https://api.amp.cisco.com/v1/computers/35d760f2-0dca-4cee-811f-5fafd7ac1b94",
"trajectory": "https://api.amp.cisco.com/v1/computers/35d760f2-0dca-4cee-811f-5fafd7ac1b94/trajectory",
"group": "https://api.amp.cisco.com/v1/groups/5caccfca-ac7f-42dc-b39c-9d1f3f717676"
}
},
"file": {
"disposition": "Clean",
"identity": {
"sha256": "c8bf1abdc9ede0264ed7a818f61bb84ba2d42f160fdea45de6ed6ef816a6425e"
},
"file_name": "chrome.exe"
},
"vulnerabilities": [
{
"name": "Google Chrome",
"version": "55.0.2883.87",
"cve": "CVE-2017-5019",
"score": 6.8,
"url": "https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2017-5019"
},
{
"cve": "CVE-2017-5012",
"score": 6.8,
"url": "https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2017-5012"
},
{
"cve": "CVE-2017-5014",
"score": 6.8,
"url": "https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2017-5014"
},
{
"cve": "CVE-2017-5009",
"score": 6.8,
"url": "https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2017-5009"
}
]
},
{
"id": 6396978716487974919,
"timestamp": 1489412608,
"timestamp_nanoseconds": 325273163,
"date": "2017-03-13T13:43:28+00:00",
"event_type": "Policy Update Failure",
"event_type_id": 2164260866,
"group_guids": [
"5caccfca-ac7f-42dc-b39c-9d1f3f717676"
],
"error": {
"error_code": 3238330375,
"description": "Cannot connect to server"
},
"computer": {
"connector_guid": "b7acf51a-282d-4e20-848c-d2d2a9d216aa",
"hostname": "soemthing.hostname",
"active": true,
"links": {
"computer": "https://api.amp.cisco.com/v1/computers/b7acf51a-28",
"trajectory": "https://api.amp.cisco.com/v1/computers/b7acf51a-28/trajectory",
"group": "https://api.amp.cisco.com/v1/groups/5caccfca-a"
}
}
}
]
}
Input and Filter:
input {
http_poller {
type => ciscoamp
urls => {
ciscoampurl => {
method => get
url => "https://api.amp.cisco.com/v1/events?limit=1"
headers => {
Accept => "application/json"
Authorization => "Basic [redacted]"
}
}
}
request_timeout => 60
schedule => { cron => "* * * * * UTC"}
codec => "json" #tried both with and without this
metadata_target => "http_poller_metadata"
}
}
filter {
if [type] == "ciscoamp" {
json {
source => "message"
}
geoip { source => "dst_ip" }
geoip { source => "src_ip" }
}
}