Hi there,
I'm new at using ELK and I have to import data with HTTP_POLLER (json statement).
I don't really understand some stuff about input plugins for Logstash.
My question is:
Do I have to download the plugin to use it or is it already a part of Logstash components? If so, where can I find the download file?
Thank's in advance for your help.
Rym
I'm new at using ELk and after importing data from a database with JDBC (SQL statement) I'm having a new goal: importing data with HTTP (json statement).
Okay, but be aware that there are two HTTP plugins; http and http_poller. It's not clear from your question if you want Logstash to make the HTTP requests or if some other program will make the requests to Logstash.
Do I have to download the plugin to use it or is it already a part of Logstash components?
It depends. Use the logstash-plugin command to list the installed plugins.
If so, where can I find the download file?
Install plugins with the logstash-plugin command.
Hello @magnusbaeck,
Thank's for these questions, it helps going ahead on the right way.
I'm actually using HTTP POLLER.
I've understood that there's no need to download any plugin as far as I'm using Logstash plugins listed on this page: https://www.elastic.co/guide/en/logstash/current/input-plugins.html
I'm trying to import a JSON document from an URL.
The problem I'm facing is that I want to split my big JSON document into several parts. I'm using filters, however it doesn't work.
The Logstash config file looks like this:
input {
http_poller {
urls => {
test => {
method => get
user => "me"
password => "mdp"
url => "my-url"
headers => {
Accept => "application/json"
}
}
}
request_timeout => 300
schedule => { cron => "*/02 * * * * UTC"}
codec => "json"
metadata_target => "http_poller_metadata"
}
}
filter {
mutate { convert => "_source" => "string" }
split { _source = "results" }
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "json_10meilleurs_logs"
user => "elastic"
password => "changeme"
}
}
Before that, I'm creating a new index with a bigger size than the default one by putting this command in Kibana:
DELETE json_10meilleurs_logs
PUT json_10meilleurs_logs/
{
"settings":{
"index.mapping.total_fields.limit":
50000
}
}
Although the import was working and importing a single big document, now it doesn't work anymore.
When I launch Logstash with the config file, it gives this result:
Cannot create pipeline {:reason=>"Expected one of #, {, \", ', } at line 22, column 32 (byte 543) after filter {\n\tmutate { convert => \"_source\" "}
I've understood that there's no need to download any plugin as far as I'm using Logstash plugins listed on this page: https://www.elastic.co/guide/en/logstash/current/input-plugins.html
All those plugins aren't installed by default.
mutate { convert => "_source" => "string" }
You're missing a pair of braces. Pay close attention to the example in the documentation: Mutate filter plugin | Logstash Reference [8.11] | Elastic
split { _source = "results" }
The split filter doesn't have a _source option. You probably mean split { field => "results" }.
Hi @magnusbaeck,
the field that I want to split is "_source"
so I put:
filter {
mutate { convert => {"source" => "string" }}
split { field = "_source" }
}
But I still have the same error:
Only String and Array types are splittable. field:_source is of type = NilClass
Can we convert the attribut "_source" and if so, how can we do it?
I don't think you have a _source field and the error message you get supports my hunch. What does an example event look like? Use a stdout { codec => rubydebug } output to get a raw dump.
Here's what an event lokks like:
We have 96 different users.
Each user have 6 events. Decreasing from 6 to 1.
Each event looks like this one:
"user": {
"6": {
"timespent": 0,
"timespentinterne": 0,
"percentIntern": 0.0,
"worklogs": {
"KFIT-14": {
"worklogsdetails": { },
"TimespentOnIssue": 0 },
"KFIT-13": {
"worklogsdetails": { },
"TimespentOnIssue": 0
},
"KFIT-7": {
"worklogsdetails": { },
"TimespentOnIssue": 0 },
"KFIT-6": {
"worklogsdetails": { },
"TimespentOnIssue": 0
},
"KFIT-5": {
"worklogsdetails": { },
"TimespentOnIssue": 0
},
"KFIT-4": {
"worklogsdetails": { },
"TimespentOnIssue": 0
},
"KFIT-3": {
"worklogsdetails": { },
"TimespentOnIssue": 0
},
"KFIT-2": {
"worklogsdetails": { },
"TimespentOnIssue": 0
},
"KFIT-1": {
"worklogsdetails": { },
"TimespentOnIssue": 0
}
}
},
I'm trying at first to import the document to Elasticsearch. (succeeded once, but doesn't want to load anymore)
Then, when it's gonna work, if it works, I'd like to import it in 96 different documents in order to have 1 document for each user.
It's been 10 days I'm trying to do this, but still beeing stuck in here...
Here's what an event lokks like:
Right, and there's no _source field there. There's also no array field so I don't see how a split field would be useful.
Then can I transform the "user" into an array to split it?
How do you want to split the event you have? What do you want to end up with in the end?
What I have now:
URL with a JSON document :
=>96 users
==> each user has 6 weeks logs.
===> each log has this information: timespent, timespentintern, percentIntern, worklogs
What I'll need to have at the end:
a matrix:
=> raws: the 96 users
=> columns: weeks (so 6 columns)
=> for each user and each week: total timespentintern
Okay. So what does a complete input document look like? What you posted earlier wasn't the full document, was it?
no it wasn't.
It's way too long
let me just post an event for one user for one week:
"6": { "timespent": 0, "timespentinterne": 0, "percentIntern": 0.0, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } },
"5": { "timespent": 32.0833333332, "timespentinterne": 8.0833333333, "percentIntern": 0.2519480519, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { "264065": { "timespent": 0.3333333333, "commentairewl": "pb de clavier bloqu\u00e9", "authorKey": "abogojevic", "issueKey": "KFIT-6", "datewl": "5", "dateReadablewl": "29-10-2017" } }, "TimespentOnIssue": 0.3333333333 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { "262956": { "timespent": 0.5, "commentairewl": "r\u00e9union lundi matin", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "262983": { "timespent": 0.25, "commentairewl": "mails", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "262997": { "timespent": 1, "commentairewl": "backlog du support - infos \u00e0 St\u00e9phane", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "263088": { "timespent": 0.25, "commentairewl": "point Steph sur backlog support", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "263305": { "timespent": 1, "commentairewl": "RUN : CR atelier RUN du 23/11", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "263545": { "timespent": 1, "commentairewl": "tutorat point Maeva", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "28-10-2017" }, "263826": { "timespent": 0.25, "commentairewl": "RUN : point Steph sur RAF", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "29-10-2017" }, "263978": { "timespent": 0.25, "commentairewl": "DECOMPTES : point inactivit\u00e9 comptes CHRIS + envoi de son c\u00f4t\u00e9 \u00e0 d\u00e9planifier", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "29-10-2017" }, "264358": { "timespent": 3, "commentairewl": "RUN : atelier Dashboard + temps de trajet Arche", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "30-10-2017" }, "264452": { "timespent": 0.25, "commentairewl": "SUPPORT : r\u00e9union du support", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "30-10-2017" } }, "TimespentOnIssue": 7.75 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } },
"4": { "timespent": 39.1833333331, "timespentinterne": 10.5000000000, "percentIntern": 0.2679710761, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { "260689": { "timespent": 0.5, "commentairewl": "r\u00e9union lundi matin", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "20-10-2017" }, "260794": { "timespent": 0.3333333333, "commentairewl": "backlog support: urgences \u00e0 traiter mail \u00e0 St\u00e9phane", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "20-10-2017" }, "260836": { "timespent": 0.4166666667, "commentairewl": "suite backlog support : CRENEWS avec dispos C\u00e9cile selon staffing Romain", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "20-10-2017" }, "260853": { "timespent": 0.4166666667, "commentairewl": "fin des prios support pour la semaine", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "20-10-2017" }, "261623": { "timespent": 0.25, "commentairewl": "point Steph sur RAF pour atelier RUN", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261669": { "timespent": 1.5833333333, "commentairewl": "r\u00e9union op\u00e9rationnelle globale men\u00e9e par Niamh", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261707": { "timespent": 0.25, "commentairewl": "RSE : point don du sang avec LaD", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261720": { "timespent": 2.25, "commentairewl": "RUN : pr\u00e9sentation package", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261723": { "timespent": 0.3333333333, "commentairewl": "RUN: point steph prez atelier", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "23-10-2017" }, "261724": { "timespent": 2.5, "commentairewl": "RUN : atelier \u00e9quipe - pr\u00e9sentation des packages", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "23-10-2017" }, "261956": { "timespent": 0.75, "commentairewl": "pr\u00e9sentation atelier RUN : slides Steph - lecture / corrections", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261973": { "timespent": 0.6666666667, "commentairewl": "suite prez Steph modifs", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "23-10-2017" } }, "TimespentOnIssue": 10.2500000000 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { "262207": { "timespent": 0.25, "commentairewl": "JIRA : filtres/dashboard avec Florent", "authorKey": "abogojevic", "issueKey": "KFIT-1", "datewl": "4", "dateReadablewl": "23-10-2017" } }, "TimespentOnIssue": 0.25 } } },
"3": { "timespent": 37.8333333334, "timespentinterne": 14.9166666667, "percentIntern": 0.3942731278, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { "260090": { "timespent": 0.1666666667, "commentairewl": "pb de connexion internet", "authorKey": "abogojevic", "issueKey": "KFIT-6", "datewl": "3", "dateReadablewl": "17-10-2017" } }, "TimespentOnIssue": 0.1666666667 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { "258533": { "timespent": 1.5, "commentairewl": "RUN : suite package", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "13-10-2017" }, "258668": { "timespent": 1.5, "commentairewl": "RUN : RDV Meriem pour DATA + suite package (GREG/ALEX/BRUNO/SIHAM)\n", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "13-10-2017" }, "258669": { "timespent": 1.5, "commentairewl": "RUN : point Meriem pour data offre RUN (objectifs / prestas)", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "13-10-2017" }, "258670": { "timespent": 0.6666666667, "commentairewl": "RUN : prestas RUN suite entretien Meriem", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "13-10-2017" }, "258673": { "timespent": 1.5833333333, "commentairewl": "RUN DECOMPTES : agenda des envois sur Google Agenda pour planification des envois", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "14-10-2017" }, "259398": { "timespent": 3.75, "commentairewl": "RUN : point Steph sur fichier des prestations du package + infos supp \u00e0 Manu pour la proposition de valeur (naming) + avanc\u00e9e sur la presta data + revision de l'onglet OFFRE", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "15-10-2017" }, "259495": { "timespent": 1.25, "commentairewl": "RUN d\u00e9comptes : suite des prios avec planification automatique dans Google Agenda + \u00e9changes Steph RUN (commentaires)", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "15-10-2017" }, "259600": { "timespent": 1.3333333333, "commentairewl": "r\u00e9union mensuelle AGILE", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "15-10-2017" }, "260133": { "timespent": 1, "commentairewl": "RUN : point INFRA story mapping sur les prestations possibles", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "17-10-2017" }, "260211": { "timespent": 0.25, "commentairewl": "RUN : point steph sur modules \u00e0 pr\u00e9senter plut\u00f4t que prestas", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "17-10-2017" } }, "TimespentOnIssue": 14.3333333333 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { "259635": { "timespent": 0.4166666667, "commentairewl": "aide de Christophe sur des graphiques Google Drive pour projet sESAN (\u00e9volutions Google - bugs)", "authorKey": "abogojevic", "issueKey": "KFIT-1", "datewl": "3", "dateReadablewl": "15-10-2017" } }, "TimespentOnIssue": 0.4166666667 } } },
"2": { "timespent": 39.2500000000, "timespentinterne": 19.3333333333, "percentIntern": 0.4925690021, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { "256584": { "timespent": 0.5, "commentairewl": "r\u00e9union du lundi matin", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "256599": { "timespent": 0.3833333333, "commentairewl": "lecture mails post cong\u00e9s", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "256712": { "timespent": 4.5333333333, "commentairewl": "travail fichier RUN - offre - package\n", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "256759": { "timespent": 0.75, "commentairewl": "RUN : suite travail sur package", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "256874": { "timespent": 0.75, "commentairewl": "RUN : suite fichier offre\n", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "257098": { "timespent": 1, "commentairewl": "RUN : Vanessa, point fichiers INRIA / IGN / MANBOU / SESAN", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "7-10-2017" }, "257100": { "timespent": 0.75, "commentairewl": "\u00b5RUN : check list des envois de Maeva vs Dashboard des envois \u00e0 jour + questions par mail", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "7-10-2017" }, "257491": { "timespent": 1, "commentairewl": "point St\u00e9phane : reprise dec omptes PRIMAXIA / CASDEN / CF NEWS / SESAN / MUTAC", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "8-10-2017" }, "257711": { "timespent": 0.75, "commentairewl": "RUN : d\u00e9comptes : echanges Maeva sur fichiers envoy\u00e9s + MAJ de mon Dashboard + d\u00e9but planif des envois du mois", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "8-10-2017" }, "257893": { "timespent": 0.5, "commentairewl": "RUN : point steph sur avanc\u00e9e du fichier avec vision package => offre 1 ou 2 + catalogue des experts", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "9-10-2017" }, "257897": { "timespent": 0.25, "commentairewl": "point NiP infos manger bouger / CRE blog / CRE NL", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "9-10-2017" }, "257899": { "timespent": 1.5, "commentairewl": "SUPPORT : prios pour backlog + NOUVEAUX COMPTES : mise en place des filtres des nouveaux projets", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "9-10-2017" }, "257913": { "timespent": 0.75, "commentairewl": "RUN : fichier des prestations pour le package", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "9-10-2017" }, "258034": { "timespent": 0.6666666667, "commentairewl": "RUN : suite onglet des prestations", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "10-10-2017" }, "258175": { "timespent": 2.5, "commentairewl": "RUN : atelier de naming", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "10-10-2017" }, "258286": { "timespent": 1.25, "commentairewl": "RUN : suivi des relances commerciales pour les comptes date de validit\u00e9 termin\u00e9e => appel de Leila pour \u00e9claircissements + rev\u00e9rification des dates de fin de validit\u00e9 avec messages Vanessa et infos manquantes sur fichier partag\u00e9", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "10-10-2017" } }, "TimespentOnIssue": 17.8333333333 }, "KFIT-3": { "worklogsdetails": { "257102": { "timespent": 1.5, "commentairewl": "visite des locaux ARCHE avec \u00e9quipe support", "authorKey": "abogojevic", "issueKey": "KFIT-3", "datewl": "2", "dateReadablewl": "7-10-2017" } }, "TimespentOnIssue": 1.5 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } },
"1": { "timespent": 24, "timespentinterne": 0, "percentIntern": 0.0, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } }, "key": "abogojevic", "workload": 35
},
Okay, but what's the structure of the document? Judging by what you wrote earlier it seems to look similar to this, yes?
{
"users": {
"user1": {
"6": { ... },
"5": { ... },
...
"1": { ... },
},
"user2": {
"6": { ... },
"5": { ... },
...
"1": { ... },
},
...
"user2": {
"6": { ... },
"5": { ... },
...
"1": { ... },
},
}
}
exactly.
I couldn't post everything in the same answer (too long)
Okay. You need to use a ruby filter to transform the hash in the users field into an array that you can split into separate documents. This is the structure you're looking for:
{
"users": [
{
"name": "user1",
"weeks": {
"6": { ... },
"5": { ... },
...
"1": { ... },
},
}
...
]
}
Perhaps something like this will work:
userlist = []
event.get('users').each { |k, v|
userlist << {"name" => k, "weeks" => v}
}
event.set('users', userlist)
Thank's for the code, but last question: where do I put it?
I'm using a config file for Logstash and Kibana
Put it in the code section of a ruby filter.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.