# HTTP\_POLLER Plugin for Logstash **URL:** \ **Category:** Logstash\ **Created:** [December 6, 2017, 10:43am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498 "2017-12-06T10:43:39Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 6, 2017, 10:43am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/1 "2017-12-06T10:43:39Z")
Hi there, I'm new at using ELK and I have to import data with HTTP\_POLLER (json statement). I don't really understand some stuff about input plugins for Logstash. My question is: Do I have to download the plugin to use it or is it already a part of Logstash components? If so, where can I find the download file? Thank's in advance for your help. Rym ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 10, 2017, 9:09pm UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/2 "2017-12-10T21:09:37Z")
> I'm new at using ELk and after importing data from a database with JDBC (SQL statement) I'm having a new goal: importing data with HTTP (json statement). Okay, but be aware that there are two HTTP plugins; http and http\_poller. It's not clear from your question if you want Logstash to make the HTTP requests or if some other program will make the requests to Logstash. > Do I have to download the plugin to use it or is it already a part of Logstash components? It depends. Use the `logstash-plugin` command to list the installed plugins. > If so, where can I find the download file? Install plugins with the `logstash-plugin` command. > **[Working with plugins | Logstash Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html)** ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 11, 2017, 9:41am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/3 "2017-12-11T09:41:39Z")
Hello @magnusbaeck, Thank's for these questions, it helps going ahead on the right way. I'm actually using HTTP POLLER. I've understood that there's no need to download any plugin as far as I'm using Logstash plugins listed on this page: [https://www.elastic.co/guide/en/logstash/current/input-plugins.html](https://www.elastic.co/guide/en/logstash/current/input-plugins.html) I'm trying to import a JSON document from an URL. The problem I'm facing is that I want to split my big JSON document into several parts. I'm using filters, however it doesn't work. The Logstash config file looks like this: ```auto input { http_poller { urls => { test => { method => get user => "me" password => "mdp" url => "my-url" headers => { Accept => "application/json" } } } request_timeout => 300 schedule => { cron => "*/02 * * * * UTC"} codec => "json" metadata_target => "http_poller_metadata" } } filter { mutate { convert => "_source" => "string" } split { _source = "results" } } output { elasticsearch { hosts => ["localhost:9200"] index => "json_10meilleurs_logs" user => "elastic" password => "changeme" } } ``` Before that, I'm creating a new index with a bigger size than the default one by putting this command in Kibana: ```auto DELETE json_10meilleurs_logs PUT json_10meilleurs_logs/ { "settings":{ "index.mapping.total_fields.limit": 50000 } } ``` Although the import was working and importing a single big document, now it doesn't work anymore. When I launch Logstash with the config file, it gives this result: ```auto Cannot create pipeline {:reason=>"Expected one of #, {, \", ', } at line 22, column 32 (byte 543) after filter {\n\tmutate { convert => \"_source\" "} ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 11, 2017, 11:06am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/4 "2017-12-11T11:06:35Z")
> I've understood that there's no need to download any plugin as far as I'm using Logstash plugins listed on this page: [https://www.elastic.co/guide/en/logstash/current/input-plugins.html](https://www.elastic.co/guide/en/logstash/current/input-plugins.html) All those plugins aren't installed by default. > ``` > mutate { convert => "_source" => "string" } > > ``` You're missing a pair of braces. Pay close attention to the example in the documentation: [Mutate filter plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-convert) > ``` > split { _source = "results" } > > ``` The split filter doesn't have a `_source` option. You probably mean `split { field => "results" }`. ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 12, 2017, 8:59am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/5 "2017-12-12T08:59:51Z")
Hi @magnusbaeck, the field that I want to split is "\_source" so I put: ```auto filter { mutate { convert => {"source" => "string" }} split { field = "_source" } } ``` But I still have the same error: ```auto Only String and Array types are splittable. field:_source is of type = NilClass ``` Can we convert the attribut "\_source" and if so, how can we do it? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 12, 2017, 9:36am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/6 "2017-12-12T09:36:37Z")
I don't think you have a `_source` field and the error message you get supports my hunch. What does an example event look like? Use a `stdout { codec => rubydebug }` output to get a raw dump. ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 12, 2017, 10:22am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/7 "2017-12-12T10:22:58Z")
Here's what an event lokks like: We have 96 different users. Each user have 6 events. Decreasing from 6 to 1. Each event looks like this one: ```auto "user": { "6": { "timespent": 0, "timespentinterne": 0, "percentIntern": 0.0, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } }, ``` I'm trying at first to import the document to Elasticsearch. (succeeded once, but doesn't want to load anymore) Then, when it's gonna work, if it works, I'd like to import it in 96 different documents in order to have 1 document for each user. It's been 10 days I'm trying to do this, but still beeing stuck in here... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 12, 2017, 10:45am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/8 "2017-12-12T10:45:21Z")
> Here's what an event lokks like: Right, and there's no `_source` field there. There's also no array field so I don't see how a split field would be useful. ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 12, 2017, 10:54am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/9 "2017-12-12T10:54:04Z")
Then can I transform the "user" into an array to split it? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 12, 2017, 10:56am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/10 "2017-12-12T10:56:26Z")
How do you want to split the event you have? What do you want to end up with in the end? ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 12, 2017, 11:03am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/11 "2017-12-12T11:03:55Z")
**What I have now:** URL with a JSON document : =\>96 users ==\> each user has 6 weeks logs. ===\> each log has this information: timespent, timespentintern, percentIntern, worklogs **What I'll need to have at the end:** a matrix: =\> raws: the 96 users =\> columns: weeks (so 6 columns) =\> for each user and each week: total timespentintern ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 12, 2017, 1:18pm UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/12 "2017-12-12T13:18:42Z")
Okay. So what does a complete input document look like? What you posted earlier wasn't the full document, was it? ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 12, 2017, 3:18pm UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/13 "2017-12-12T15:18:50Z")
no it wasn't. It's way too long let me just post an event for one user for one week: ```auto "6": { "timespent": 0, "timespentinterne": 0, "percentIntern": 0.0, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } }, "5": { "timespent": 32.0833333332, "timespentinterne": 8.0833333333, "percentIntern": 0.2519480519, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { "264065": { "timespent": 0.3333333333, "commentairewl": "pb de clavier bloqu\u00e9", "authorKey": "abogojevic", "issueKey": "KFIT-6", "datewl": "5", "dateReadablewl": "29-10-2017" } }, "TimespentOnIssue": 0.3333333333 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { "262956": { "timespent": 0.5, "commentairewl": "r\u00e9union lundi matin", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "262983": { "timespent": 0.25, "commentairewl": "mails", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "262997": { "timespent": 1, "commentairewl": "backlog du support - infos \u00e0 St\u00e9phane", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "263088": { "timespent": 0.25, "commentairewl": "point Steph sur backlog support", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "263305": { "timespent": 1, "commentairewl": "RUN : CR atelier RUN du 23/11", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "27-10-2017" }, "263545": { "timespent": 1, "commentairewl": "tutorat point Maeva", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "28-10-2017" }, "263826": { "timespent": 0.25, "commentairewl": "RUN : point Steph sur RAF", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "29-10-2017" }, "263978": { "timespent": 0.25, "commentairewl": "DECOMPTES : point inactivit\u00e9 comptes CHRIS + envoi de son c\u00f4t\u00e9 \u00e0 d\u00e9planifier", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "29-10-2017" }, "264358": { "timespent": 3, "commentairewl": "RUN : atelier Dashboard + temps de trajet Arche", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "30-10-2017" }, "264452": { "timespent": 0.25, "commentairewl": "SUPPORT : r\u00e9union du support", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "5", "dateReadablewl": "30-10-2017" } }, "TimespentOnIssue": 7.75 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } }, "4": { "timespent": 39.1833333331, "timespentinterne": 10.5000000000, "percentIntern": 0.2679710761, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { "260689": { "timespent": 0.5, "commentairewl": "r\u00e9union lundi matin", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "20-10-2017" }, "260794": { "timespent": 0.3333333333, "commentairewl": "backlog support: urgences \u00e0 traiter mail \u00e0 St\u00e9phane", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "20-10-2017" }, "260836": { "timespent": 0.4166666667, "commentairewl": "suite backlog support : CRENEWS avec dispos C\u00e9cile selon staffing Romain", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "20-10-2017" }, "260853": { "timespent": 0.4166666667, "commentairewl": "fin des prios support pour la semaine", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "20-10-2017" }, "261623": { "timespent": 0.25, "commentairewl": "point Steph sur RAF pour atelier RUN", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261669": { "timespent": 1.5833333333, "commentairewl": "r\u00e9union op\u00e9rationnelle globale men\u00e9e par Niamh", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261707": { "timespent": 0.25, "commentairewl": "RSE : point don du sang avec LaD", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261720": { "timespent": 2.25, "commentairewl": "RUN : pr\u00e9sentation package", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261723": { "timespent": 0.3333333333, "commentairewl": "RUN: point steph prez atelier", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "23-10-2017" }, "261724": { "timespent": 2.5, "commentairewl": "RUN : atelier \u00e9quipe - pr\u00e9sentation des packages", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "23-10-2017" }, "261956": { "timespent": 0.75, "commentairewl": "pr\u00e9sentation atelier RUN : slides Steph - lecture / corrections", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "22-10-2017" }, "261973": { "timespent": 0.6666666667, "commentairewl": "suite prez Steph modifs", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "4", "dateReadablewl": "23-10-2017" } }, "TimespentOnIssue": 10.2500000000 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { "262207": { "timespent": 0.25, "commentairewl": "JIRA : filtres/dashboard avec Florent", "authorKey": "abogojevic", "issueKey": "KFIT-1", "datewl": "4", "dateReadablewl": "23-10-2017" } }, "TimespentOnIssue": 0.25 } } }, ``` ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 12, 2017, 3:19pm UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/14 "2017-12-12T15:19:56Z")
```auto "3": { "timespent": 37.8333333334, "timespentinterne": 14.9166666667, "percentIntern": 0.3942731278, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { "260090": { "timespent": 0.1666666667, "commentairewl": "pb de connexion internet", "authorKey": "abogojevic", "issueKey": "KFIT-6", "datewl": "3", "dateReadablewl": "17-10-2017" } }, "TimespentOnIssue": 0.1666666667 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { "258533": { "timespent": 1.5, "commentairewl": "RUN : suite package", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "13-10-2017" }, "258668": { "timespent": 1.5, "commentairewl": "RUN : RDV Meriem pour DATA + suite package (GREG/ALEX/BRUNO/SIHAM)\n", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "13-10-2017" }, "258669": { "timespent": 1.5, "commentairewl": "RUN : point Meriem pour data offre RUN (objectifs / prestas)", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "13-10-2017" }, "258670": { "timespent": 0.6666666667, "commentairewl": "RUN : prestas RUN suite entretien Meriem", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "13-10-2017" }, "258673": { "timespent": 1.5833333333, "commentairewl": "RUN DECOMPTES : agenda des envois sur Google Agenda pour planification des envois", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "14-10-2017" }, "259398": { "timespent": 3.75, "commentairewl": "RUN : point Steph sur fichier des prestations du package + infos supp \u00e0 Manu pour la proposition de valeur (naming) + avanc\u00e9e sur la presta data + revision de l'onglet OFFRE", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "15-10-2017" }, "259495": { "timespent": 1.25, "commentairewl": "RUN d\u00e9comptes : suite des prios avec planification automatique dans Google Agenda + \u00e9changes Steph RUN (commentaires)", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "15-10-2017" }, "259600": { "timespent": 1.3333333333, "commentairewl": "r\u00e9union mensuelle AGILE", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "15-10-2017" }, "260133": { "timespent": 1, "commentairewl": "RUN : point INFRA story mapping sur les prestations possibles", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "17-10-2017" }, "260211": { "timespent": 0.25, "commentairewl": "RUN : point steph sur modules \u00e0 pr\u00e9senter plut\u00f4t que prestas", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "3", "dateReadablewl": "17-10-2017" } }, "TimespentOnIssue": 14.3333333333 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { "259635": { "timespent": 0.4166666667, "commentairewl": "aide de Christophe sur des graphiques Google Drive pour projet sESAN (\u00e9volutions Google - bugs)", "authorKey": "abogojevic", "issueKey": "KFIT-1", "datewl": "3", "dateReadablewl": "15-10-2017" } }, "TimespentOnIssue": 0.4166666667 } } }, ``` ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 12, 2017, 3:20pm UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/15 "2017-12-12T15:20:03Z")
```auto "2": { "timespent": 39.2500000000, "timespentinterne": 19.3333333333, "percentIntern": 0.4925690021, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { "256584": { "timespent": 0.5, "commentairewl": "r\u00e9union du lundi matin", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "256599": { "timespent": 0.3833333333, "commentairewl": "lecture mails post cong\u00e9s", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "256712": { "timespent": 4.5333333333, "commentairewl": "travail fichier RUN - offre - package\n", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "256759": { "timespent": 0.75, "commentairewl": "RUN : suite travail sur package", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "256874": { "timespent": 0.75, "commentairewl": "RUN : suite fichier offre\n", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "6-10-2017" }, "257098": { "timespent": 1, "commentairewl": "RUN : Vanessa, point fichiers INRIA / IGN / MANBOU / SESAN", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "7-10-2017" }, "257100": { "timespent": 0.75, "commentairewl": "\u00b5RUN : check list des envois de Maeva vs Dashboard des envois \u00e0 jour + questions par mail", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "7-10-2017" }, "257491": { "timespent": 1, "commentairewl": "point St\u00e9phane : reprise dec omptes PRIMAXIA / CASDEN / CF NEWS / SESAN / MUTAC", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "8-10-2017" }, "257711": { "timespent": 0.75, "commentairewl": "RUN : d\u00e9comptes : echanges Maeva sur fichiers envoy\u00e9s + MAJ de mon Dashboard + d\u00e9but planif des envois du mois", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "8-10-2017" }, "257893": { "timespent": 0.5, "commentairewl": "RUN : point steph sur avanc\u00e9e du fichier avec vision package => offre 1 ou 2 + catalogue des experts", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "9-10-2017" }, "257897": { "timespent": 0.25, "commentairewl": "point NiP infos manger bouger / CRE blog / CRE NL", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "9-10-2017" }, "257899": { "timespent": 1.5, "commentairewl": "SUPPORT : prios pour backlog + NOUVEAUX COMPTES : mise en place des filtres des nouveaux projets", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "9-10-2017" }, "257913": { "timespent": 0.75, "commentairewl": "RUN : fichier des prestations pour le package", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "9-10-2017" }, "258034": { "timespent": 0.6666666667, "commentairewl": "RUN : suite onglet des prestations", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "10-10-2017" }, "258175": { "timespent": 2.5, "commentairewl": "RUN : atelier de naming", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "10-10-2017" }, "258286": { "timespent": 1.25, "commentairewl": "RUN : suivi des relances commerciales pour les comptes date de validit\u00e9 termin\u00e9e => appel de Leila pour \u00e9claircissements + rev\u00e9rification des dates de fin de validit\u00e9 avec messages Vanessa et infos manquantes sur fichier partag\u00e9", "authorKey": "abogojevic", "issueKey": "KFIT-4", "datewl": "2", "dateReadablewl": "10-10-2017" } }, "TimespentOnIssue": 17.8333333333 }, "KFIT-3": { "worklogsdetails": { "257102": { "timespent": 1.5, "commentairewl": "visite des locaux ARCHE avec \u00e9quipe support", "authorKey": "abogojevic", "issueKey": "KFIT-3", "datewl": "2", "dateReadablewl": "7-10-2017" } }, "TimespentOnIssue": 1.5 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } }, "1": { "timespent": 24, "timespentinterne": 0, "percentIntern": 0.0, "worklogs": { "KFIT-14": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-13": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-7": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-6": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-5": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-4": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-3": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-2": { "worklogsdetails": { }, "TimespentOnIssue": 0 }, "KFIT-1": { "worklogsdetails": { }, "TimespentOnIssue": 0 } } }, "key": "abogojevic", "workload": 35 }, ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 12, 2017, 3:23pm UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/16 "2017-12-12T15:23:23Z")
Okay, but what's the structure of the document? Judging by what you wrote earlier it seems to look similar to this, yes? ```nohighlight { "users": { "user1": { "6": { ... }, "5": { ... }, ... "1": { ... }, }, "user2": { "6": { ... }, "5": { ... }, ... "1": { ... }, }, ... "user2": { "6": { ... }, "5": { ... }, ... "1": { ... }, }, } } ``` ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 13, 2017, 10:21am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/17 "2017-12-13T10:21:14Z")
exactly. I couldn't post everything in the same answer (too long) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 15, 2017, 6:50am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/18 "2017-12-15T06:50:04Z")
Okay. You need to use a ruby filter to transform the hash in the `users` field into an array that you can split into separate documents. This is the structure you're looking for: ```nohighlight { "users": [ { "name": "user1", "weeks": { "6": { ... }, "5": { ... }, ... "1": { ... }, }, } ... ] } ``` Perhaps something like this will work: ```nohighlight userlist = [] event.get('users').each { |k, v| userlist << {"name" => k, "weeks" => v} } event.set('users', userlist) ``` ---
**Author:** ![Rym\_Guerbi\_Michaut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rym_guerbi_michaut/32/25496_2.png) [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Post date:** [December 15, 2017, 8:43am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/19 "2017-12-15T08:43:19Z")
Thank's for the code, but last question: where do I put it? I'm using a config file for Logstash and Kibana ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 15, 2017, 10:31am UTC](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498/20 "2017-12-15T10:31:27Z")
Put it in the `code` section of a ruby filter. [Next page](https://discuss.elastic.co/t/http-poller-plugin-for-logstash/110498.md?page=2)