Hello,
Can I somehow feed Elasticsearch with the json file my web service produce? Like : nameofserver/api/thejsonfile ?
Or do I have to download it and feed it manually?
Hello,
Can I somehow feed Elasticsearch with the json file my web service produce? Like : nameofserver/api/thejsonfile ?
Or do I have to download it and feed it manually?
You cannot do it directly, no.
Something like Logstash will do it, of you can do a manual curl download and then upload to ES.
OK. Thank you. Will have to look into Logstash then.
Are there any spesific formatting I need to use on the JSON file? When I download the JSON file locally and try to feed it to ES I seem to do something wrong.
If I use : curl XPOST localhost:9200/server/server/_bulk?pretty --databinary @Server.json I get the no request added error.
If I use: curl XPOST localhost:9200/server/server?pretty --databinary @Server.json it indexes it only as one object.
Here is a sample of my Server.json file, some info is changed for security reasons:
{
"machineID":1,
"isVirtualMachine":true,
"name":"Server1",
"lastActivity":"2015-06-27T06:47:03",
"domain":"NO",
"deviceOS":"Microsoft Windows Server 2012 Standard",
"lastBoot":"2015-06-11T03:11:31",
"patchePlan":"Auto",
"owner":1
},
{
"machineID":2,
"isVirtualMachine":true,
"name":"Server2",
"lastActivity":"2015-06-27T06:50:57",
"domain":"NO",
"deviceOS":"Microsoft Windows Server 2008 R2 Standard",
"lastBoot":"2015-06-10T22:14:27",
"patchePlan":"Auto",
"owner":1
},
{
"machineID":3,
"isVirtualMachine":false,
"name":"Server3",
"lastActivity":"2015-06-27T06:47:25",
"domain":"NO",
"deviceOS":"Microsoft Windows Server 2012 R2 Standard",
"lastBoot":"2015-06-11T04:11:37",
"patchePlan":"Auto",
"owner":1
},
{
"machineID":4,
"isVirtualMachine":true,
"name":"Server4",
"lastActivity":"2015-06-27T07:01:51",
"domain":"NO",
"deviceOS":"Microsoft Windows Server 2012 R2 Standard",
"lastBoot":"2015-06-11T03:12:31",
"patchePlan":"Auto",
"owner":1
},
{
"machineID":5,
"isVirtualMachine":true,
"name":"Server5",
"lastActivity":"2015-06-27T06:55:27",
"domain":"NO",
"deviceOS":"Microsoft Windows Server 2008 R2 Enterprise",
"lastBoot":"2015-06-11T17:43:17",
"patchePlan":"Manual",
"owner":1
}
Is this format in JSON wrong? If so, how should it be?
If you use bulk you need to respect bulk format:
Header
Payload
Header
Payload
..,
See https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html
So basically the same format as in the accounts.json file in the guide?
{"index":{"_id":"1"}}
{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"amberduke@pyrami.com","city":"Brogan","state":"IL"}
Is this something we have to do manually, or do we have a script/formatter to automatic format json-files correctly with Header, Payload, Header, Payload?
Well it depends on which client you are using.
In Java, you have some classes for example to simplify all that.
As @warkolm suggested, you could look at logstash. May be this plugin might help you: https://github.com/logstash-plugins/logstash-input-http-poller
Yeah, seems like that´s the way to go.
Thank you.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.