How to indexing realtime data in to Elasticsearch?

Elastic Stack Elasticsearch
1

hello,
i am new Elasticsearch user, i would like to know how i can indexing realtime data from webSocket API and index this data, using Javascript.

even some link page to start, that will be helpful.

thank you.

2

you would need a component that receives data on a websocket and creates elasticsearch bulk requests (that are sent via HTTP) out of that stream of data. Elasticsearch itself cannot communicate via websockets, just regular HTTP.

Logstash features a websocket input that might fit your needs.

1 Like
3

Thanks Alexander,

i managed to work with websocket input, and run it in my machine. now i want to consume data from https://www.blockchain.com/api/api_websocket, which requires subscribe to channel . in order to insert the message data into an Elasticsearch container using Javascript.

when i use Javascript to consume the data, it work perfec my code below.

    <script>
            var ws = new WebSocket(" wss://ws.blockchain.info/inv");
            ws.onopen = function(){
                ws.send(JSON.stringify({"op":"unconfirmed_sub"}))
                console.log('mash')
            };
            ws.onmessage = function(msg){
                var response = JSON.parse(msg.data);
                console.log('response')
            }
        </script>

However, when i implement in Websocket input plugin, and below my Websocket input plugin config code.

input {
    websocket {
      mode => "client"
      url => "wss://ws.blockchain.info/inv"
      type => "string"
    }
  }

  output {
    elasticsearch {
      hosts => [ "localhost:9200" ]
      index => "calidog"
    }
    stdout { codec => rubydebug }
  }

when i run the config file, does't show anything.

[2019-04-13T18:15:16,387][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2019-04-13T18:15:21,190][INFO ][logstash.runner          ] Logstash shut down.

Thank you .

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.