I get error when i try to parse json to elastic search


(ridwan syarifudin) #1

when i use the curl command to parse json to elastic
with this curl command:
while read f1
do curl -XPOST 'http://localhost:9200/master_product_info/message?pipeline=parse_master_produk_pipeline' -H "Content-Type: application/json" -d "{ "message": "$f1" }"
done < ~/Downloads/eksportmasterproduk.csv

and i get this error

curl: (7) Failed to connect to 127.0.0.1 port 8888: Connection refused

how can i fix this ? thank you


(David Pilato) #2

The log message doesn't seem to be related to your curl command. Not the same host, not the same port.
Is it a real call?


(ridwan syarifudin) #3

im sorry sir, i just did what the tutorial told me, i just did exactly the same, inserting it to elastic search with localhost:9200
but its never work, theres nothing in google to fix it, im so confused, im sorry sir


(David Pilato) #4

I don't know what tutorial you mean. But when doing a curl localhost:9200 I don't think curl can answer 127.0.0.1:8888 unless you have a proxy on your machine may be?

Sharing elasticsearch logs would help as well.


(ridwan syarifudin) #5
[2019-04-15T16:28:43,098][INFO ][o.e.e.NodeEnvironment    ] [_VVJsGe] using [1] data paths, mounts [[/home (/dev/sda4)]], net usable_space [273.7gb], net total_space [355.9gb], types [ext4]
[2019-04-15T16:28:43,102][INFO ][o.e.e.NodeEnvironment    ] [_VVJsGe] heap size [990.7mb], compressed ordinary object pointers [true]
[2019-04-15T16:28:43,103][INFO ][o.e.n.Node               ] [_VVJsGe] node name derived from node ID [_VVJsGeoTuC17J2iMOH4Bw]; set [node.name] to override
[2019-04-15T16:28:43,104][INFO ][o.e.n.Node               ] [_VVJsGe] version[6.7.1], pid[5589], build[default/tar/2f32220/2019-04-02T15:59:27.961366Z], OS[Linux/3.13.0-161-generic/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_92/25.92-b14]
[2019-04-15T16:28:43,104][INFO ][o.e.n.Node               ] [_VVJsGe] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-6328724218719287698, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/sts/Downloads/elasticsearch-6.7.1, -Des.path.conf=/home/sts/Downloads/elasticsearch-6.7.1/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]

[2019-04-15T16:28:45,983][INFO ][o.e.p.PluginsService     ] [_VVJsGe] loaded module [x-pack-upgrade]
[2019-04-15T16:28:45,983][INFO ][o.e.p.PluginsService     ] [_VVJsGe] loaded module [x-pack-watcher]
[2019-04-15T16:28:45,984][INFO ][o.e.p.PluginsService     ] [_VVJsGe] no plugins loaded
[2019-04-15T16:28:51,873][INFO ][o.e.x.s.a.s.FileRolesStore] [_VVJsGe] parsed [0] roles from file [/home/sts/Downloads/elasticsearch-6.7.1/config/roles.yml]
[2019-04-15T16:28:52,750][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [_VVJsGe] [controller/5663] [Main.cc@109] controller (64 bit): Version 6.7.1 (Build e1f492de67a719) Copyright (c) 2019 Elasticsearch BV
[2019-04-15T16:28:53,550][DEBUG][o.e.a.ActionModule       ] [_VVJsGe] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2019-04-15T16:28:53,933][INFO ][o.e.d.DiscoveryModule    ] [_VVJsGe] using discovery type [zen] and host providers [settings]
[2019-04-15T16:28:55,097][INFO ][o.e.n.Node               ] [_VVJsGe] initialized
[2019-04-15T16:28:55,097][INFO ][o.e.n.Node               ] [_VVJsGe] starting ...
[2019-04-15T16:28:55,283][INFO ][o.e.t.TransportService   ] [_VVJsGe] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2019-04-15T16:28:55,311][WARN ][o.e.b.BootstrapChecks    ] [_VVJsGe] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2019-04-15T16:28:55,312][WARN ][o.e.b.BootstrapChecks    ] [_VVJsGe] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-04-15T16:28:58,394][INFO ][o.e.c.s.MasterService    ] [_VVJsGe] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {_VVJsGe}{_VVJsGeoTuC17J2iMOH4Bw}{jOSXjf9kSayBtPy7cpwbZQ}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=8289153024, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2019-04-15T16:28:58,399][INFO ][o.e.c.s.ClusterApplierService] [_VVJsGe] new_master {_VVJsGe}{_VVJsGeoTuC17J2iMOH4Bw}{jOSXjf9kSayBtPy7cpwbZQ}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=8289153024, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {_VVJsGe}{_VVJsGeoTuC17J2iMOH4Bw}{jOSXjf9kSayBtPy7cpwbZQ}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=8289153024, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2019-04-15T16:28:58,523][INFO ][o.e.h.n.Netty4HttpServerTransport] [_VVJsGe] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2019-04-15T16:28:58,523][INFO ][o.e.n.Node               ] [_VVJsGe] started
[2019-04-15T16:28:58,587][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [_VVJsGe] Failed to clear cache for realms [[]]
[2019-04-15T16:28:58,661][INFO ][o.e.g.GatewayService     ] [_VVJsGe] recovered [0] indices into cluster_state
[2019-04-15T16:28:58,864][INFO ][o.e.c.m.MetaDataIndexTemplateService] [_VVJsGe] adding template [.triggered_watches] for index patterns [.triggered_watches*]
[2019-04-15T16:28:59,000][INFO ][o.e.c.m.MetaDataIndexTemplateService] [_VVJsGe] adding template [.watch-history-9] for index patterns [.watcher-history-9*]
[2019-04-15T16:28:59,088][INFO ][o.e.c.m.MetaDataIndexTemplateService] [_VVJsGe] adding template [.watches] for index patterns [.watches*]
[2019-04-15T16:28:59,185][INFO ][o.e.c.m.MetaDataIndexTemplateService] [_VVJsGe] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-6-*]
[2019-04-15T16:28:59,311][INFO ][o.e.c.m.MetaDataIndexTemplateService] [_VVJsGe] adding template [.monitoring-es] for index patterns [.monitoring-es-6-*]
[2019-04-15T16:28:59,419][INFO ][o.e.c.m.MetaDataIndexTemplateService] [_VVJsGe] adding template [.monitoring-beats] for index patterns [.monitoring-beats-6-*]
[2019-04-15T16:28:59,510][INFO ][o.e.c.m.MetaDataIndexTemplateService] [_VVJsGe] adding template [.monitoring-alerts] for index patterns [.monitoring-alerts-6]
[2019-04-15T16:28:59,601][INFO ][o.e.c.m.MetaDataIndexTemplateService] [_VVJsGe] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-6-*]
[2019-04-15T16:28:59,824][INFO ][o.e.l.LicenseService     ] [_VVJsGe] license [5ed9985f-b660-4589-9aff-93cce586ef90] mode [basic] - valid

this is my log elastic search sir


(David Pilato) #6

What is the output of

curl 127.0.0.1:9200

(ridwan syarifudin) #7

thank you sir, the problem has been solved


(David Pilato) #8

What was it?