Do I need a server to Elasticsearch?

Elastic Stack Elasticsearch
1

Hi, Really new to this and I have to ask if I have to have a server,like IIS or something similar, to ES.I want to access some XBRL files on a web service, and my first goal is just sending queries to retrieve the info.
further more I have some CORS issues, and I tried the;

 *$ grep cors elasticsearch-1.4.0.Beta1/config/elasticsearch.yml*
*http.cors.allow-origin: "/.*/"*
*http.cors.enabled: true*`indent preformatted text by 4 spaces`

But this didn't solve my problems.

Any suggestions and good advice is more than welcome.
-Thanks

2

Any reason you are using the 1.4 beta?

ES doesn't need a web server in front of it, no.
However what your problems are is not very clear, can you please elaborate?

3

Hi Warkolm, thanks for reply. My fault, I copy pasted the info from a site and not from my ES. I'm still using the 1.5.2. I have working on gaining access to some XML files from a remote server, and I couldn't figure it out. It turned out that they weren't accessible for the last 2-3 weeks :):):slight_smile:

4

I have working on gaining access to some XML files from a remote server

I don't think this part is very clear. Could you explain in more detail what you're trying to do? What software component should access the XML files? What should be done with the files and how should they be processed?

5

Hi Maguns, I'm trying to access the XBRL data from this Danish site. http://datahub.virk.dk/dataset/system-til-system-adgang-til-regnskabsdata. They give some CURL with queries which should search through the files. I only get either a message about CORS settings or I get the page from http://datahub.virk.dk/ "Page not here". Furthermore they provide an API; http://datahub.virk.dk/api. So I use the API' URL and try with the different queries, but no luck.
All the files from http://datahub.virk.dk/ should be in gzip format. Can that be a problem?
More important, last week the file weren't accessible and It's quit hard to get an answer if all is up and running.
I'm using ES 1.5.2 with Marvel 1.3.1.
I'm complete rookie here, I only got 3 weeks of experience.
-Regards

6

Okay. And you want to use Logstash to fetch the files for you and index them into Elasticsearch?

7

Sorry for stupid questions, but do I need Logstash? But yes I wan't search in the files and index them.

8

Then you need Logstash. Elasticsearch can't fetch URLs and index the contents. Well, maybe it's possible with river plugins but those are deprecated so you should use Logstash.

9

Hi Magnus, Thanks for reply.
I’m trying out the Logstash 1.5.0 and I’m using the guide from https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html#_up_and_running . I get some errors, but still it says that logstash starts?

It's the "IO console..." part which confuses me. If I follow the guide to the part about entering some test input, I get;

logstash recon.PNG670×686 30 KB

When I check the curl 'http://localhost:9200/_search?pretty' I don't see my Logstash, but instead the "_index": ".kibana".
Regards

10

Well, you're only configuring Logstash to send logs to stdout (and it appears to do so correctly). If you want messages to be sent to Elasticsearch you'll have to include an elasticsearch output.

11

Arrrrh man :slight_smile: Thank you.
OK so I'm up and running with the ELK and thanks a lot for help. My next issue is now connecting to the files.
I have trying to connect the files with the supported CURLs, but I don't strike gold.
Its the connection to remote files that troubles me, and I have played with some local files, with no issues.
I still have the API http://datahub.virk.dk/api and the CURL curl -XGET http://distribution.virk.dk/offentliggoerelser --data-binary @query_regnskaber.json
But I can't figure it out.
So I have been trying different solutions just out of curiosity.
I made the Logstash-simple.config out from the guide from Elasticsearch, and I'm wondering if I can put the API "http://datahub.virk.dk/api" in the "input { stdin { } }" in Logstash?
Furthermore I'm not sure if its legal to change the address in the"Server" field in Marvel?

12

What do you mean, exactly? The stdin input reads from stdin. If you—outside of Logstash—fetch the data you need and then invoke Logstash and pass the data via stdin that should work fine. Or, fetch the data via the API to local files that you have Logstash read via the file input.

Furthermore I'm not sure if its legal to change the address in the"Server" field in Marvel?

I have no idea. You should probably ask in a separate thread in the Marvel category.