Problem with csv import into a fresh elasticsearch and kibana environment

Elastic Stack Kibana
1

Hi guys,

I'm hoping to get some help with a problem I'm having.
I just installed two Elasticsearch nodes in a cluster with a Kibana frontend (also freshly installed). I have configured everything according to the documentation (at least I think), but when I import a CSV file, it freezes when uploading the file (wizard).

image
image1244×206 9.55 KB

The pipeline and the index appear to be created correctly. I can also see the loading-icon in the top left-hand corner.

When I close the import help and go to the discovery screen, I see values being imported, but it takes a while for all the data to appear to be imported. The problem is that it is not all the data and many entries are imported multiple times.

I have a similarly configured instance (complete ELK) in my lab and it is configured in exactly the same way (except for the second elastic node). However, the data is imported cleanly there and the wizard also runs normally.

Some further infos to the cluster-health:

{
  "cluster_name" : "analyze",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "active_primary_shards" : 32,
  "active_shards" : 64,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

Every node is running on his own VM and also they are all in one subnet. No local firewall is active.

Just tell me, if you need some further information... thanks everyone for the help!

Kind regards,

Ulli

2

Hi @um3n Welcome to the community.

How big is the file (lines)?
How many fields?
How complex are the fields ...quoted etc?
How big are the data nodes RAM / CPU?

Did you just wait a bit?

What is a while?

3

Hey stephenb,

I'll do my best to anwser your Questions :wink:

The file I try to import is about 3,27 mb and has about 12000 lines.

We are talking about 21 fields.

It is an export from a ticketsystem, so the solution, comments and descriptions could be a little bit bigger.

Both nodes have 16GB RAM and 8 Cores of CPU.
The CPU is a "Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz"

I'll wait about 10 to 15 minutes before I click elsewhere.

Hard to say, but while the Icon in the top left corner is showing this loading circle, in discovery you can see, how the amount of documents is increasing.

Thanks alot for your help!

Kind regards

Ulli

4

Thanks for the information.

In short that file should load quickly say a minute or less especially with your hardware... So it seems to be something else... What that is? I am not sure.

Do you have another file to try?

Are there any error messages?

5

Hey stephenb,

thanks for your time so far!
I added an integration with elastic-agent and added the kibana.
This is running fine.

I also tried to upload the csv file without changeing anything and this time it is running fine... I tested this last week as well, but now it works...

I'll close this for now!

Kind regards,

    Ulli
1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.