Timeout for exporting csv in ES|QL

Elastic Stack Kibana
1

I'm an Elasticsearch beginner.
I have a problem about exporting csv in ES|QL.

Environment

OS: Debian 12
Elasticsearch Version: 8.15.2 (elasticsearch and kibana)

Problem

I tried exporting csv and got timeout error.
The message was as bellow.

Report contains warnings

Encountered an unknown error: Request timed out
Output
Status
Completed
Kibana version
8.15.2
Attempts
1 of 3
Content type
text/csv
Size in bytes
0
CSV rows
0
Search strategy
scroll
Report job ID
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Processed by
hostname (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
Timeout
3600 seconds

Timestamps
Time zone
Asia/Tokyo
Created at
2024-12-24T15:31:29.928+09:00
Started at
2024-12-24T15:31:32.401+09:00
Completed at
2024-12-24T15:32:04.304+09:00
Queue time
2.473 seconds
Execution time
31.903 seconds

I had changed the following parameters in kibana.yml but got the error yet.

xpack.reporting.queue.timeout: 3600000
xpack.reporting.csv.scroll.strategy: scroll

I want to get the result in csv format.
What should I do?

2

Thanks for reaching out, @petition2307. I have a few follow-up questions here:

  • Do you have a sample of the query you were running?
  • How were you exporting the CSV?

Best,

Jessica

3

Thank you for your reply.

Do you have a sample of the query you were running?

FROM sample-index_* METADATA _index, _id
| WHERE hostname IN ("AAAAAA","BBBBBB","CCCCCC","DDDDDD","EEEEEE","FFFFFF","GGGGGG","HHHHHH","IIIIII","JJJJJJ","KKKKKK","LLLLLL","MMMMMM","NNNNNN","OOOOOO","PPPPPP","QQQQQQ","RRRRRR")
| STATS mintime=MIN(@timestamp),maxtime=MAX(@timestamp) BY hostname,fieldA
| SORT mintime ASC

The total rows are 151.

How were you exporting the CSV?

After the search result has been displayed, I accessed as belows.
share -> Export -> Generate CSV -> "[Stack Management > Reporting]"

If you have any questions, please feel free to ask me.

Best Regards,

4

Thanks @petition2307,

You could try specifying the format directly in Dev Tools:

POST _query?format=csv
{
  "query": """
from sample-index_*
| where hostname in ("AAAAAA", "BBBBBB", "CCCCCC", "DDDDDD", "EEEEEE", "FFFFFF", "GGGGGG", "HHHHHH", "IIIIII", "JJJJJJ", "KKKKKK", "LLLLLL", "MMMMMM", "NNNNNN", "OOOOOO", "PPPPPP", "QQQQQQ", "RRRRRR")
| stats mintime = min(@timestamp), maxtime = max(@timestamp) by hostname, fieldA
| sort mintime asc
  """
}
5

Thank you, but I had already tried that.

image
image965×118 5.31 KB

I tried query async API, but it is not compatible with format=csv...

Best Regards,

6

Thanks for following up, @petition2307. Did you try it without the async API? Another workaround I thought of was running it in Python as described in this post and exporting from there.