Can not query when inserting data

Elastic Stack Elasticsearch
1

I am a newbie to elastic and use spring boots to interact with elastic.
I have a scenario as below:

  1. inserting a document like A
  2. inserting a document like B ( and this action takes 5m to complete) and I also insert other documents like C and D after but quickly
    then I performed searching but I got only A because the B was not completed
    after 5m I queried again and got all A B C and D.
    it seems that I can't query new data C and D when performing an action insert or update B
    so, could you tell me why and how can I fix this case by config or something else?
    Thanks.
2

Welcome!

Most likely you need to call the _refresh API.

But please read the warning on that page:

Refreshes are resource-intensive. To ensure good cluster performance, we recommend waiting for Elasticsearch’s periodic refresh rather than performing an explicit refresh when possible.

If your application workflow indexes documents and then runs a search to retrieve the indexed document, we recommend using the index API's refresh=wait_for query parameter option. This option ensures the indexing operation waits for a periodic refresh before running the search.

3

Thank you @dadoonet
But when I am inserting perhaps there is a transaction to lock the index, I called _refresh but it didn't work immediately and took a long time and it waited until the insert action finished then it worked

this is my insert code

  elasticsearchClient.index(i ->
                        i.index(elasticSearchConfig.fileIndex)
                                .id(finalInput.getId().toString())
                                .document(finalInput)

                );

query code


        SearchHits<FileDocumentContentDto> searchResponse = elasticsearchOperations.search(
                searchQuery,
                FileDocumentContentDto.class,
                IndexCoordinates.of(elasticSearchConfig.fileIndex));
4

This sounds incredibly long. What is the size and configuration of your cluster? Where is it running?

What is the size of your document? What type of data does it contain?

5

I parsed a PDF 180M to a text and then stored it in the index. is that ok?

I used a default configuration like that

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 16-04-2024 14:12:48
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
#----------------------- END SECURITY AUTO CONFIGURATION -------------------------
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.