Python parallel_bulk, info says no errors, but nothing in index

Elastic Stack Elasticsearch
1

Here is the part of my python code. Took me a while to get this working without any errors as I didn't know what was the input structure function was accepting... Anyway here it is:

docs = [
           {
            "Created":  "2021-05-30T18:35:16.587Z",
            "Action":  "AccessRequestCancelled",
            "Type":  "ACCESS_REQUESTT",
            "Actor":  "mateparac",
            "Target":  "mateparac",
            "Stack":  "wps",
            "TrackingNumber":  "2c9180867a5ca31f017a5e332e763f9",
            "Objects":  "ACCESS",
            "Operation":  "REQUEST",
            "Status":  "CANCELLED",
            "TechnicalName":  "ACCESS_REQUEST_CANCELLEDs",
            "Name":  "Request Access Cancelled!"
        },
        {
            "Created":  "2021-04-30T18:35:16.587Z",
            "Action":  "AccessRequestCancelled",
            "Type":  "ACCESS_REQUESTT",
            "Actor":  "mateparac",
            "Target":  "mateparac",
            "Stack":  "wps",
            "TrackingNumber":  "2c9180867a5ca31f017a5e332e763f9",
            "Objects":  "ACCESS",
            "Operation":  "REQUEST",
            "Status":  "CANCELLED",
            "TechnicalName":  "ACCESS_REQUEST_CANCELLEDs",
            "Name":  "Request Access Cancelled!"
        },
        {
            "Created":  "2021-04-30T18:35:16.587Z",
            "Action":  "AccessRequestCancelled",
            "Type":  "ACCESS_REQUESTT",
            "Actor":  "mateparac",
            "Target":  "mateparac",
            "Stack":  "wps",
            "TrackingNumber":  "2c9180867a5ca31f017a5e332e763f9",
            "Objects":  "ACCESS",
            "Operation":  "REQUEST",
            "Status":  "CANCELLED",
            "TechnicalName":  "ACCESS_REQUEST_CANCELLEDs",
            "Name":  "Request Access Cancelled!"
        }
]


def insert_data(documents, indexname):
    for document in documents:
        json_line = {}
        for key in document:
            json_line[key] = document[key]
        yield {
            '_op_type': 'index',
            '_index': indexname,
            '_type': 'doc',
            '_source': json.dumps(json_line)
        }


for success, info in parallel_bulk(es, insert_data(docs, "sailpont-dev-logs"), chunk_size=5000, thread_count=4, queue_size=4):
    if not success:
        print('A document failed:', info)

During debug, info returns all good, no errors but I am not finding these documents in index.
Here is debug result:
debug|690x81

Not sure what is the problem here :confused:

2

Typo in index name :frowning: , it works fine.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.