_split_type_failure

Elastic Stack Logstash
1

Hi all, i have config logstash like this

input {
    http_poller {
        urls => {
            users => {
                method => GET
                user => "user@gmail.com"
                password => "password"
                url => "https://api.id/api/report/product"
                headers => {
                    "Content-Type" => "application/json"
                    Authorization => "bearer token"
                }
            }   
        }
        request_timeout => 60
        schedule => { every => "20s"}
        codec => "json" 
        type => "Http_poller"  
    }
}

filter {
    split {
        field => "[result][data][products]"
    }
    split {
        field => "[result][data][products][loctypes]"
    }
}

output {
    elasticsearch {
        hosts => ["localhost:9200"]
        index => "products"
    }
    stdout {
        codec => rubydebug 
    }
}

and the result in discover kibana like this

tes1
tes11089×536 23 KB

Any one can help me?

2

Only arrays and strings can be operated on by a split filter. So one or both of "[result][data][products]" and "[result][data][products][loctypes]" are of some other type. The logstash log will have messages to tell you that.

The event contains a [message] field which contains the text "error", which suggests your http_poller is not working the way you expect.

3

This is the data source form my API

"message": "success",
    "result": {
        "data": {
            "products": [
                {
                    "id": "id",
                    "code": "Code",
                    "brand": "Brand",
                    "loctypes": [
                        {
                            "description": "AD"
                        }
                    ],
                    "desc": "ADD",
                    "purchasecontractd": null,
                    "pods": [
                         {
                            "qty": 1
                        }
                    ],
                    "transferservice": null,
                    "balanceservice": null,
                    "prequisitionds": [
                        {
                            "qty": 1
                        }
                    ],
                    "categories": [
                        {
                            "desc": "ADS"
                        }
                    ],
                    "units": [],
                    "qty": 1,
                    "productattrs": [
                        {
                            "groupattr": 1
                        }
                    ],
                    "items": [],
                    "money": 10000,
                    "status": "available",
                    "createdAt": "2020-11-08T01:59:02.381Z",
                    "updatedAt": "2020-11-25T07:26:32.404Z"
                },

If i use the config it would be error.

filter {
    split {
        field => "[result][data][products]"
    }
    split {
        field => "[result][data][products][loctypes]"
    }
}

How to split the data to be like this?

                    "id": "id",
                    "code": "Code",
                    "brand": "Brand",
                    "loctypes": "AD"
                    "desc": "ADD",
                    "purchasecontractd": null,
                    "pods": 1
                    "transferservice": null,
                    "balanceservice": null,
                    "prequisitionds": 1
                    "categories": "ADS"
                    "units": "BC",
                    "qty": 1,
                    "productattrs": 1
                    "items": "items",
                    "money": 10000,
                    "status": "available",
                    "createdAt": "2020-11-08T01:59:02.381Z",
                    "updatedAt": "2020-11-25T07:26:32.404Z"
4

No, it is not. Look at the screen shot you posted. The message field contains "error", not "success".

If the response from the API looked the way you think it should then your split filters would work. You would then have to move things around with mutate filters like

    mutate {
        rename => {
            "[result][data][products][productattrs][0][groupattr]" => "[result][data][products][productattrs]"
            "[result][data][products][prequisitionds][0][qty]" => "[result][data][products][prequisitionds]"
        }
    }
5

Hello badger, i do your solution

and the result like this

resukt
resukt797×604 28.9 KB

the message is succes, but why the tags is _split_type_failure?

6

Does the [result][date][products] field exist?

7

Hello Badger, i have 8 table API. But the table product not include to index pattern. And other tables are include to index pattern, the tags change to _split_type_failure. I only use 1 index pattern to 8 table API, it is true or wrong?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.