delete the index and reindex it,kibana could not discover the data?
I could use curl to get the data,but in kibana discover page,nothing show,any one meet this problem?
ā ~ curl -XGET 'localhost:9200/_cat/indices?v&s=index&pretty'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana uHzsyU6_R8S_HON5E0O9Gg 1 1 7 0 44kb 44kb
yellow open cash_flow_statement 3EKlnpUiR8m8WQBlG4yXbA 5 1 3375 0 5.1mb 5.1mb
yellow open income_statement 93t024BBTWacZNsKeGDvMw 5 1 3379 0 2.6mb 2.6mb
yellow open index_china_day_kdata wib80Y-mQ1GB0GnEKx_oXg 5 1 30845 0 8.6mb 8.6mb
yellow open stock_china_day_kdata gkr2KcvYRrGPXTMUrGKDQQ 5 1 8227016 0 2.2gb 2.2gb
yellow open stock_meta -hO16JILTCW_Ab6dK8B_cw 5 1 3445 0 444.1kb 444.1kb
I see something strage in kibana:
Time reportEventDate timestamp
January 1st 1970, 13:36:10.930 October 25th 2017, 08:00:00.000 -
January 1st 1970, 13:36:10.930 October 28th 2017, 08:00:00.000
why the time is 1970?
And my timefield is reportEventDate...
Hi,
Have you created index patterns in Kibana for your indices?
This is where you create index patterns: https://www.elastic.co/guide/en/kibana/6.2/index-patterns.html
Thanks,
Bhavya
yes,I have created the index pattern.
In fact I could found the data, but it seems the time in kibana is very strage.
ā ~ curl -XPOST 'localhost:9200/income_statement/_search?pretty' -H 'Content-Type: application/json' -d'
{
"query": {
"constant_score": {
"filter": {
"term": {
"id": "stock_sh_603707_20171231"
}
}
}
}
}
'
{
"took" : 14,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [
{
"_index" : "income_statement",
"_type" : "doc",
"_id" : "stock_sh_603707_20171231",
"_score" : 1.0,
"_source" : {
"nonOperatingIncome" : 4063079.89,
"code" : "603707",
"sellingExpenses" : 4.696877972E7,
"OperatingTotalCosts" : 7.6705641194E8,
"id" : "stock_sh_603707_20171231",
"totalProfits" : 3.5620431768E8,
"operatingProfit" : 3.5244199594E8,
"exchangeGains" : 0.0,
"attributableToOwnersOfParentCompany" : 3.1390973003E8,
"financingExpenses" : -4342459.58,
"investmentIncomeFromRelatedEnterpriseAndJointlyOperating" : "526941.55",
"reportDate" : "20171231",
"disposalLossOnNonCurrentLiability" : 0.0,
"incomeTaxExpense" : 4.198191831E7,
"EPS" : 0.8131,
"nonOperatingExpenditure" : 300758.15,
"incomeFromChangesInFairValue" : 0.0,
"attributableToMinorityShareholders" : 0.0,
"reportEventDate" : "2018-01-27",
"netProfitAttributedToParentCompanyOwner" : 3.1422239937E8,
"accumulatedOtherComprehensiveIncome" : 3.1390973003E8,
"businessTaxesAndSurcharges" : 3815442.94,
"investmentIncome" : 6772404.66,
"operatingRevenue" : 1.11272600322E9,
"OperatingCosts" : 5.9668439996E8,
"securityId" : "stock_sh_603707",
"minorityInterestIncome" : 0.0,
"ManagingCosts" : 1.2296594498E8,
"assetsDevaluation" : 964303.92,
"dilutedEPS" : 0.8131,
"netProfit" : 3.1422239937E8,
"otherComprehensiveIncome" : -312669.34
}
}
]
}
}
you could see the time filed:
"reportDate" : "20171231"
But in kibana:
the time filed:
"reportDate"January 1st 1970, 13:36:11.231
In fact, when i index the data,the date format is different between reportDate(yyyyMMdd) and reportEventDate(yyyy-MM-dd).
And from the dsl query directly, all of themn are correct as you can see in the curl result.
Thanks for your reply.
Hi,
When you are creating your index pattern in Kibana - which field did you use for your timefilter? From the screenshot - it looks it's "reportEventDate" . And for this particular index - reportEventDate has the value of "reportEventDate" : "2018-01-27", and that is getting displayed correctly. You can read this for more information: https://www.elastic.co/guide/en/kibana/current/tutorial-define-index.html
I don't know whats happening with "reportDate" : "20171231". You might have to transform it using an ingest filter before ingesting the data into Elastic. I will find that out for you in a few hours.
Thanks,
Bhavya
Thanks!
more details:
the first time I index the data to es, evety thing is correct in kibana.
then I delete all of the index and reindex them again.
the strange things happen.
um..I used reportDate for the time field in kibana.
correct it:
it shoud be the time=reportDate and all of them is :1970...(so the problem is here)
I think es treat 20171231 as the seconds that have elapsed since January 1, 1970...
curl -XPOST 'localhost:9200/income_statement/doc/_search?pretty&filter_path=hits.hits._source' -H 'Content-Type: application/json' -d'
{
"query": {
"range" : {
"reportDate" : {
"gte" : "2017-06-30",
"lte" : "2017-06-30"
}
}
},
"size": 10,
"sort": [
{
"netProfit": {
"order": "desc"
}
}
]
}
'
{ }
the result is empty
and
curl -XPOST 'localhost:9200/income_statement/doc/_search?pretty&filter_path=hits.hits._source' -H 'Content-Type: application/json' -d'
{
"query": {
"range" : {
"reportDate" : {
"gte" : 20170630,
"lte" : 20170630
}
}
},
"size": 10,
"sort": [
{
"netProfit": {
"order": "desc"
}
}
]
}
'
{
"hits" : {
"hits" : [
{
"_source" : {
"nonOperatingIncome" : 8.33E8,
"code" : "600028",
"sellingExpenses" : 2.5955E10,
"OperatingTotalCosts" : 1.126122E12,
"id" : "stock_sh_600028_20170630",
"totalProfits" : 4.5032E10,
"operatingProfit" : 4.5015E10,
"exchangeGains" : 0.0,
"attributableToOwnersOfParentCompany" : 2.645E10,
"financingExpenses" : 1.289E9,
"investmentIncomeFromRelatedEnterpriseAndJointlyOperating" : "0",
"reportDate" : "20170630",
"disposalLossOnNonCurrentLiability" : 0.0,
"incomeTaxExpense" : 8.915E9,
"EPS" : 0.224,
"nonOperatingExpenditure" : 8.16E8,
"incomeFromChangesInFairValue" : 3.69E8,
"attributableToMinorityShareholders" : 8.557E9,
"reportEventDate" : "2017-08-28",
"netProfitAttributedToParentCompanyOwner" : 2.7092E10,
"accumulatedOtherComprehensiveIncome" : 3.5007E10,
"businessTaxesAndSurcharges" : 1.16297E11,
"investmentIncome" : 8.152E9,
"operatingRevenue" : 1.165837E12,
"OperatingCosts" : 9.42602E11,
"securityId" : "stock_sh_600028",
"minorityInterestIncome" : 9.025E9,
"ManagingCosts" : 3.5903E10,
"assetsDevaluation" : 4.076E9,
"dilutedEPS" : 0.224,
"netProfit" : 3.6117E10,
"otherComprehensiveIncome" : -1.11E9
}
},
the result is ok
So es don't handle timeformat "YYYYmmdd" corrcectly?
I think "YYYYmmdd" and "YYYY-mm-dd" always treated as valid date...
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.