Timestamp

Hello Expert,
If i would like to insert old data for example last year data in ES, could I change the timestamp as date of last year? If No, then what is the solution to insert the data which correspondence the time?
Thanks in advance

Elasticsearch will accept any timestamp you want, it doesn't need to be a current one.

but my old data dont have any timestamp... so can i able to put last year timestamp ?

You can add it in, yes.

can you able to give an example how to fix timestamp in index.conf file ?

I am not sure what you are referring to there.

I will give reference of last year date with date field... ?
pls give me an example

That depends on how to send the data to Elasticsearch

For example, If i would like to give timestamp of last month such as 11-05-2021, then how can i replace current timestamp into 11-05-2021....

`filter{
    csv{
    skip_empty_rows => "true"
    skip_header => "true"
    separator => ","
    columns => ["Job_ID", "Job_Type", "Client", "Client_Group", "Agent", "Instance", "Backup_Set", "Subclient", "Storage_Policy", "Job_Size(GB)", "SnapShot", "Backup_Date"]
    }
    mutate { gsub => [ "message", ";;;;;;;;;;;\r", " " ] }
    #mutate { gsub => [ "message", ";", "," ] }
    if [message] =~ /^,+$/  {drop { }}
    if ![message] {
            drop{ }
    }
    if [message] =~ /^([^,]*,){5}[^,]*$/ {
    drop {}
    }
    if [message] =~ /^([^,]*,){4}[^,]*$/ {
    drop {}
    }

    mutate {convert => ["Job_ID","integer"]}
    mutate {convert => ["Job_Size(GB)","float"]}
    #remove_field => ["message"]
    date {
            match => ["Backup_Date", "yyyy-MM-dd'T'HH:mm:ss'.'SSS'Z'"]
            timezone => "UCT"
            target => "@timestamp"
            }

Output but date parce failure and i want to fix Backup_date into timestamp

{
"@timestamp" => 2021-06-03T07:55:40.141Z,
"Storage_Policy" => "SP_Bouge_Chamballud_BUR",
"Agent" => "Windows 2003 32-bit File System",
"Instance" => "N/A",
"message" => "10740363,Backup,yoda,_Client_sur_Cmmat-003; Angers; Bouge_Chamballud; svp1-vcent-004; DBA_serveur; Oracle; svp1-vcent-004-Bouge_Chambalud; Reboot_MA_Hebdo; client FS,Windows 2003 32-bit File System,N/A,BS_yoda_FS,SC_yoda_FS,SP_Bouge_Chamballud_BUR,238.41,No,2021/06/02;;;;;;; ",
"Client_Group" => "_Client_sur_Cmmat-003; Angers; Bouge_Chamballud; svp1-vcent-004; DBA_serveur; Oracle; svp1-vcent-004-Bouge_Chambalud; Reboot_MA_Hebdo; client FS",
"Job_Size(GB)" => 238.41,
"Job_ID" => 10740363,
"Subclient" => "SC_yoda_FS",
"SnapShot" => "No",
"Client" => "yoda",
"@version" => "1",
"path" => "/etc/logstash/CSV/new/L.csv",
"Backup_Date" => "2021/06/02;;;;;;;;;;;;;;;;;;",
"Job_Type" => "Backup",
"host" => "svp2-elasta-001.entreprise.corp",
"Backup_Set" => "BS_yoda_FS",
"tags" => [
[0] "_dateparsefailure"
]
}`

@warkolm
` date {
match => ["Backup_Date","yyyy-MM-dd'T' HH:mm:ss.SSS:'Z'"]
timezone => "UCT"
target => "@timestamp"
}

output

{
"tags" => [
[0] "_dateparsefailure"
],
"@version" => "1",
"@timestamp" => 2021-06-03T14:42:15.305Z,
"Client" => "ZZZ_svp1-spool-046",
"Backup_Date" => "2021-05-28 20:24:46",
"Job_Size(GB)" => 32.0
}`

That is not a valid date.

Thank you @warkolm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.