[leo_object_storage] Find incorrect some data-blocks during the data-compaction

[yosukehara]

Last Friday, I found incorrect some data-blocks at some Leo Storage's AVS files as below:

Error log

[E] storage_0@127.0.0.1 2015-06-15 00:45:15.130856 +0000    1434329115  leo_compact_fsm_worker:execute_1/4  1036    [{obj_container_path,"/var/lib/jenkins/jobs/Tests_for_aws-sdk_clients_with_LeoFS/workspace/leofs/package/leo_storage_0/avs/object/2.avs_63601547870"},{error_pos_start,577},{error_pos_end,739},{errors,[{invalid_format,unexpected_time_format}]}]
[E] storage_0@127.0.0.1 2015-06-15 00:45:15.132416 +0000    1434329115  leo_compact_fsm_worker:execute_1/4  1036    [{obj_container_path,"/var/lib/jenkins/jobs/Tests_for_aws-sdk_clients_with_LeoFS/workspace/leofs/package/leo_storage_0/avs/object/1.avs_63601547870"},{error_pos_start,586},{error_pos_end,749},{errors,[{invalid_format,unexpected_time_format}]}]
[E] storage_0@127.0.0.1 2015-06-15 00:45:15.169797 +0000    1434329115  leo_compact_fsm_worker:execute_1/4  1036    [{obj_container_path,"/var/lib/jenkins/jobs/Tests_for_aws-sdk_clients_with_LeoFS/workspace/leofs/package/leo_storage_0/avs/object/1.avs_63601547870"},{error_pos_start,945},{error_pos_end,1108},{errors,[{invalid_format,unexpected_time_format}]}]
[E] storage_0@127.0.0.1 2015-06-15 00:45:15.668450 +0000    1434329115  leo_compact_fsm_worker:execute_1/4  1036    [{obj_container_path,"/var/lib/jenkins/jobs/Tests_for_aws-sdk_clients_with_LeoFS/workspace/leofs/package/leo_storage_0/avs/object/1.avs_63601547870"},{error_pos_start,31501},{error_pos_end,31652},{errors,[{invalid_format,unexpected_time_format}]}]

And then, I've recognised S3 Client OR Leo Storage does not set correctly timestamp of an object when removing it as below:

%% Error log during the compaction:
[E] storage_0@127.0.0.1 2015-06-16 01:49:02.324330 +0000    1434419342  leo_object_storage_haystack:put_fun_2/3 586 [{key,<<"test7877/testFile s.org\n1">>},{del,1},{timestamp,0},{cause,"Not set timestamp correctly"}]
[E] storage_0@127.0.0.1 2015-06-16 01:49:23.739688 +0000    1434419363  leo_object_storage_haystack:put_fun_2/3 586 [{key,<<"test/testFile\n1">>},{del,1},{timestamp,0},{cause,"Not set timestamp correctly"}]
[E] storage_0@127.0.0.1 2015-06-16 01:49:23.766323 +0000    1434419363  leo_object_storage_haystack:put_fun_2/3 586 [{key,<<"test/testFile.copy\n1">>},{del,1},{timestamp,0},{cause,"Not set timestamp correctly"}]
[E] storage_0@127.0.0.1 2015-06-16 01:49:29.770379 +0000    1434419369  leo_object_storage_haystack:put_fun_2/3 586 [{key,<<"test/testFile\n2">>},{del,1},{timestamp,0},{cause,"Not set timestamp correctly"}]
[E] storage_0@127.0.0.1 2015-06-16 01:49:29.774589 +0000    1434419369  leo_object_storage_haystack:put_fun_2/3 586 [{key,<<"test/testFile\n1">>},{del,1},{timestamp,0},{cause,"Not set timestamp correctly"}]
[E] storage_0@127.0.0.1 2015-06-16 01:49:29.786912 +0000    1434419369  leo_object_storage_haystack:put_fun_2/3 586 [{key,<<"test/testFile.copy\n1">>},{del,1},{timestamp,0},{cause,"Not set timestamp correctly"}]
[E] storage_0@127.0.0.1 2015-06-16 01:49:29.800619 +0000    1434419369  leo_object_storage_haystack:put_fun_2/3 586 [{key,<<"test/testFile.single\n1">>},{del,1},{timestamp,0},{cause,"Not set timestamp correctly"}]

Take a measure

• Avoid inserting incorrect data into a AVS file and Set correct timestamp of an object as well as a metadata

[yosukehara]

We've fixed this issue:

• leo-project/leo_object_storage@5770d2e
• leo-project/leo_storage@6685a63?diff=split