We are in the process of migrating this forum. A new space will be available soon. We are sorry for the inconvenience.

BUG in Large Object Support ?


maverck
03-28-2015, 07:05 PM
Hi Vincent,

One more time thanks for the answer.

Is there any chance that there will be any documentation of hubic 'addons' to openstack swift or rather the way hubic use the api?

As far as I discovered indeed segments are stored in default_segments in path looks like:
default_segments/UUID/timestamp/some_number/part_number
1) UUID is version 4 which means it's random. (hope I get it right)
2) Timestamp is normal UNIX format (http://www.unixtimestamp.com/)
3) Some number is magic (no ideas here). Only know it's 8 digit. Looks random.
4) Part number is 8 digit

And there is special Content-type: hubic/uploading.
It makes all the hubic apps ignore that file until Content-type is removed after upload.
This makes hubic use content-type based on file extension.

I also do the research about app I using to backup NAS, and it looks like code is in python and very easy to decompile and change.
Maybe that's the way to do it hubic-compatible

vcasse
03-28-2015, 10:12 AM
Hi maverck,

Our apps use 100MB segments for synchronisation, and different size of segments for backups (since 4MB to 5GB)

Best regards,
Vincent

maverck
03-28-2015, 09:48 AM
And one more question.
Whats the file size that makes hubic original app to split it into segments ?

maverck
03-27-2015, 06:15 PM
Quote Originally Posted by vcasse
A good practice inside openstack swift is to store segments inside a different container. (By instance, we store segments from 'default' container inside 'default_segment' container, as openstack good practice propose).
Thank you for the response.

I found this when using Openstack Swift app for QNAP.
And then check with specification. (some tests with curl)
Unfortunatelly there is no option in that app to store segments in different container

Could you write in this thread after QA team will find something ?

vcasse
03-27-2015, 05:41 PM
Hi Maverck,

Indeed, there is maybe a bug. Our QA team will check it.
I think this duplicate is caused by your segments locations : you store your segments inside a pseudo folder.

A good practice inside openstack swift is to store segments inside a different container. (By instance, we store segments from 'default' container inside 'default_segment' container, as openstack good practice propose).

Best regards,
Vincent

maverck
03-27-2015, 05:25 PM
And screenshot from swift explorer
Attachment 392

maverck
03-27-2015, 04:27 PM
Quote Originally Posted by vcasse
Hi,

This behavior is strange. Could you send a result of listing with openstack api ?

Regards,
Vincent
This is part of the output of GET command at default container.


test4
d41d8cd98f00b204e9800998ecf8427e
0
application/directory
2015-03-27T15:01:46.298350


test4/test
d41d8cd98f00b204e9800998ecf8427e
0
application/x-www-form-urlencoded
2015-03-27T15:15:12.091490


test4/test/1
c4ca4238a0b923820dcc509a6f75849b
1
application/x-www-form-urlencoded
2015-03-27T15:13:18.314900


test4/test/2
c81e728d9d4c2f636f067f89cc14862c
1
application/x-www-form-urlencoded
2015-03-27T15:13:23.831390


test4/test/3
eccbc87e4b5ce2fe28308fd9f2a7baf3
1
application/x-www-form-urlencoded
2015-03-27T15:13:29.194820

vcasse
03-27-2015, 03:39 PM
Hi,

This behavior is strange. Could you send a result of listing with openstack api ?

Regards,
Vincent

maverck
03-27-2015, 03:23 PM
Hi,
I think that I found a bug in large object support from openstack swift.

I tried to upload multi-part file as in following instruction (Dynamic Large Object):
http://docs.openstack.org/developer/...tml#direct-api

After putting last request - adding X-Object-Manifest
I found duplicate entry on my account in web browser (screenshot below).

https://forums.hubic.com/attachment....0&d=1427469755

Is it possible to repair this ?

Regards,
Maciek