It is currently Sat Aug 22, 2020 4:26 am


All times are UTC




Post new topic Reply to topic  [ 4 posts ] 
Author Message
 Post subject: Serialization question
PostPosted: Fri Aug 01, 2014 11:49 am 

Joined: Fri Aug 01, 2014 11:39 am
Posts: 2
Hi,

Thanks for working on PolyVox. The more recent versions of PolyVox removed built in serialization, asking us to do it ourselves. That's fine, but the only public way I can find to access the voxel data is through getVoxel(). That would work fine for small Volumes, but for something large it won't work very well. Is there a way to access compressed blocks in a LargeVolume so that I can directly read and write the compressed blocks to/from disk?

Thanks!


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Serialization question
PostPosted: Fri Aug 01, 2014 1:34 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
decev wrote:
...the only public way I can find to access the voxel data is through getVoxel(). That would work fine for small Volumes, but for something large it won't work very well. Is there a way to access compressed blocks in a LargeVolume so that I can directly read and write the compressed blocks to/from disk? Thanks!


As I recall, getVoxel()/setVoxel() is currently the only correct way to get/set the data, and you are correct that this is probably sub-optimal from a performance point of view. I think there's some unnecessary compression/decompression going on there as well.

It might well make sense to expose a pointer to the underlying data, or to overload '<<' to allow steaming of all the data at once. But actually the internal ordering of the voxels could change in the future, for example to store them in Z-Curve order rather than linear order. A streaming operator might be able to handle this though.

Actually, the plan is top replace LargeVolume with a 'PagedVolume' which removes the two-level hierarchy (compressed and uncompressed blocks) and stores/pages only uncompressed blocks, leaving compression to the user. This should make everything a bit simpler and more direct.

Practically speaking, you might just want to expose a pointer to the raw data in the mean time.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Serialization question
PostPosted: Fri Aug 01, 2014 11:31 pm 

Joined: Tue Apr 08, 2014 5:10 pm
Posts: 124
As all of Polyvox is open-source, just hack the Large Volume yourself, expose this nasty private pointer to the entire universe and dump it's pointee as you like. It seems to work but if you upgrade polyvox, your map format might become useless. I tried this approach but ... cancelled my wish, and implemented own file-per-chunk approach with boost::serialization::binary_*archive and get/setVoxelAt().


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Serialization question
PostPosted: Mon Aug 04, 2014 12:46 am 

Joined: Fri Aug 01, 2014 11:39 am
Posts: 2
Makes sense, thanks guys. I just wanted to check to make sure there wasn't another way before doing the pointer approach.


Top
Offline Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
Theme created StylerBB.net