It is currently Sat Aug 22, 2020 12:30 pm


All times are UTC




Post new topic Reply to topic  [ 150 posts ]  Go to page Previous  1 ... 9, 10, 11, 12, 13, 14, 15  Next
Author Message
 Post subject: Re: Streaming Volume
PostPosted: Sun Mar 20, 2011 7:21 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
ker wrote:
doesn't seem like that at all, the current interface is much better.
did you also create a way to disable rle?

Yep, it's just gone in, You can now call setCompressionEnabled(false). From my point of view I now happy with the features. I'm going to make your example into a seperate 'PagingExample' and revert the Basic example back to it's previous state. Then write some documentation, and then I think we can merge back to the trunk.

ker wrote:
I think we need one more thing for streaming thou. and that is a function to force a load of a region, and force an unload of a region. (obviously force-unload only unloads full blocks inside the given region, and force-load loads full blocks even if they only touch the given region with 1 voxel)
these functions might even return some value (like a region) of what was actually loaded/unloaded

I do not require the force-load at all, but the force-unload is something that can't even be triggered from the outside.

This is an interesting question. This sounds a bit abstract, but as I see it the paging mechanism isn't intended as a way of loading data into the volume (although I accept it can be used for this). In your case I would say that the whole world exists in the volume at all times, it's just that parts of it are being cached in memory for faster access.

I'm not sure if I'm talking sense here I'm just trying to decide what are the right concepts. But on this basis, I might ask why you want to force part of your volume out of memory. If it's because the application is shutting down and so you want to save any changes, then remember you can still access all the data (whether it's in memory or not) using Volume::getVoxelAt().

Ok, but dispite saying that I can see it would make users code simpler if they could force-unload, as they can have all their saving code in one place. And it's quite a simple feature to implement.

Actually, can't you just call setMaxNumberOfBlocksInMemory(0) to achieve this?

As for force loading, yes I agree. This is a useful way or warning the volume that you are about to use a particular part of the volume and it should be ready for it. It can take a region, compute which blocks are inside it, and simply call getUncompressedBlock() on each. You want to implement that? :)


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Sun Mar 20, 2011 8:20 pm 
User avatar

Joined: Wed Jan 26, 2011 3:20 pm
Posts: 203
Location: Germany
both force unloading and force loading are not for the actual representation of the world, but for performance.

about force-unloading:
you unload a few blocks by position (which is significantly faster than by timestamp)
then when loading new blocks there will be no necessity to find the least recently used block.

a scenario where this would be useful is when you want a region around the player to be loaded at all times.
once the player is detected to have moved so far that new blocks needs to be loaded, he'll also have moved so far that you know what blocks you can unload because they are far away.
if this could be done inside the volume class, it would not only be extremely fast (only accessing the block map by key) but it would remove many many unplannable calls to the overflow handler.

I currently have no clue if that is a good way to do this or makes sense from a game-perspective, but it might be necessary in the future when we're playing around with performance settings.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Sun Mar 20, 2011 9:18 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Ok, I'm convinced, these both make sense. What shall we call them? Because I see the data in memory as a kind of cache containing the most recently used part of the volume, I think the terms 'prefetch' and 'flush' make sense?


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Sun Mar 20, 2011 10:32 pm 
User avatar

Joined: Wed Jan 26, 2011 3:20 pm
Posts: 203
Location: Germany
yes, flush and prefetch are perfect. Once you have committed your latest changes I can implement them.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Sun Mar 20, 2011 11:12 pm 

Joined: Tue Mar 08, 2011 3:57 am
Posts: 46
Been watching this thread for awhile now - Real nice work guys!
I might, however, just note that some updated documentation might be required as I've had trouble understanding the new method.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Mon Mar 21, 2011 12:03 am 
User avatar

Joined: Wed Jan 26, 2011 3:20 pm
Posts: 203
Location: Germany
DJDD wrote:
Been watching this thread for awhile now - Real nice work guys!
I might, however, just note that some updated documentation might be required as I've had trouble understanding the new method.


hehe very good point...
the paging example (current svn basic example) shows how it's used.
the volume class will obviously be documented (I believe to remember David saying it's the best documented class of polyvox :D and I'll do my best not to let my changes stop that from being true)
but a small tutorial or two in the wiki shouldn't be too much work either.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Mon Mar 21, 2011 10:27 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Yes, documentation will be important - I have a go at it at the weekend. This is basically the core class of PolyVox so I'll try to keep it detailed.

I've just deprecated the original constructor which took a width, height, and depth. The problem was there is a new constructor taking two pointers and a int, and unfortunatly C++ will happily cast between ints and pointers meaning the wrong constructor could easily be called. The original constructor will now assert and/or abort, so if you want a fixed size volume you must pass in a Region.

@ker - I think I'm done for now so I'll hand it back to you do implement the flush and prefetch.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Mon Mar 21, 2011 10:52 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Oh, and I renamed your example to 'PagingExample' - in case your wondering where it went.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Tue Mar 22, 2011 11:23 am 
User avatar

Joined: Wed Jan 26, 2011 3:20 pm
Posts: 203
Location: Germany
I found something in the Volume-class...
Code:
    template <typename VoxelType>
    void Volume<VoxelType>::setMaxNumberOfBlocksInMemory(uint16_t uMaxNumberOfBlocksInMemory)
    {
        //FIXME? - I'm concerned about the logic in here... particularly the
        //way timestamps are handled. Perhaps extract all timestamps, sort them,
        //identify the cut-off point, and then discard those above it?


well that depends on what this function is supposed to do...
in setMaxNumberOfUncompressedBlocks you simply unload your whole cache...
why not do the same here? (in case the number is decreased, otherwise it doesn't matter anyway) this is a "cache", too
for me any solution is fine, I set the number once on startup and never again


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Streaming Volume
PostPosted: Tue Mar 22, 2011 11:30 am 
User avatar

Joined: Wed Jan 26, 2011 3:20 pm
Posts: 203
Location: Germany
and another thing from getUncompressedBlock:

Quote:
//FIXME - can we pass the block around so that we don't have to find it again when we recursively call this function?


I have been thinking about this.
Since now there is a ConstVolumeProxy... this is now really simple, just give the block to the ConstVolumeProxy and remove the evil-setVoxel-const-function from Volume.

I just had no Idea how to do this before you came up with ConstVolumeProxy.


Top
Offline Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 150 posts ]  Go to page Previous  1 ... 9, 10, 11, 12, 13, 14, 15  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
Theme created StylerBB.net