Volumes Of Fun
http://www.volumesoffun.com/phpBB3/

Faster volume resampling
http://www.volumesoffun.com/phpBB3/viewtopic.php?f=14&t=512		 Page 1 of 1
Author:  Illidan [ Sun May 12, 2013 8:18 am ]
Post subject:  Faster volume resampling
Hello,
I'm facing some performance problems in my game so after doing some profiling I narrowed it down to the volume resampling.

In order to perform extractions in other threads, I "copy" the interested regions from the LargeVolume in several RawVolumes on which I do the extractions.
When this happens, it's usually a lot of regions at once (usually between 20-40 32x32x32 regions) and while the time required for a single resampling is unnoticeable, it quickly adds up.
From my profiling in release mode (with all optimizations enabled), it takes about ~110 ms to resample 40 regions from a LargeVolume to a RawVolume using a uint32_t as material, and while this seems a relatively small time, when it happens in game the hiccup is very noticeable.
Note that the LargeVolume has enough uncompressed blocks to fit all the regions in this profiling. With just 10 regions and using the RLE compressor, it takes about 160 ms.

I still have to do some proper profiling, but before I start I just wanted to ask if there's a way I can make this faster? Maybe I have some settings wrong on the Large Volume or something simple like that.

Considering on the client I don't really use the LargeVolume for what it was created for (at least I think, I have an infinite world but I manually flush regions when they're far enough from the player so there's always a fixed number of regions in the volume), maybe I'm better off using another volume type?

Thanks in advance for any answer.

EDIT: I'm using the latest revision of PolyVox.
Author:  Freakazo [ Sun May 12, 2013 11:01 am ]
Post subject:  Re: Faster volume resampling
Can you elaborate on why you are resampling the volumes and not just copying them over? Is this for LOD?
Author:  Illidan [ Sun May 12, 2013 11:11 am ]
Post subject:  Re: Faster volume resampling
Freakazo wrote:
Can you elaborate on why you are resampling the volumes and not just copying them over? Is this for LOD?

If you use the resampler with 2 regions of the same size it just copies the volume over 1:1

I tried doing it manually but it makes no difference performance-wise.
Author:  Freakazo [ Sun May 12, 2013 1:34 pm ]
Post subject:  Re: Faster volume resampling
I get nearly the exact same performance you, so it's not just your setup. It looks like getVoxel and set voxel is the big performance problem.

Hmmm, I can't think of a clean solution to this. Only solution I can think of is instead of working at the voxel level is to work at the block level and copy the voxel data into the rawvolume voxel data. You'd have to make a lot of changes to polyvox to be able to do that.
Author:  Illidan [ Sun May 12, 2013 2:16 pm ]
Post subject:  Re: Faster volume resampling
Well, that was actually a nice idea Freakazo.

I have set the getUncompressedBlock function of LargeVolume as public. This allows you to get the raw data of a block as it's declared as public on Block. Then I added a new constructor to RawVolume that just copies the data over with a memcpy (no error checking atm).

I don't know if it's the best way of doing it nor if it actually works (I just tried in the test I prepared before), but the new performance for the same test is 19ms, so it seems like the right way to go, assuming it works.

I think the library would really benefit from having more "low-level" functions like this considering how fast they seem to be compared to the other more convenient functions.

I still can't try it in my game because I extract a region bigger than a block (1 voxel bigger in every direction), but maybe a getUncompressedRegion function can be added although that would be a little more complicated.

Thanks for pointing me in the right direction ;)

EDIT: after a bit of testing, it seems to work for the first X row of voxels (x,0,0) but not for the others (the data is different), any idea why? The order used in Block and RawVolume seems the same to me.
The new constructor is just the normal one with the memcpy call after initialise (data is a VoxelType* pointer to the uncompressed block raw data):
Code:
      memcpy(m_pData, data, this->getWidth() * this->getHeight()* this->getDepth() * sizeof(VoxelType));
Author:  Freakazo [ Sun May 12, 2013 2:50 pm ]
Post subject:  Re: Faster volume resampling
Does your memcpy line look something like this:

memcpy ( rawVolData, uncBlock, sizeof(uint32_t) * x*y*z );

Only problem I can think of.

Exposing low level functions like this is nice but it leaves a lot of functionality for the user to implement, especially because of the heavy use of templates (The volume resampler casts it to the destinations data type for you, as an example). But yeah I frequently move members from private to public because I need to do something unique/speed is important.

If I get back into coding again the first thing I'd do is try and implement correct threading for the large volume class, it's probably the only reason why I have to do hackish things like this.

[edit] Just saw your edit, hmm is the data completely random? Try to set each row to a unique. Should make debugging it a bit easier.
[edit2]This peace of code in the accesors are different, could that be the source of the problem?
Code:
rawVolume::getVoxelAt()...
         const Vector3DInt32& v3dLowerCorner = this->m_regValidRegion.getLowerCorner();
         int32_t iLocalXPos = uXPos - v3dLowerCorner.getX();
         int32_t iLocalYPos = uYPos - v3dLowerCorner.getY();
         int32_t iLocalZPos = uZPos - v3dLowerCorner.getZ();
Author:  Illidan [ Sun May 12, 2013 3:05 pm ]
Post subject:  Re: Faster volume resampling
Nevermind it was just a stupid error in the test (I created the raw volume as 33x33x33 instead of 32x32x32). It works fine.
Author:  David Williams [ Mon May 13, 2013 8:37 am ]
Post subject:  Re: Faster volume resampling
Interesting discussion :-) As you observe the VolumeResampler does take a fairly naive approach of copying a single voxel at a time, but of course the reason it does this is to remain independant of the underlying volume type. If you have knowledge of a volumes internal representation then you can perform the copy more efficiently using memcpy().

Illidan wrote:
...I still can't try it in my game because I extract a region bigger than a block (1 voxel bigger in every direction), but maybe a getUncompressedRegion function can be added...


Yes, this might be a reasonable approach. Perhaps even LargeVolume::getRegionAsRawVolume()? As you say the logic needs some thought, but it could take advantage of it's knowledge of the internal structure of LargeVolume.

Illidan wrote:
I still have to do some proper profiling, but before I start I just wanted to ask if there's a way I can make this faster?


You say your doing this 40 times, so can you consider just speading the work over 40 frames? It'll still take less than one second and shouldn't hurt your framerate so much.

Freakazo wrote:
If I get back into coding again the first thing I'd do is try and implement correct threading for the large volume class, it's probably the only reason why I have to do hackish things like this.


I've been giving it some thought recently because it's starting to become a priority for Cubiquity. But I think I should start a new thread to discuss this in the coming days.
Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/