Yes, I would expect so.



It will certainly be interesting to try but there's a couple of things to keep in mind. Firstly a surface extraction is usually very fast ( a few ms) so I don't know whether benefits will outweigh the cost of starting/syncing the threads. I've literally got no idea how long these overheads take, but a thread pool should help.

Secondly, although you may be able to process each cell individually you will probably hit syncing issues when generating the single mesh. Ideally vertices should be shared between adjacent triangles. But you may be able to have each thread write separate mesh structures and them combine them all at the end.

It will be interesting to see how it plays out anyway.

Hey guys long time since I've seen any news about PolyVox specifically.

I cannot express my gratitude towards you two with mere words.
You both have put fourth such an effort to bring about a truly amazing library. I'm sure everyone would have a similar opinion.

The main 3 things I'm personally interested in are as follows:
Header only implementation.
Thread-Safe API.
Better LOD support.

I've recently found some very detailed videos on the standard thread library. They cover most advanced topics and is presented in a quick efficient manor. It may help you guys in your endeavors!

As for LOD, personal opinion is it should be left up to the programmer to actually implement the bulk of LOD themselves as it could be handled in various ways depending on the given situation. However PolyVox may be able to make lives easier by extending the SurfaceExtractor's to instead of giving us vertex data in raw form, maybe give us a QuadTree of the vertex data which would then be quite trivial to implement a LOD algorithm.

Well that's just my two cents, keep up the fantastic work! <3

_________________
Dream Big Or Go Home.
ENet - http://enet.bespin.org
Recast - https://github.com/memononen/recastnavigation
Irrlicht - http://irrlicht.sourceforge.net/forum
PolyVox - http://www.volumesoffun.com/phpBB3/
Help Me Help You.

Thanks! It's really great to know that PolyVox means so much to people. You are right that development is a little slow right now, but that's because we're working so hard on Cubiquity. I think we should be releasing it in just 1-2 weeks so then PolyVox will get some more attention (it is at the core of Cubiquity after all).

Did you mean to link them? Sounds like they could be interesting.

LOD can basically be performed in two ways, either by downsampling the volume data or by decimating the extracted mesh. We have much more experience working with volume data rather than mesh data and that shows in what we have been able to achieve. We have volume based LOD working fairly well in Cubiquity (see here) but implementing mesh decimation did not go so well in the past.

Looking to the future, I think we would like to include more volume processing code (resampling, blurring, etc) in PolyVox, but leave mesh processing to other external libraries such as OpenMesh. This seems like a sensible separation of responsibilities anyway.

Oh I did mean to link them whoops! A lot going on in my brain >.<
https://www.youtube.com/playlist?list=P ... 0hTQoJL74M

The only reason I mention LOD is for the obvious fact that a LargeVolume can output millions and millions of verticies. For this reason a Quadtree would be optimal but an Octree would work also.

I imagine it would go something along these lines:
Create the PolyVox volume and fill it with some data.
Create a PolyVox Octree 'container'
Preform SurfaceExtractions on specific parts of the volData straight into the Octree.
--Surface extractions would basically look and feel exactly the same but instead of extracting into a PolyVox Mesh you extract into a PolyVox Octree, the appropriate nodes would be updated accordingly adding/removing them if needed
Users can then query the Octree with their own methods to get the vertex/index data they need to create a mesh.

Just brainstorming ^.^

_________________
Dream Big Or Go Home.
ENet - http://enet.bespin.org
Recast - https://github.com/memononen/recastnavigation
Irrlicht - http://irrlicht.sourceforge.net/forum
PolyVox - http://www.volumesoffun.com/phpBB3/
Help Me Help You.

I will add about threading.

Currently in my multi-RawVolume (with all of its implications) engine, I have mutex locking on the chunk sections and smart setter/getter stacking.

I lock() the chunk and then run the surface extractors / voxel data modifiers on this chunk from other threads.

If another thread calls my ChunkManager::setVoxel(x,y,z,..., bool important) with important=true, and the target chunk is lock()-ed, the operation is added to voxel updation map and performed in first gametick when target chunk is no longer locked. This way I ensure updation stays in sync.
Due to the fact every chunk have its own RawVolume, no threading races occur in PolyVox, but handling all the overlapping cases AND locked/unlocked cases is tricky.

You can consider similar locking pattern with LargeVolume and region locking. A thread can lock or as you named it, pina region for exclusive access and not to be unpaged, and other threads should try_lock() / wait for unlock state before attempting to modify the locked region.

Proper thread-safety on LargeVolume will definitely help me A LOT, because with Multiple Volumes I have to handle several issues myself, most notably:

[*] Overlapping 2 voxels is essential for proper normals,
[*] Updating 1, 2 or 4 chunks in case or border voxel coordinates,
[*] A* Path finding requires custom A* pathfinder class (I copied the one from PolyVox and modified it to use the Chunk Manager's accessors and lock-waits, which degrades performance, but as compensation it's thread-safe.
[*] Just Volume-per-chunk maybe isn't the ideal solution, I should implement volume-per-region approach and a region will have chunks, and chunks shuold have chunk sections (ehh, Minecraft...). This way setting and getting data from volumes will require fewer locking, but Surface Extraction may not work flawlessly / well paralelised. I have to experiment how to balance the whole system. BUT With current implementation, Number of CPUs REALLY help! People with Core i7 definitely get the best performance here because of paralelism, and people with single-core CPU s*** ****s but as the target are gamers we can safely assert powerful multi-core CPU here

And real-life games we will definitely NEED to have a chunk manager anyway.

I am using boost mutexes here as Ogre already forces us to use Boost anyway.

p.s. my Chunk Manager's source code will soon be committed to a SourceForge repo.

Thanks, they look good. I usually read text instead of watching videos but these ones are nice and short. Bookmarked.



These are some interesting ideas, but maybe the octree would build on top of PolyVox rather than being part of it. Actually, we do something like this in Cubiquity. Cubiquity does maintain an octree and each node covers a certain region. We pass each region to a PolyVox surface extractor to generate the mesh for that region, and store the mesh in the node if one is generated (if it's non-empty). This works not just for leaf node but also for internal ones, which is where the LOD comes from.

It's a bit messy at the moment and we haven't yet exposed it (probably not for a few months) but in principle there is something here. In this sense Cubiquity is serving as a nice testing ground for PolyVox, and PolyVox can/will be adjusted to make these things easier.



Yes, I think this is what we will do. The threading isn't the first priority though - after we release Cubiquity we first need to merge across the changes we made to PolyVox (the interpolation improvements discussed in another thread). Then I think we will simplify the LargeVolume to remove the different compressed/uncompressed states as these complicate it needlessly. Then I think the threading will happen :-)

Right, this thread has been quiet for a while but it's finally time for some more LargeVolume improvements. Top of my list is to remove compression of blocks which are held in memory.

Blocks (chunks) of voxels in the LargeVolume currently exist in one of three states. The can be stored compressed on disk, stored compressed in memory (a kind of cache), or stored uncompressed in memory (ready for voxel access). The Pager class is responsible for loading compressed chunks into and out of memory, and the Compressor class is responsible for compressing and decompressing them.

The code for this three-level hierarchy is surprisingly complex, which makes profiling and threading difficult. I also think it's unnecessary, as the compressed blocks in memory were supposed to be a cache of recently used uncompressed blocks, but in practice all filesystems and databasing provide caching anyway. Lastly it makes procedural volumes difficult, because a user has to generate the volume data, then compress it to pass to PolyVox, and then PolyVox immediately decompresses it again.

I am proposing to remove the middle 'state', so that block go directly from being compressed on disk to being uncompressed in memory. The LargeVolume will have no concept of compressed blocks - it will operate on uncompressed blocks and will page uncompressed blocks out to the user (of course, the user can then compress these if desired).

Let me know if you have any thoughts or questions.

Yes I think this is a good idea. I remember constantly running into a bug where PolyVox would assert when trying to procedurally generate my volume.

I don't believe compression in memory is needed as long as we can easily tell PolyVox to load/unload a specific region from the volume in/out of memory.

_________________
Dream Big Or Go Home.
ENet - http://enet.bespin.org
Recast - https://github.com/memononen/recastnavigation
Irrlicht - http://irrlicht.sourceforge.net/forum
PolyVox - http://www.volumesoffun.com/phpBB3/
Help Me Help You.

Yeah, I'm also hitting this myself sometimes and the code is a little too complex to see what is really going on.



Indeed, and of course the user can still choose to compress data once it is paged out of PolyVox.

Time is ticking, my engine too

I dropped LargeVolume completely. Now my chunk manager is working with flat 1D arrays with 3D indexing for blazing-fast random access, because for me, the compressing and uncompressing nature of LargeVolume turned to be awful and serious bottleneck. I reworked my old Multiple RawVolumes per chunk implementation, but replaced RawVolumes with flat arrays and removed the overlapping of the data.

When I extract meshes, I create RawVolume from the chunk manager and extract this - and now my light spreading works far more better than expected.... for now, I hope that the Colored Lighting will not become the next overkill