It is currently Sat Aug 22, 2020 4:26 am


All times are UTC




Post new topic Reply to topic  [ 32 posts ]  Go to page Previous  1, 2, 3, 4  Next
Author Message
 Post subject: Re: Incoming PolyVox changes
PostPosted: Mon May 26, 2014 6:57 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
The compiler situation is frustrating indeed. I experienced the same when attempting to move to Qt 5.2 while using VS 2012 as there was only a pre-built version for VS 2013, and building Qt was a pain. I think this is now resolved and we'll look at Qt 5 again soon.

I can hold off on the template aliasing as it is not so important, but there is a good chance that further work on PolyVox over the coming week might find more areas where VS 2012 doesn't support what we want to use. I can't say for sure until I start looking at it, and don't want to lose too much time supporting compilers we don't use.

There are a couple of minor other things which have broken VS 2012 support but I think I can fix them quite easily. I'll try and get it working again in the next couple of days.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Tue May 27, 2014 9:35 am 

Joined: Tue Apr 08, 2014 5:10 pm
Posts: 124
Qt 5.2 already have VS 2010 compiled SDK available :) From MONTHS.
They first released the vs2013 only release but week later we got ALL of them.

It's now Qt 5.3.0, it has 2010, 2012 and 2013
http://qt-project.org/downloads

Current examples that target Qt4 actually work flawlessly without modifications on Qt 5.2 :)

Still, Ogre3D is the concern here so better do not force us switching (Ogre is 2012, I CAN update to 2012 but i want to avoid 2013) - C++11 is good but should not be "Everybody likes it"

Really, I only use lambdas and std::function from C++11... and possibly something that works on VS2010 and I'm not aware it's C++11 :)


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Wed May 28, 2014 8:43 am 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Hi Guys,

Our 'vertex-and-example-refactor' branch is building again on VS 2012 and we will try to keep it that way at least for now. I think it will be merged back into 'develop' in the next week or so. It also contains some further changes to the mesh/vertex formats which you will want to know about.

At the moment, PolyVox stores both the vertex positions and vetex normals as a set of three floats. This is pretty conventional and suitable for direct rendering on the GPU, but is rather wasteful in terms of space. Storing the position and normal in this way requires six floats (which is 48 bytes), but we can do better than this.

For example, the vertices from the cubic surface extractor are positioned on a regular grid, and so we don't really need floating point values for these. If we limit the size of the extracted region to 256 voels per side then we can store our vertex position using three uint8_t variables (3 bytes). As another example, we could probably encode the surface normal for the marching cubes surface extractor using just 2 bytes (various normal encoding schemes are possible) rather than using three full floats.

The catch is that such efficient vertex formats need to be decoded before rendering. We are providing a 'decode()' function which does this and will therefore let PolyVox be used more or less as before, but we expect advanced users may instead want to decode the vertex formats on the GPU in shader code. We will provide an example of this.

This will require renaming and refactoring all vertex/mesh related classes, but the result should be much better. I also intend to limit the size of extracted regions to 256 voxels per side, so that the vertex positions can be more easily encoded with less memory. Does anyone have a reason why an extracted region would need to be more than 256 voxels along any one side? Note that you can still do huge terrains, it's just that the size of each 'chunk' would be limited to 256 voxels.

Any thoughts?


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Mon Jun 02, 2014 8:57 am 

Joined: Thu May 22, 2014 12:47 pm
Posts: 8
My thoughts as a "beginning" user:

David Williams wrote:
Storing the position and normal in this way requires six floats (which is 48 bytes), but we can do better than this.

Definitely. The lower the memory footprint, the better.

David Williams wrote:
If we limit the size of the extracted region to 256 voxels per side then we can store our vertex position using three uint8_t variables (3 bytes).

Sounds great. I don't see the need for SurfaceExtractors for regions larger than 256 voxels. And if you do, you can always split it into smaller chunks. Also, this will probably speed up the SurfaceExtractor which is still one of the most computational intense parts of my program.

David Williams wrote:
As another example, we could probably encode the surface normal for the marching cubes surface extractor using just 2 bytes (various normal encoding schemes are possible) rather than using three full floats.

I don't personally use the MarchingCubesExtractor.

David Williams wrote:
The catch is that such efficient vertex formats need to be decoded before rendering. We are providing a 'decode()' function which does this and will therefore let PolyVox be used more or less as before, but we expect advanced users may instead want to decode the vertex formats on the GPU in shader code. We will provide an example of this.

I can live with this.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Mon Jun 02, 2014 11:37 am 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Ok, great, it seems there are no objections to this. I have refactored the system as described and it seems to work well. The CubicVertex stores the position in 3 bytes and the MarchingCubesVertex stores the position in 6 bytes and the normal in another 2 bytes (8 bytes total). The 'material' member is gone and is replaced by 'data', which has whatever type the user wants (the vertex classes are templatized).

There are also 'decode()' functions to convert these efficient formats to an expanded format if desired. Most of the examples have been updated to use these decode() functions, while a new example demonstrates how the vertices can instead be decoded in GLSL to minimize GPU memory usage.

It seems that this is also a sensible time to overhaul the Mesh class (was previously SurfaceMesh). As well as removing some old code, I'm also thinking that most meshes should not contain more than 65536 vertices and so 16-bit indices should be sufficient. This will halve the size of the index buffer which currently uses 32 bits per index. The Mesh class could be templatized on index type so 32 bit indices could still be possible if desired.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Mon Jun 02, 2014 12:34 pm 

Joined: Thu May 22, 2014 12:47 pm
Posts: 8
David Williams wrote:
As well as removing some old code, I'm also thinking that most meshes should not contain more than 65536 vertices and so 16-bit indices should be sufficient. This will halve the size of the index buffer which currently uses 32 bits per index. The Mesh class could be templatized on index type so 32 bit indices could still be possible if desired.

I'd be careful with this limitation. Limiting the extraction region for a SurfaceExtractor is one thing, but limiting the extracted number of vertices is another. Imagine the worst case surface to be extracted: all vertices. Limiting to 65536 vertices will essentially limit a "chunk" to 40 (40^3 < 65536 < 41^3).

The biggest danger of this limitation is that it will not be a problem most of the times, until a complex mesh is extracted, which is a very sneaky bug.

My advice: Keep default at 32 bits, templatize to support 16 bits.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Mon Jun 02, 2014 2:36 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
You are correct that a 16-bit index means that a 41^3 mesh can theoretically fail, but only in the worst-case scenario when the voxels are arranged as a checkerboard. Such an arrangement seems unlikely in practice.

But it's hard to visualize how quickly a 'realistic' scenario would exceed the 65536 vertex limit. Perhaps it becomes easy to exceed at 64^3, or 80^3? For sure 256^3 can be a problem. So I do now agree that we should err on the side of caution and default to 32-bit. But I think we'll use 16-bits inside Cubiquity and see how fast those vertices run out.

Of course, we'll put in appropriate error handling and throw an exception if the number of vertices exceeds the max value of the index type.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Tue Jun 03, 2014 4:18 pm 

Joined: Mon Feb 17, 2014 2:27 pm
Posts: 6
I'm looking forward to the code base changes, but I was also curious about the status of the Dual contour branch. Last time I tested it the functionality still seemed unfinished (comments about speed improvement to be done in the future) and there haven't been any significant code updates since then in the branch I don't think. Is it currently considered feature complete?

If you guys are letting that branch depreciate and not merging it into the master then I'd like to know for sure so that I can start looking at a custom extractor. I have the library working as a plugin in UE4 but I'd really like some cleaner meshing than the default marching cubes extractor gives. I don't want to roll my own implementation for voxels because your library is well written and I love the concept of an open voxel library and your dedication to it.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Wed Jun 04, 2014 8:32 am 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Hi all,

Last night I merged the 'vertex-and-example-refactor' branch into develop. Here's a reminder of the main changes:

  • 'SurfaceMesh' is now called just 'Mesh' and a lot of old/dead code has been removed.
  • 'PositionMaterial' and 'PositionMaterialNormal' are removed and replaced with 'CubicVertex' and 'MarchingCubesVertex'. These encode the vertex data in an efficient way.
  • We're added 'decode()' functions to convert the encoded vertex types into a regular 'Vertex' type. You can instead do this decoding on the GPU if you prefer. We've added both cases to the examples.
  • The 'MarchingCubesSurfaceExtractor' and 'CubicSurfaceExtractor' are replaced with free functions called 'extractMarchingCubesMesh()' and 'extractCubicMesh()'. Actually the old classes are still present but will be removed in the future.
  • Works with VS 2012 again

The Mesh class has also been templatized on IndexType, but actually this process is more complex than I imagined. To make use of it the surface extrators need to know which IndexType to use, which means also templatising them on the IndexType (or more likely on the MeshType). I haven't done this yet and it needs some more thought, so you cannot use 16-bit indices yet.

mordentral wrote:
I'm looking forward to the code base changes, but I was also curious about the status of the Dual contour branch.


I talked to Matt about this recently (he's the one who actually implemented the Dual Contouring). I think our main concern it that it is not clear how a user would interact with the Hermite data which the Dual Contouring algorithm uses. With each 'voxel' the Dual Contouring approach stores a normal and distance along the edge of each of the intersection points (as I recall) and this is basically just an encoding of the mesh... do we really want the user to be reading and writing these directly? Is that actually useful functionality?

Other solutions (VoxelFarm, Upvoid, etc) seem to operate on distance fields, octrees and CSG, with procedural generation being used to create most of the content on the fly. This is quite different from what PolyVox does, which is to store an explicit representation of the volume data. It seems that the tools also become a lot more important in Dual Contouring engines, again as a result of the underlying representation being more complex. This is probably higher level than we want to deal with in PolyVox.

That said, we do have some idea for using Dual Contouring within PolyVox, but I don't think it's high priority and it needs a lot of thought/research to see how it would be used. It basically depends whether we find a use for it in Cubiquity.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Incoming PolyVox changes
PostPosted: Tue Aug 19, 2014 8:15 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Right, with another Cubiquity update out of the door I'm turning back to PolyVox for a few weeks. I've just merged a bunch of mesh changes into the develop branch. The primary changes here are that the mesh class is now templatised on index type and the surface extraction code is now templatised on mesh type. This serves a few purposes:

    1) You can create meshes with 16 bit indices (previously only 32 bit was possible).

    2) You can provide you own mesh as a target for surface extraction (rather than PolyVox allocating one). This means you could maintain a pool of meshes for memory management purposes.

    3) Rather than using a PolyVox Mesh instance, you could provide your own mesh class and PolyVox would write directly into that. For example, PolyVox could write directly in OpenGL vertex buffers or an Ogre::ManualObject, thereby reducing how many copies you have to perform.

The last one is very much *in theory* and is basically untested. We might try something like this in Cubiquity but I'm not sure yet. Note that this is all advanced functionality and you'll want a strong stomach for templates!

We've also updated the examples and the tests to demonstrate the new usages, and the tests are now much more thourough. Remember, the extractor classes and dead and you should use the free functions (extractCubicSurface() et al) instead.


Top
Offline Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 32 posts ]  Go to page Previous  1, 2, 3, 4  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
Theme created StylerBB.net