I'm wondering about thread safety issues for Volume in PolyVox. What I'm trying to do is instead of having individual chunks as Volume objects, I want a single large Volume divided into chunks. But I'm currently doing things using threads, both generation and surface extraction. Right now, both are done in the same job (Ogre WorkQueue tasks). My plan is to separate the two. So my question is, if when I'm updating Volume, but my region is such that they do not overlap, do I still need to worry about thread safety issues in this case? Well I know I should probably look in the code myself, but I haven't begun working--refactoring my current code WRT chunks & volumes--on this yet so I thought I'd ask here first.

Thanks.

Yeah, interesting question. I haven't given a lot of thought to thread safety in PolyVox (it's not really an area where I have much expertise) and I haven't made use of any threading features (which are pretty limited in C++ anyway) so the short answer it that it generally can't be considered thread safe.

That said, I'm already using threads in Thermite for tasks such as the surface extraction which require only read access to the volume. This is working pretty well, and was one of the reasons why PolyVox has that slightly strange approach of building a SurfaceExtractor object and then calling the 'execute()' method. This allows to to build the object on the main thread, pass it to a different thread (the SurfaceExtractor carries all the information it needs) and then call 'execute()' from the seperate thread. It's working pretty well for me, and I plan to apply the same idea to new algorithms like pathfinding, but I'm not sure how conventional it is...

For your specific case of both reading and writing to the volume at the same time (though different parts of it), the main thing you need to watch out for is the block sharing. When a block is completly one material (homogenous) PolyVox will share it with other blocks which have all the same material. This allows the compression of large volumes of empty space, for example. But if you change a voxel in a shared block then PolyVox has to unshare it and allocate it a new area of memory. This will be bad if you are trying to read from the area of memory at the same time...

In general, if you don't read and write to the same block at the same time (and don't call tidyUpMemory()) you will probably be ok. But I can't be sure, and of course you'll be relying on an implementation detail of the Volume class which isn't desirable.

Thanks. Yeah I don't want to rely on implementation details either. I guess I will stick with a volume for every chunk. Now that I think about it, the overhead won't matter all that much. Maybe I'm wrong but I will try this first and worry about it later. What I will do now is for my surface extractor I will just have to allow access to nearby chunks for reading.

Sorry, in this context I was talking about if you are reading at writing at the same. If you read and write at the same time, but do it to different blocks, the you will probably be ok but I can't be sure and I wouldn't recommend it.

If your only reading (e.g. while surface extracting), then I'm pretty sure that it's ok to do it simultaneously from multiple threads.

Maybe that's what you understood anyway, but I wasn't very clear.

Got it. Maybe what I'll do instead is just stitch the borders together. They will both generate shared borders together. It's does some extra work but won't generate extra faces on those shared borders if they are the same type of material, which is what I want in the end. The trick is to figure out how to do the stitching properly.

Well anyway, I'm not thinking clearly lol. I'm really fill rate bound at this point. If I have my SSAO stuff off then I get 200+ fps, it's only when I turn it on I get around 48-50ish for some views. Dang it. I should try to reduce fill-rate. Maybe look into deferred shading.

I reverted to an earlier version and I can say the frame rate gain is about +70fps with SSAO off. With SSAO on it's the same.

I guess I will worry about this later. I had some problems with this "border fix" still.

I don't really have a nice way to test my code right now so I'm just basing it on frame rates, and frame rates with FRAPS at that, no data collection whatsoever. Which is pretty bad.

I know I should worry about other stuff it's just I can't stand these massive exposed faces on the border of every 32x32x256 chunks.

Update:

OH BTW, when I say "try deferred shading" I mean deferred shading plus switching to SSAO using deferred shading. The way I'm doing SSAO right now it has to render my geom in two passes (so computing the normal in screen space two times!!!). So hopefully, with deferred shading I can reduce some of that overhead.

I'm not 100% sure if this is what your talking about... but I'm going to try and contribute, because I think it is a problem I've been thinking about. When you get the surface from surface extractor it is the entire surface, but really you want only as much as you can see. A little extra isn't really a problem, but you don't want the entire sides being drawn if they are going to be covered up by another surface extracted mesh. The solution I was thinking of was to use an "exposed" flag in my internal data structure. That way, when I ask for surface extraction, I only ask for the surface of "exposed" blocks. Everything with neighboring solid blocks is not exposed. I'm actually fairly certain this is what minecraft does, because if you get those lag rendering where you see the tunnels pop in first, then other stuff, it looks like all blocks with one face facing air are drawn, but all the rest of it is not.

This solution obviously does not take care of 100% of what you would ideally get rid of, but it probably takes care of ~80-90% of the problem. The cost of raytracing to see if a tunnel is visible or not is probably not going to net a savings, but who knows, I could be wrong especially if the app is seriously graphics limited.

NOTE: obivously a flag is not nessessary, you could also do 6 if statements checking for a block at each of the 6 faces, but I'm not sure that is that great of an idea, as that is 6 x number of blocks in a chunk (6 * 36*36*256 ~= 20 million if statements...) checks per update, and you have to update every single time there is a modification... might be worth a bit per cube to just do a fetch instead of the 6 ifs, no idea on the reality of that though.

Afterthought: also, I guess if you're saving chunks in data structures in memory then maybe you could generate the flag when you pull the chunk from disk - you only really need to save the currently active chunk as a full data structure (with all the flags and crap) because all the other chunks that are part of the terrain which are for all intents and purposes static only need to be computed the first time. (I don't know if I'm wording this well enough) But the basic idea is that you only need the current chunk, and you don't even have to have a flag, you would just need to check the 6 blocks around the modified block... so then all you have to do is ask your data structure for 6 blocks, modify, and re-extract. Basically this all revolves around having 2 sets of data for the current chunk, one set with only exposed surfaces, and another with every single voxel (which you would only ever care about every single voxel when it is possible to modify one of them, which is usually just going to be in the local chunk(s)). This would seriously benefit memory usage in large environments, because you prune out all the voxel data you don't need when it isn't going to be used.... when you walk onto another chunk then you can use another thread to populate the data structures with full voxel data again.... Actually I'm pretty sure this is what i'll try and implement, the more I describe it, the better I like it -

I will just have the entire world ask for exposed voxels, save those surface extracted meshs, and then not worry about them. I will then have in memory all the voxels in the chunk my char is on, and every chunk around him. This will serve as a fast way to check what to update for my surface, get a new surface, and bam, it will all be taken care of super efficiently. Also, seeing as disk space is pretty much free (you have so much disk space adding an extra byte to every voxel won't matter at all) I will save my voxels out to disk with a flag if they are exposed or not, so I won't have to check when loading, only when modifying - all this will require is a quick update to disk every time we change anything. I think this is the best system...

EDIT: there is no way to fix this post to be more coherent, besides re-writing it, because I basically thought an idea through and refined it as I wrote... sorry about that, if you have any questions ask me, and I'll try and clarify and maybe even draw a picture.

If you have a volume and you run the surface extractor on two neighbouring regions of the same volume, then it does not incorrectly generate triangles between them. The surface extractor only generates polygons when a solid voxel is next to an empty voxel. The exception to this occurs when the region for which you are generating the mesh goes over the edge of the volume. In this case PolyVox can be instructed to consider the area outside the volume as being solid or as being empty, and it will'close off' (or not) the resulting mesh as appropriate.

If you are unsure about exactly what triangles are being generated then remember you can switch Ogre to wireframe mode by putting 'polygon_mode wireframe' in your pass.

As I understand it, the problem beyzend is having is because he has multiple volumes making up the world, rather than having one larger volume and extracting multiple regions from it. He is doing this because he then pages the volumes to and from disk, in order to support infinate volumes. But, as described in my previous paragraph, he then has the issue that the surface extractor goes over the edge of the volume and closes off the resulting mesh. This gives extra faces between the volumes which are alweays hidden.

The ideal solution would be to create a new volume class which handles the paging of data to/from disk internally. This would mean the volume could be created to be any size, and the surface extractor wouldn't need to know it was any different from a regular volume. Unfortunatly I currently have no plans to implement a paged volume as I'm not expecting to go beyond volumes of 1024x1024x1024 or so. And at this point I'm finding rendering to be more of a problem than memory.

@Paycheck I'm only having the exposed border problem like David mentioned. Or are you talking about skipping non-exposed faces entirely? Well, I thought about using an octree for that, but decided it's not a top priority right now, and most likely I probably don't even needed unless I go the extreme on cutting down memory usage. Really the problem was that I was "tripping" about the symptoms of low frame-rates and not the cause. Basically going oh crap, 40fps!!! THESE BORDERS are killing me!!! AHHHHH!!! Oh wait... SSAO is on.

Anyhow I will still fix this because now I need to compute some light propagation (exactly like minecraft) so it's easier to have a single large volume. The plan is to write a translation layer between Ogre's paging system and Volume. It will just continuously page into this single volume.