I knew I didn't do a good enough job explaining myself...

Basically the idea boils down to this - I don't want a visible limit on my world, even from atop a mountain or something, so no matter what I do, I'm going to need multiple volumes. I'm going to hide every single unexposed cube from memory - like it doesn't even exist - unless it happens to be on a nearby volume.

This solves 2 problems (in theory?) One it will reduce memory consumption on ultra huge rendered areas (think like 2k blocks in any direction), and the other is that it will remove those skirts on the edges of volumes from being rendered more than a block deep, because if a block isn't visible, then the volume you are asking polyvox to get the surface of doesn't even include it, therefore it won't give you it's surface. The only time you ever need those unexposed cubes is when you remove a block that is covering one up... which should usually be local - your local volume - so then you just have the entire local volume in memory (including unexposed cubes) and only the exposed cubes in memory for the rest of the volumes. Assuming it's not a checkerboard style map with 1 cube holes throughout your mountains or w/e this would save a ton of memory.


I do like your idea of having your volumes move... maybe I'll try something like that myself, that way you only need to worry about the "current" volume, and it could be much larger... implementing this seems pretty tricky to me because of all the edge cases you have to consider, but once done it should save a lot of headache.

Okay let me get this straight (dude it's pretty late so bear with me ha), when you say "hide unexposed cube from memory" do you mean the actual data in the volume or the surface extracted? I went back and reread what you wrote earlier, at first it sounded like you meant the extracted surface, if so, PolyVox does this already, it won't extract any surfaces that are not exposed to air. If you mean actually not even bothering generating them, sort of like just in time generation--that's interesting, even though I didn't really get the description of your plan. Edit: If you want to do this with PolyVox it would be kind of hard because--I think--Volume preallocate all the blocks on creation. It's only when you call tidyMemory() that it compresses blocks. So the only way I could think of doing it is using an octree, in order to do on-the-fly compression. Don't quote me on this though.

Can't say I really thought about it like this, except using an octree or some other compression methods.

As for saving, yes, I thought about this, saving the deltas only. Does Minecraft do this, only save deltas? I was under the impression that he did not that he saved entire chunks, instead.

For skirts, I don't see you can mark it as unexposed without check what is beyond the the edge. What the surface extractor does, is it looks at neighbor voxel, and extract the appropriate surface depending on threshold/type. For skirts at border there is no other way around it UNLESS you let it access what is beyond the volume to neighboring volumes.

Also, moving the regions inside a single Volume is not possible, I was thinking of paging earlier and this is not possible with unmodified Volume class because they are basically array structures, i.e. moving would mean copying memory. So I guess my translation will just have to create new volumes as pages come in. I can also get rid of the skirts entirely because I can give my custom extractor access to neighboring volumes. I simply have to treat edges separately from interiors.

This is what hiding unexposed cubes would do for you. That is the entire point. If you never tell the volume about them, then the extractor will never extract anything for them. Therefore your skirts would only ever be one block deep per exposed cube on the edge of a volume (not 128).

However, there is actually a HUGE problem with my idea, and that is that you will actually probably double the number of worthless triangles drawn without backface culling, because now your surfaces are all 1 cube thick, and the surface extractor will give you all those back sides, which doesn't really save anything. Your modified extractor is definately the best way to do things... at this point I think investing time into making 2d RLE and a surface extractor that will take as input N number of volumes is the best you can do.

I guess I need to go back and read, but is compression done on a volume level?

Actually. Even that I take back - there is a ton of stuff that could be optimized to make a minecraft world. Hell minecraft is actually pretty terrible when it comes to this, it does draw all the tunnels and stuff. There has to be a decent way to not waste so many resources doing that. Enough of this for now... I'll think about it today.

But how would you hide it? When you extract the surface on the border, you have to know what blocks are surrounding it, and without knowing this, you have to assume what is beyond the border, to get the correct (visual) results you have to assume it's air. If you don't, you may have holes in your terrain.

I'm just not understanding hiding it. The regular extractor already hides surfaces it it's surrounding voxels are all the same type. If some are air, then it only extract the face touching air.

You could wrap around, so that if one chunk is up against the side of the volume, then the next chunk could just be placed in the other side of the volume. Afterall, your basically just using the volume as a scratchpad for the process of surface extraction. I'm not saying this is a good idea (or even that it works!) it's just something to consider.



Compression is done but not much. The volumes is stored as a collection of blocks, and those blocks which are all one material are shared. This seems to compress the volume down to something like 25% of it's original size (of course, this depends a lot on the volume).

But I have been thinking about this, I think it would be desirable to actually compress the contents of the blocks, at least for those blocks which haven't been touched for a while. I believe blocks will easily compress down by a factor of 100 or so - but there is a trade off between good compression while also allowing the data to be modified quickly.

I'll probably make a change like this, but I'm not sure how soon.

Thinking about it, if you want to do this then you'll also need to make the VolumeSampler wrap around when it reaches the edge of a volume. And also the various 'peek' functions. Hmmm... could get tricky...

I recently implemented compression for volumes which get saved to the harddisk. I used ZLib and depending on the volume I have I can get some really good compression. My volumes are 32^3 voxels big with 1 Byte each, so each volume is 32 KB in memory. When stored to disk they are below 1 KB. I used the standard ZLib compression. I'm not sure if the speed will be good enough though.

Yeah, I've seen similar compression rates when storing volumes in ZIP files (is that the same algorithm as ZLib, I'm not sure?). The only catch is that it needs to be stored in a way that it can still be easily modified at run time, but I think that's possible.

I know nothing about the actual algorithm. However one can choose between best compression or best speed when running it, so maybe volume compression with ZLib for example is a viable option.