It is currently Sat Aug 22, 2020 2:02 pm


All times are UTC




Post new topic Reply to topic  [ 15 posts ]  Go to page Previous  1, 2
Author Message
 Post subject: Re: Multithreaded Volume loading/unloading
PostPosted: Fri Apr 15, 2011 8:51 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Sounds interesting. I didn't try the code but I did skim over it (and it's great to see you cloned the Git repository :-) ) So you haven't touched the internals of PolyVox at all, this is just a wrapper class which sits around the main volume and a second one? It sounds ok in principle, but does this mean you need to create a new volume for each thread you have running surface extraction?

Anyway, I'll be interested to see where this goes...


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Multithreaded Volume loading/unloading
PostPosted: Sat Apr 16, 2011 5:10 pm 
User avatar

Joined: Wed Jan 26, 2011 3:20 pm
Posts: 203
Location: Germany
I modified the volume class to have a block-copying function and a region-copying function.
so I did some internal changes. but I didn't need to change any existing functions. (I use getUncompressedBlock)

about the 2 volume solution:
the second volume inside the thread is only used for extraction.
the number of loaded blocks can be decreased significantly (like 10x10x10 blocks which should cover most of the actively changed area)
also it doesn't need RLE, because there's no point in having RLE for such a small volume, especially if we want to extract stuff all the time)

yes there will be one extra volume per thread.
but I don't see where this should ever be a problem, since the memory overhead is quite minimal.

also, for anybody reading this: do NOT use the class I posted, it is not fully thread safe yet, it will crash at some point randomly


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Multithreaded Volume loading/unloading
PostPosted: Sat Apr 16, 2011 6:05 pm 

Joined: Sat Sep 18, 2010 9:45 pm
Posts: 189
Is there any specific reason to use a Volume for your buffer to copy from in the loading threads instead of PolyVox::Array?

What do you think the performance penalties are? I guess the penalties are copying and I can see it being relatively small.

I'm still on the old system so I can test it there first. I really don't need to thread safety now since I decoupled generation from extraction which resolves the thread safety issues I was having earlier. I guess I will test it here first to see what the timings are like.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Multithreaded Volume loading/unloading
PostPosted: Sun Apr 17, 2011 6:17 am 
User avatar

Joined: Wed Jan 26, 2011 3:20 pm
Posts: 203
Location: Germany
well... you can't pass an array to the surface exctractor...
rewriting the surface exctractor, to allow this, might actually be a really good idea.

so I understand your setup is that your main volume is an array (obv no paging) and you copy parts of it to a volume that then is used in a thread to extract the surface?

I just thought if volumes were used everywhere, it would be the most practical, as no changes to any methods/algorithms would be needed.
does this make sense?

or what are the advantages of the array method?


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Multithreaded Volume loading/unloading
PostPosted: Sun Apr 17, 2011 8:02 am 

Joined: Sat Sep 18, 2010 9:45 pm
Posts: 189
Nope. My setup is I have a main volume from the old polyvox before paging, then I just send PolyVox::Regions to different threads (I have a task-based threading system) and generating them there completely ignore any thread safety issues (LOL throwing caution to the wind). It worked okay up to this point--I assume--is because I don't have any over lapping memory access in the threads--my regions matches with the block dimension inside volume. I always thought that since it worked, I will just leave it alone and make it thread safe when I get around to it.

I'm in the middle of something here but once I get it done I will probably move over to the paging system.

As for the Array thing, oh yeah I didn't think about that extractor don't work with Arrays. The only reason I thought it may be better is because Array have less overhead. I'm not sure if that is true. I also wrote some accessor class the wraps Array, to do things like range checks.


Top
Offline Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 15 posts ]  Go to page Previous  1, 2

All times are UTC


Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
Theme created StylerBB.net