It is currently Sat Aug 22, 2020 4:16 am


All times are UTC




Post new topic Reply to topic  [ 29 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject: Re: PolyVox build dashboard
PostPosted: Thu Mar 04, 2010 12:58 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
It's no problem, I get warnings in Visual Studio too so I can fix most of them there.

It's great your doing this - I can see there are a lot of commits going through! I feel a bit bad I'm not having time to work on it myself at the moment but it should all quiet again in a month or so...


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Thu Mar 04, 2010 4:51 pm 
Developer
User avatar

Joined: Sun May 11, 2008 4:29 pm
Posts: 198
Location: UK
If I get time tonight, I might have a go at getting CMake working properly in Windows with VS 2008 or 2010RC.

Don't worry about not having much time. I'm just trying to neaten up the build system and the testing side of things. Next week I'll see about writing more more extensive unit tests.

_________________
Matt Williams
Linux/CMake guy


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Thu Mar 04, 2010 11:43 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
I have made some progress here - I used the links you provided to install the libraries and executables to the same folder on Windows. It is better now :-)

It did sucessfully submit a CDash result but it seems the tests didn't actually run. I'll look into this when I get a chance.

Actually, it only works properly in release mode. In debug mode we append the '_d' to the end of the library name and I think this is why it doesn't find it.

Do you think we should keep this '_d' thing? In some ways it is nice to manually tell debug and release versions apart, though I'm not sure I've actually needed it in practice. Some other projects do this, but on the other hand should we really care about the difference? I think they should be interchangable anyway...

Does Linux have any stabdard approach to handling release vs. debug libraries?


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Fri Mar 05, 2010 11:20 am 
Developer
User avatar

Joined: Sun May 11, 2008 4:29 pm
Posts: 198
Location: UK
(sorry for the long post...)
David Williams wrote:
I have made some progress here - I used the links you provided to install the libraries and executables to the same folder on Windows. It is better now :-)

Good, I'm glad that was the main problem and not something terribly complicated.

David Williams wrote:
It did successfully submit a CDash result but it seems the tests didn't actually run. I'll look into this when I get a chance.

I see the submission. Looking at the test report it looks like it simply didn't find the test binary. This is due to a mismatch caused by the binaries being built in a different place and the CREATE_TEST() macro I made not knowing this. I'll need to work out which variable really describes where the files are written.

I'll need to read into the various bug reports about this stuff but it looks like your problem with RUNTIME_OUTPUT_DIRECTORY not working in Windows might be fixed in CMake 2.8. Furthermore, it might get even easier in 2.8.1 (see this bug) which should be released some time very soon. If you get a chance, could you try it with 2.8 to see if you fare any better? We can always raise the required version if necessary (and will need to for VS2010 anyway).

David Williams wrote:
Actually, it only works properly in release mode. In debug mode we append the '_d' to the end of the library name and I think this is why it doesn't find it.

Yes, and also it's probably going to try to put it in a /Debug directory or something given the chance :)

David Williams wrote:
Do you think we should keep this '_d' thing? In some ways it is nice to manually tell debug and release versions apart, though I'm not sure I've actually needed it in practice. Some other projects do this, but on the other hand should we really care about the difference? I think they should be interchangeable anyway...

The two should be binary compatible, yes. I guess that really, most of the time, you simply use one or the other and don't switch between the two that often. If they're still being output to a Debug or Release directory then I guess the _d suffix isn't really necessary. It could simplify things to get rid of it.

David Williams wrote:
Does Linux have any standard approach to handling release vs. debug libraries?

In Linux we don't tend to bother with 'debug' libraries. Any library we get from our distro will have been build with the RelWithDebInfo CMake build type which sets -O2 -g. This means that it's optimised but not to the point of mangling line numbers and symbols and the -g means that debug symbols are included. Then, before it's all put into an RPM, the debug symbols are stripped out and put into a separate package. This way we can run with almost fully optimised libraries but can get the debuginfo if we really need it.

The Debug build type will just set -g so there is 'no' optimisation in order to get most accurate symbol locations. This isn't used much since RelWithDebInfo is usually fine.

Finally, Release sets the flags -O3 -DNDEBUG which obviously optimises more but also sets -DNDEBUG which turns off asserts etc.

So really the only difference between release and debug (RelWithDebInfo) builds is a little bit of optimisation, inclusion of debug symbols and the lack of asserts. Because of this I think that most people developing against a copy of PolyVox they built themselves would build PolyVox with RelWithDebInfo and then only switch to a release build if they were then going to include the library (static or shared) along with their project.

I don't know how that all differs from Windows though? You can see what would be set for the different configurations by looking in ccmake or cmake-gui at the CMAKE_CXX_FLAGS_<config> variables.

_________________
Matt Williams
Linux/CMake guy


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Fri Mar 05, 2010 6:23 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
milliams wrote:
Furthermore, it might get even easier in 2.8.1 (see this bug) which should be released some time very soon. If you get a chance, could you try it with 2.8 to see if you fare any better? We can always raise the required version if necessary (and will need to for VS2010 anyway).


Unfortunatly the other project I am working on uses VTK (and so also CMake) and I can't really risk breaking anything at the moment. I'll try upgrading as soon as I can...

milliams wrote:
The two should be binary compatible, yes. I guess that really, most of the time, you simply use one or the other and don't switch between the two that often. If they're still being output to a Debug or Release directory then I guess the _d suffix isn't really necessary. It could simplify things to get rid of it.


Maybe that's best... again when working with VTK I notice they only have one version installed at a time. That is, switching between debug and release builds involves opening the VTK solution and builing the appropriate 'INSTALL' target.

milliams wrote:
In Linux we don't tend to bother with 'debug' libraries. Any library we get from our distro will have been build with the RelWithDebInfo CMake build type which sets -O2 -g. This means that it's optimised but not to the point of mangling line numbers and symbols and the -g means that debug symbols are included. Then, before it's all put into an RPM, the debug symbols are stripped out and put into a separate package. This way we can run with almost fully optimised libraries but can get the debuginfo if we really need it.

The Debug build type will just set -g so there is 'no' optimisation in order to get most accurate symbol locations. This isn't used much since RelWithDebInfo is usually fine.

Finally, Release sets the flags -O3 -DNDEBUG which obviously optimises more but also sets -DNDEBUG which turns off asserts etc.

So really the only difference between release and debug (RelWithDebInfo) builds is a little bit of optimisation, inclusion of debug symbols and the lack of asserts. Because of this I think that most people developing against a copy of PolyVox they built themselves would build PolyVox with RelWithDebInfo and then only switch to a release build if they were then going to include the library (static or shared) along with their project.

I don't know how that all differs from Windows though? You can see what would be set for the different configurations by looking in ccmake or cmake-gui at the CMAKE_CXX_FLAGS_<config> variables.


I don't think it's that different, I've seen a couple of articles recommending that you even ship your final version with debug info included. The performance overhead is supposed to be small, and the actual debug info is actually in a seperate file (.pdb I think) so it doesn't add much to the file size. But ultimatly we are only making this choice for any binaries we ship - an end user building from source code can choose which ever build type they want.

That said, I do find on Windows that debug builds are *a lot* slower than release builds (by an order of magnitude maybe). Don't know about release with debug info. I have been assuming the reason is the large number of asserts in the PolyVox source code. There are asserts to validate the parameters every time a voxel is read or written, and this happens a lot. Of course, it's a good thing from a robustness point of view but if asserts are the performance issue then we should make sure that ReleaseWithDebugInfo doesn't include them.

I'm away for the weekend now, I'll pick this up when I return...


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Mon Mar 15, 2010 10:06 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Well, the CDash dashboard is useful! I forgot to commit a file last night, and this morning I got an email telling me the Linux build had failed. All fixed now :-)


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Tue Mar 16, 2010 4:27 pm 
Developer
User avatar

Joined: Sun May 11, 2008 4:29 pm
Posts: 198
Location: UK
David Williams wrote:
milliams wrote:
Furthermore, it might get even easier in 2.8.1 (see this bug) which should be released some time very soon. If you get a chance, could you try it with 2.8 to see if you fare any better? We can always raise the required version if necessary (and will need to for VS2010 anyway).

Unfortunatly the other project I am working on uses VTK (and so also CMake) and I can't really risk breaking anything at the moment. I'll try upgrading as soon as I can...

Ok, well there's no rush. Everything's working just fine in Linux with 2.6 so it's only Windows that would be affected.

David Williams wrote:
milliams wrote:
The two should be binary compatible, yes. I guess that really, most of the time, you simply use one or the other and don't switch between the two that often. If they're still being output to a Debug or Release directory then I guess the _d suffix isn't really necessary. It could simplify things to get rid of it.

Maybe that's best... again when working with VTK I notice they only have one version installed at a time. That is, switching between debug and release builds involves opening the VTK solution and builing the appropriate 'INSTALL' target.

Well I see you've made this change now. It has no effect under Linux since it doesn't use the CMake 'configurations' thing but I assume it's going fine under Windows?

David Williams wrote:
milliams wrote:
In Linux we don't tend to bother with 'debug' libraries.
<snip>

I don't think it's that different, I've seen a couple of articles recommending that you even ship your final version with debug info included. The performance overhead is supposed to be small, and the actual debug info is actually in a seperate file (.pdb I think) so it doesn't add much to the file size. But ultimatly we are only making this choice for any binaries we ship - an end user building from source code can choose which ever build type they want.

The .pdb thing sounds almost exactly like what Linux does so that simplifies things. When we get round to another binary release I'll make sure they're built and handled correctly in the CPack installer.

David Williams wrote:
That said, I do find on Windows that debug builds are *a lot* slower than release builds (by an order of magnitude maybe). Don't know about release with debug info. I have been assuming the reason is the large number of asserts in the PolyVox source code. There are asserts to validate the parameters every time a voxel is read or written, and this happens a lot. Of course, it's a good thing from a robustness point of view but if asserts are the performance issue then we should make sure that ReleaseWithDebugInfo doesn't include them.

I haven't done any profiling/benchmarking under Linux to compare debug to non-debug but I would imagine it would be the same thing. Having a full debug version (with asserts) around for testing purposes would be useful but I guess for day-to-day developing, RelWithDebugInfo should suffice.

In another post David Williams also wrote:
Well, the CDash dashboard is useful! I forgot to commit a file last night, and this morning I got an email telling me the Linux build had failed. All fixed now :-)

Excellent. I'm glad it works as expected.

_________________
Matt Williams
Linux/CMake guy


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Wed Mar 17, 2010 7:09 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
milliams wrote:
David Williams wrote:
milliams wrote:
The two should be binary compatible, yes. I guess that really, most of the time, you simply use one or the other and don't switch between the two that often. If they're still being output to a Debug or Release directory then I guess the _d suffix isn't really necessary. It could simplify things to get rid of it.

Maybe that's best... again when working with VTK I notice they only have one version installed at a time. That is, switching between debug and release builds involves opening the VTK solution and builing the appropriate 'INSTALL' target.

Well I see you've made this change now. It has no effect under Linux since it doesn't use the CMake 'configurations' thing but I assume it's going fine under Windows?


Yes, but actually it seems that the debug and release version are not directly interchangable. If you build PolyVox as release/debug then you have to do the same one for Thermite. And actually I'm finding the same thing when working with VTK. It's not really a problem at the moment but I'll look into it at some point

milliams wrote:
In another post David Williams also wrote:
Well, the CDash dashboard is useful! I forgot to commit a file last night, and this morning I got an email telling me the Linux build had failed. All fixed now :-)

Excellent. I'm glad it works as expected.

Actually, having to fix the website after the attack reminded me that we have more than a webserver there. It's a general Linux machine really. So if it is a problem for you to be running the CDash nightly builds from university then we can probably do it from the webserver instead. Not sure if we can run tests though. But if it's not problem then it's probably easier to stick with the existing solution.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Mon May 24, 2010 9:49 pm 
Developer
User avatar

Joined: Sun May 04, 2008 6:35 pm
Posts: 1827
Ok, as you've already noticed I've finally been back on this. My contract work isn't quite finished but it much quieter now so I've got some time again.

I've updated my system to Windows 7 and Visual Studio 2010. This means I was finally able to remove the awkward C++0x compatability stuff as it builds properly on VS 2010 and GCC. I've also updated to OgreSDK 1.7.1 and took the latest version of Qt from Git (heading towards 4.7 now). Also, my CMake is now on 2.8.1. Just a few more bits and pieces to do to get everything running smoothly again.

Also, I've done some work adding scripting support to the QtOgreFramework. It's very preliminary (I'm still working out how stuff fits together) but I've got keyboard input and wrapped the Ogre::Vector3 class. The camera can now be controlled through QtScript :-) There's lots to do here but it looks promising.

I'm running OpenSuse 11.2 in VirtualBox so I was able to verify those fixes you made (thanks for that!). I was going to fix it myself but you got there first. Shows that CDash is working as expected :-) I'll try to check the QtOgre framework works under Linux after I've made my scripting changes.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: PolyVox build dashboard
PostPosted: Tue May 25, 2010 2:55 pm 

Joined: Mon May 24, 2010 7:03 pm
Posts: 1
This is very good option. It provides many great features and functionalities. It works well. There is no problem related to any platform.

_________________
r4 gold


Top
Offline Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 29 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
Theme created StylerBB.net