Wednesday, 8 February 2012

Slow compile times in C++

I just thought I would put a short post on this, to do the world a favour.. probably not relevant to most people but c++ programmers might find this post on google.

Is your c++ project taking ages to compile? Do you have a brand spanking new machine but it takes tens of minutes to compile your projects?

This problem is a pet hate of mine, and something that (should) become apparent pretty quickly when working on big projects / using third party sdks. However I'm constantly surprised at the amount of people who are oblivious to this problem, or perhaps can't be bothered to change their working practices to take account of it.

The reason for slow compilation is, in 99% of cases, not due to the size of your project, or the number of files (maybe there's some freudian reason why some guys might like to impress with the size of their project by the slow compilation). No. Most half way modern computers can zoom through opening and processing 100s, or 1000s of files.

The problem is due to something called NESTED INCLUDES.

You know those statements you use in your code: #include

#include in a .cpp file : GOOD
#include in a .h file : BAD

The reason for slow compilation is for each .cpp file there is a chain of dependencies. In order to compile that .cpp file, it has to open and look through EVERY header file that is referenced by it, either directly OR INDIRECTLY, through massive long chains of header files including other header files that include other header files that reference other header files that contain things that aren't even needed for the original header file etc etc ad infinitum!!!

You can actually view this in some compilers, I think there is a switch /showincludes that may be available.

A lot of people simply give up, and use the compilers best bodge to rectify the situation: precompiled headers. While precompiled headers can get around a lot of the problems, particularly in third party code, they don't work for your own code, unless you put loads of your own headers in them, in which case it has to recreate the precompiled data everytime your change them, which kind of defeats the object.

A better way is to understand why the problem exists, and come up with slightly altered coding techniques that avoid it almost completely.


For fast compilation the name of the game is to STOP THE CHAIN of header includes, as soon as possible down the dependency line.

This means, absolutely not having #includes in header files unless absolutely necessary. And, where necessary where you must #include in a header file, make sure to #include something that doesn't itself have lots of dependencies.

IN YOUR OWN CODE

Instead of #including in a header file, as much as possible use FORWARD DECLARATION instead.

For instance, consider the example class CChicken:

class CChicken
{
public:
CLegs * m_pLegs;
CWings * m_pWings;
CHead * m_pHead;
};

You may be thinking 'It doesn't know what CLegs, CWings and CHead are! I need to #include them!!'. If you did you would be wrong. You don't need to #include them, you just need to forward declare them. Add this where you would normally put your #includes:

class CLegs;
class CWings;
class CHead;

class CChicken
{
public:
CLegs * m_pLegs;
CWings * m_pWings;
CHead * m_pHead;
};

It will then compile happily. Then when you come to actually USE the classes in the cpp file, put the #includes in the cpp file, NOT the header! :) The compiler actually doesn't need to know the definition of the class until it needs to know how to use it. If you are just using pointers to the class in a header file, it doesn't need to have the definition, so forward declaration is sufficient.

In this example, we have saved the processing of 3 additional files every time CChicken is compiled, and also EVERY TIME that CChicken.h is included in another place in the code. That is, we have (hugely simplified) speeded up compilation by 4x.

Now consider if CWings.h #included other header files itself! And if the h file included by CWings itself includes more header files. Quickly the chain can become astronomically large.

But but using foward declaration we can break this chain.

Sometimes however you can't use forward declaration, you need to define an object as part of your class, rather than just a pointer. In this case, make sure as best you can that you are using objects with header files that don't include lots of others. i.e. The dependency chain is as short as possible.

THIRD PARTY LIBRARIES

Within your own code it's possible to have total control, but what I hear you say, happens when using third party libraries? What if it's a third party 3d engine, or god forbid, WINDOWS?

This normally means the third party library is needed for #defines, or class definitions.

Needless header file including doesn't come much worse than when it's used to access simple #defines. For instance microsoft windows makes heavy use of DWORDs. I've actually seen (I kid you not) people have header files of the form:

#include <windows.h>

class MyClass
{
public:
DWORD m_dwValue;
};

NO NO NOOOOOOOO!!!!!
This kind of thing makes me want to go outside, wait for sunrise, then fall on my sword.

Firstly, use browse info to look up what a DWORD actually is. It's not some super dooper class, it's just ultimately an unsigned int.

So your file could be replaced by:

class MyClass
{
public:
unsigned int m_uiValue;
};

If you've ever #included <windows.h> you'll have an idea of how much time this will save.

Another trick is to create your own SIMPLE header file, duplicating the common defines that you need to access in the third party bloatware. That way you can refer to them by the same name, just by including one header instead of a myriad.

Other simple classes such as RECT, or a Vector3 :
DON'T include huge bloated stuff like windows.h just to use small classes like RECT.
Instead, create your own binary compatible class (with no dependencies in the .h file) and include THAT instead. Then when you want to pass it to windows (or 3rd party code), just CAST it to the type the compiler wants.

One gotcha is to make sure it really is binary compatible. Usually this is simple, but in some special cases there can be alignment issues (see #pragma pack documentation). This doesn't crop up often but is worth bearing in mind.

While this speeds up things using 3rd party libaries a lot .. there is another tip that can be very useful, as 3rd party libraries are often written by retards, who have no understanding of the concept of compilation time.

In this scenario, you want to call the 3rd party library from your cpp files. There may be lots of your cpp files that want to use the library, and each one ends up doing the #include for it, and taking ages to compile.

Instead of doing this, consider writing an 'interface' set of functions, or class, that offers a translation layer where you call them through your own simple header file, and the interface cpp file is the only one that has to talk to the third party library.

This means that instead of #including the dog slow third party library umpteen times throughout your own code and paying the penalty multiple times, you consolidate the penalty into ONE cpp file, which only needs to compile once, and once done, the functions are available to the rest of your app, almost for free in terms of penalty.

The added benefit is, that while you are writing code, you should be constantly compiling to check for errors etc and debugging. Now, instead of having to wait 20 seconds for your file to compile, you only wait 1 second because it doesn't have to #include the 3rd party library every time, only your simple interface to it.

Of course, this is duplicating to some extent an actual use for precompiled headers. However, using an interface can be preferable for two reasons: Firstly, you don't have to worry about setting up precompiled headers, and all those include stdafxs throughout your code. And secondly, the use of interfaces is good practice, if for nothing else than because it allows you to update your app to use a new version of a third party library with the minimum of change to your codebase.

Anyway, this concludes my recommendations on the subject. Of course there will be exceptions, there are always trade offs, sometimes between compile time and runtime speed. But if you try and use these recommendations wherever possible it will greatly speed up your compile time, and hence your productivity.

No comments: