ccache-swig2.0 - Online in the Cloud

PROGRAM:

NAME

ccache-swig - a fast compiler cache

SYNOPSIS

ccache-swig [OPTION]

ccache-swig <compiler> [COMPILER OPTIONS]

<compiler> [COMPILER OPTIONS]

DESCRIPTION

ccache-swig is a compiler cache. It speeds up re-compilation of C/C++/SWIG code by caching
previous compiles and detecting when the same compile is being done again. ccache-swig is
ccache plus support for SWIG. ccache and ccache-swig are used interchangeably in this
document.

OPTIONS SUMMARY

Here is a summary of the options to ccache-swig.

-s show statistics summary
-z zero statistics
-c run a cache cleanup
-C clear the cache completely
-F <n> set maximum files in cache
-M <n> set maximum size of cache (use G, M or K)
-h this help page
-V print version number

OPTIONS

These options only apply when you invoke ccache as "ccache-swig". When invoked as a
compiler none of these options apply. In that case your normal compiler options apply and
you should refer to your compilers documentation.

-h Print a options summary page

-s Print the current statistics summary for the cache. The statistics are stored
spread across the subdirectories of the cache. Using "ccache-swig -s" adds up the
statistics across all subdirectories and prints the totals.

-z Zero the cache statistics.

-V Print the ccache version number

-c Clean the cache and re-calculate the cache file count and size totals. Normally the
-c option should not be necessary as ccache keeps the cache below the specified
limits at runtime and keeps statistics up to date on each compile. This option is
mostly useful if you manually modify the cache contents or believe that the cache
size statistics may be inaccurate.

-C Clear the entire cache, removing all cached files.

-F <maxfiles>
This sets the maximum number of files allowed in the cache. The value is stored
inside the cache directory and applies to all future compiles. Due to the way the
value is stored the actual value used is always rounded down to the nearest
multiple of 16.

-M <maxsize>
This sets the maximum cache size. You can specify a value in gigabytes, megabytes
or kilobytes by appending a G, M or K to the value. The default is gigabytes. The
actual value stored is rounded down to the nearest multiple of 16 kilobytes.

INSTALLATION

There are two ways to use ccache. You can either prefix your compile commands with
"ccache-swig" or you can create a symbolic link between ccache-swig and the names of your
compilers. The first method is most convenient if you just want to try out ccache or wish
to use it for some specific projects. The second method is most useful for when you wish
to use ccache for all your compiles.

To install for usage by the first method just copy ccache-swig to somewhere in your path.

To install for the second method do something like this:

cp ccache-swig /usr/local/bin/
ln -s /usr/local/bin/ccache-swig /usr/local/bin/gcc
ln -s /usr/local/bin/ccache-swig /usr/local/bin/g++
ln -s /usr/local/bin/ccache-swig /usr/local/bin/cc
ln -s /usr/local/bin/ccache-swig /usr/local/bin/swig

This will work as long as /usr/local/bin comes before the path to gcc (which is usually in
/usr/bin). After installing you may wish to run "which gcc" to make sure that the correct
link is being used.

Note! Do not use a hard link, use a symbolic link. A hardlink will cause "interesting"
problems.

EXTRA OPTIONS

When run as a compiler front end ccache usually just takes the same command line options
as the compiler you are using. The only exception to this is the option ’--ccache-skip’.
That option can be used to tell ccache that the next option is definitely not a input
filename, and should be passed along to the compiler as-is.

The reason this can be important is that ccache does need to parse the command line and
determine what is an input filename and what is a compiler option, as it needs the input
filename to determine the name of the resulting object file (among other things). The
heuristic ccache uses in this parse is that any string on the command line that exists as
a file is treated as an input file name (usually a C file). By using --ccache-skip you can
force an option to not be treated as an input file name and instead be passed along to the
compiler as a command line option.

ENVIRONMENT VARIABLES

ccache uses a number of environment variables to control operation. In most cases you
won’t need any of these as the defaults will be fine.

CCACHE_DIR
the CCACHE_DIR environment variable specifies where ccache will keep its cached
compiler output. The default is "$HOME/.ccache".

CCACHE_TEMPDIR
the CCACHE_TEMPDIR environment variable specifies where ccache will put temporary
files. The default is the same as CCACHE_DIR. Note that the CCACHE_TEMPDIR path
must be on the same filesystem as the CCACHE_DIR path, so that renames of files
between the two directories can work.

CCACHE_LOGFILE
If you set the CCACHE_LOGFILE environment variable then ccache will write some log
information on cache hits and misses in that file. This is useful for tracking down
problems.

CCACHE_VERBOSE
If you set the CCACHE_VERBOSE environment variable then ccache will display on
stdout all the compiler invocations that it makes. This can useful for debugging
unexpected problems.

CCACHE_PATH
You can optionally set CCACHE_PATH to a colon separated path where ccache will look
for the real compilers. If you don’t do this then ccache will look for the first
executable matching the compiler name in the normal PATH that isn’t a symbolic link
to ccache itself.

CCACHE_CC
You can optionally set CCACHE_CC to force the name of the compiler to use. If you
don’t do this then ccache works it out from the command line.

CCACHE_PREFIX
This option adds a prefix to the command line that ccache runs when invoking the
compiler. Also see the section below on using ccache with distcc.

CCACHE_DISABLE
If you set the environment variable CCACHE_DISABLE then ccache will just call the
real compiler, bypassing the cache completely.

CCACHE_READONLY
the CCACHE_READONLY environment variable tells ccache to attempt to use existing
cached object files, but not to try to add anything new to the cache. If you are
using this because your CCACHE_DIR is read-only, then you may find that you also
need to set CCACHE_TEMPDIR as otherwise ccache will fail to create the temporary
files.

CCACHE_CPP2
If you set the environment variable CCACHE_CPP2 then ccache will not use the
optimisation of avoiding the 2nd call to the pre-processor by compiling the
pre-processed output that was used for finding the hash in the case of a cache
miss. This is primarily a debugging option, although it is possible that some
unusual compilers will have problems with the intermediate filename extensions used
in this optimisation, in which case this option could allow ccache to be used.

CCACHE_NOCOMPRESS
If you set the environment variable CCACHE_NOCOMPRESS then there is no compression
used on files that go into the cache. However, this setting has no effect on how
files are retrieved from the cache, compressed results will still be usable.

CCACHE_NOSTATS
If you set the environment variable CCACHE_NOSTATS then ccache will not update the
statistics files on each compile.

CCACHE_NLEVELS
The environment variable CCACHE_NLEVELS allows you to choose the number of levels
of hash in the cache directory. The default is 2. The minimum is 1 and the maximum
is 8.

CCACHE_HARDLINK
If you set the environment variable CCACHE_HARDLINK then ccache will attempt to use
hard links from the cache directory when creating the compiler output rather than
using a file copy. Using hard links is faster, but can confuse programs like ’make’
that rely on modification times. Hard links are never made for compressed cache
files.

CCACHE_RECACHE
This forces ccache to not use any cached results, even if it finds them. New
results are still cached, but existing cache entries are ignored.

CCACHE_UMASK
This sets the umask for ccache and all child processes (such as the compiler). This
is mostly useful when you wish to share your cache with other users. Note that this
also affects the file permissions set on the object files created from your
compilations.

CCACHE_HASHDIR
This tells ccache to hash the current working directory when calculating the hash
that is used to distinguish two compiles. This prevents a problem with the storage
of the current working directory in the debug info of a object file, which can lead
ccache to give a cached object file that has the working directory in the debug
info set incorrectly. This option is off by default as the incorrect setting of
this debug info rarely causes problems. If you strike problems with gdb not using
the correct directory then enable this option.

CCACHE_UNIFY
If you set the environment variable CCACHE_UNIFY then ccache will use the C/C++
unifier when hashing the pre-processor output if -g is not used in the compile. The
unifier is slower than a normal hash, so setting this environment variable loses a
little bit of speed, but it means that ccache can take advantage of not recompiling
when the changes to the source code consist of reformatting only. Note that using
CCACHE_UNIFY changes the hash, so cached compiles with CCACHE_UNIFY set cannot be
used when CCACHE_UNIFY is not set and vice versa. The reason the unifier is off by
default is that it can give incorrect line number information in compiler warning
messages.

CCACHE_EXTENSION
Normally ccache tries to automatically determine the extension to use for
intermediate C pre-processor files based on the type of file being compiled.
Unfortunately this sometimes doesn’t work, for example when using the aCC compiler
on HP-UX. On systems like this you can use the CCACHE_EXTENSION option to override
the default. On HP-UX set this environment variable to "i" if you use the aCC
compiler.

CCACHE_STRIPC
If you set the environment variable CCACHE_STRIPC then ccache will strip the -c
option when invoking the preprocessor. This option is primarily for the Sun
Workshop C++ compiler as without this option an unwarranted warning is displayed:
CC: Warning: "-E" redefines product from "object" to "source (stdout)" when -E and
-c is used together.

CCACHE_SWIG
When using SWIG as the compiler and it does not have ’swig’ in the executable name,
then the CCACHE_SWIG environment variable needs to be set in order for ccache to
work correctly with SWIG. The use of CCACHE_CPP2 is also recommended for SWIG due
to some preprocessor quirks, however, use of CCACHE_CPP2 can often be skipped --
check your generated code with and without this option set. Known problems are
using preprocessor directives within %inline blocks and the use of ’#pragma SWIG’.

CACHE SIZE MANAGEMENT

By default ccache has a one gigabyte limit on the cache size and no maximum number of
files. You can set a different limit using the "ccache -M" and "ccache -F" options, which
set the size and number of files limits.

When these limits are reached ccache will reduce the cache to 20% below the numbers you
specified in order to avoid doing the cache clean operation too often.

CACHE COMPRESSION

By default on most platforms ccache will compress all files it puts into the cache using
the zlib compression. While this involves a negligible performance slowdown, it
significantly increases the number of files that fit in the cache. You can turn off
compression setting the CCACHE_NOCOMPRESS environment variable.

HOW IT WORKS

The basic idea is to detect when you are compiling exactly the same code a 2nd time and
use the previously compiled output. You detect that it is the same code by forming a hash
of:

o the pre-processor output from running the compiler with -E

o the command line options

o the real compilers size and modification time

o any stderr output generated by the compiler

These are hashed using md4 (a strong hash) and a cache file is formed based on that hash
result. When the same compilation is done a second time ccache is able to supply the
correct compiler output (including all warnings etc) from the cache.

ccache has been carefully written to always produce exactly the same compiler output that
you would get without the cache. If you ever discover a case where ccache changes the
output of your compiler then please let me know.

USING CCACHE WITH DISTCC

distcc is a very useful program for distributing compilation across a range of compiler
servers. It is often useful to combine distcc with ccache, so that compiles that are done
are sped up by distcc, but that ccache avoids the compile completely where possible.

To use distcc with ccache I recommend using the CCACHE_PREFIX option. You just need to set
the environment variable CCACHE_PREFIX to ’distcc’ and ccache will prefix the command line
used with the compiler with the command ’distcc’.

SHARING A CACHE

A group of developers can increase the cache hit rate by sharing a cache directory. The
hard links however cause unwanted side effects, as all links to a cached file share the
file’s modification timestamp. This results in false dependencies to be triggered by
timestamp-based build systems whenever another user links to an existing file. Typically,
users will see that their libraries and binaries are relinked without reason. To share a
cache without side effects, the following conditions need to be met:

o Use the same CCACHE_DIR environment variable setting

o Unset the CCACHE_HARDLINK environment variable

o Make sure everyone sets the CCACHE_UMASK environment variable to 002, this ensures
that cached files are accessible to everyone in the group.

o Make sure that all users have write permission in the entire cache directory (and
that you trust all users of the shared cache).

o Make sure that the setgid bit is set on all directories in the cache. This tells
the filesystem to inherit group ownership for new directories. The command "chmod
g+s `find $CCACHE_DIR -type d`" might be useful for this.

o Set CCACHE_NOCOMPRESS for all users, if there are users with versions of ccache
that do not support compression.

HISTORY

ccache was inspired by the compilercache shell script script written by Erik Thiele and I
would like to thank him for an excellent piece of work. See
http://www.erikyyy.de/compilercache/ for the Erik’s scripts. ccache-swig is a port of the
original ccache with support added for use with SWIG.

I wrote ccache because I wanted to get a bit more speed out of a compiler cache and I
wanted to remove some of the limitations of the shell-script version.

DIFFERENCES FROM COMPILERCACHE

The biggest differences between Erik’s compilercache script and ccache are:

o ccache is written in C, which makes it a bit faster (calling out to external
programs is mostly what slowed down the scripts).

o ccache can automatically find the real compiler

o ccache keeps statistics on hits/misses

o ccache can do automatic cache management

o ccache can cache compiler output that includes warnings. In many cases this gives
ccache a much higher cache hit rate.

o ccache can handle a much wider ranger of compiler options

o ccache avoids a double call to cpp on a cache miss

CREDITS

Thanks to the following people for their contributions to ccache

o Erik Thiele for the original compilercache script

o Luciano Rocha for the idea of compiling the pre-processor output to avoid a 2nd cpp
pass

o Paul Russell for many suggestions and the debian packaging

Use ccache-swig2.0 online using onworks.net services