[mythtv] FYI: Recursive Make Considered Harmful

[Dan Conti]

The writer of this article has no clue what he is talking about. Recursive
make does not prevent you from rebuilding in a single directory without
scanning all directories. However, using a single makefile requires that
all dependency checking for all targets happen prior to determining which
targets to compile. I.e. with a single makefile you pay this penalty
_every time_ regardless. On systems like cygwin, where file i/o is slow,
this is a huge price to pay. Even on a normal system, do you really want
to stat 1000 files to build one? You dont have to with recursive make, but
you do with single file make.

Here's another gem: this guy recommends VPATH. Anyone with half a clue
and some concern for make performance avoids VPATH like the plague.
Basically instead of using fixed locations for everything VPATH lets make
search around for files that could be...anywhere. Hope you dont have any
duplicate filenames anywhere in your project, and not too many directories
or files since it will impact VPATH performance.

He also doesn't generate dependencies in the compile step, which is
another waste; in a 1000 file project you end up almost doubling your
compile time because you have to run GCC twice, once to generate the new
dep and another to compile the file. The smart way to do this is to pass
the appropriate options to gcc so it generates the new .d when it
compiles.

But again, this guy isn't too smart. :)

-Dan


On Fri, 23 Jan 2004, Brad Allen wrote:

> The below is relevant to Myth.  Below abstract copied from
> <URL:"http://WWW.PCUG.Org.Au/~millerp/rmch/recu-make-cons-harm.html">;
> paper at <URL:"http://AEGIS.SourceForge.Net/auug97.pdf">:
>
>
>                        Recursive Make Considered Harmful
>
>                                  Peter Miller
>                            millerp at canb.auug.org.au
>
> Copyright (C) 1997 Peter Miller. All rights reserved.
>
> This paper is available in PDF format.  [See above for paper link.]
>
> -------------------------------------------------------------------------------
>
>                                    ABSTRACT
>
>     For large UNIX projects, the traditional method of building the
>     project is to use recursive make. On some projects, this results
>     in build times which are unacceptably large, when all you want to
>     do is change one file. In examining the source of the overly long
>     build times, it became evident that a number of apparently
>     unrelated problems combine to produce the delay, but on analysis
>     all have the same root cause.
>
>     This paper explores an number of problems regarding the use of
>     recursive make, and shows that they are all symptoms of the same
>     problem. Symptoms that the UNIX community have long accepted as a
>     fact of life, but which need not be endured any longer. These
>     problems include recursive makes which take ``forever'' to work
>     out that they need to do nothing, recursive makes which do too
>     much, or too little, recursive makes which are overly sensitive to
>     changes in the source code and require constant Makefile
>     intervention to keep them working.
>
>     The resolution of these problems can be found by looking at what
>     make does, from first principles, and then analyzing the effects
>     of introducing recursive make to this activity. The analysis shows
>     that the problem stems from the artificial partitioning of the
>     build into separate subsets. This, in turn, leads to the symptoms
>     described. To avoid the symptoms, it is only necessary to avoid
>     the separation; to use a single Makefile for the whole project.
>
>     This conclusion runs counter to much accumulated folk wisdom in
>     building large projects on UNIX. Some of the main objections
>     raised by this folk wisdom are examined and shown to be
>     unfounded. The results of actual use are far more encouraging,
>     with routine development performance improvements significantly
>     faster than intuition may indicate. The use of a single project
>     Makefile is not as difficult to put into practice as it may first
>     appear.
>