A short introduction to Make
Part of an ongoing series of essays tentatively entitled Don’t embarrass me, Don’t embarrass yourself: Notes on thinking in C and Unix.
As you likely know if you are a C programmer (or, in fact, anyone who does programming involving the command line), at some point, you need to develop some arcane incantations to build your programs. For example, we recently considered a simple C project that included a command-line tool for computing greatest-common divisors. Let’s look at the commands we were using to build that tool [1].
$ cc -g -Wall -c -o gcd.o gcd.c
$ cc -g -Wall -c -o mathlib-gcd.o mathlib-gcd.c
$ cc -o gcd gcd.o mathlib-gcd.o
How are you going to remember those? Well, I suppose that you could write them down somewhere. You’d probably put them in a file, right? And then maybe you’d copy and paste from the file.
As I think I told you, one of the ways real programmers approach
the world (or at least real Unix programmers) is that they strive to do
things by code
, rather than by hand
. Copying and pasting is doing
things by hand. We should have an alternative. Here’s one simple one,
we can build a shell script.
#!/bin/bash
# Build the amazing gcd application
cc -g -Wall -c -o gcd.o gcd.c
cc -g -Wall -c -o mathlib-gcd.o mathlib-gcd.c
cc -o gcd gcd.o mathlib-gcd.o
Now, whenever we want to build this gcd
, we just run this script. That’s
all well and good, right?
I’ll note that this approach, while reasonable, has some flaws. First,
how do we remember when we have to rebuild gcd
? Clearly, we need to
rebuild it when we change gcd.c
, but will we remember that we also
have to rebuild it when we change mathlib-gcd.c
or mathlib.h
for that
matter? In addition, we’ve hard-coded a particular set of compiler flags.
What if we decide we want to add others, such as -DNDEBUG
? As sensible
programmers, we should probably use variables.
This project is also small. What happens when we get to a larger project, one with dozens, or hundreds, or even thousands of source files? Do we want to re-compile each file if we change just one [2]?
As you might expect, these are not new questions. Sensible programmers have thought about these issues for a long time, and have developed a variety of build automation tools. Most C programmers use a tool called Make, originally developed by Stuart Feldman [3] in the mid-1970’s. Like many Unix programmers, I use Make not just for my C programs, but also to manage a wide variety of other kinds of projects, including my Web sites.
Make is a powerful and complex enough tool that it’s impossible to cover all of Make in a few essays. So I’m going to focus on some key points that I find valuable, and note that there are great resources for learning more about Make, including Managing Projects in GNU Make and The GNU Make Manual.
Let’s start with the basics. If you think about the instructions we wrote above, as well as the related questions, you’ll see that there are generally three things we need to think about when deciding the steps of building software.
First, there are the targets, the things that we build. In our simple
project, we have the .o
files, the main executable, the test program,
and, perhaps, a library file. Second, there are the dependencies.
These indicate what we have to rebuild when something changes.
For example, if we change mathlib-gcd.c
, we’ll need to rebuild both
gcd
and test-gcd
. However, if we change test-gcd.c
, we only need
to rebuild test-gcd
. Finally, we have the instructions that indicate
how we build our targets from the dependencies.
In Make, we lay out this information using the following form.
TARGET: DEPENDENCIES
INSTRUCTIONS
That is, we write the target, a colon, a list of files the target depends upon, and the instructions for building the target. Note that the whitespace before the instructions must be a tab; a sequence of spaces doesn’t count.
So, let’s look at some of the rules for our project. First, we will note
that we need to rebuild mathlib-gcd.o
if either mathlib-gcd.c
or
mathlib.h
changes.
mathlib-gcd.o: mathlib-gcd.c mathlib.h
cc -g -Wall -c -o mathlib-gcd.o mathlib-gcd.c
Similarly, we need to rebuild gcd.o
if either gcd.c
or mathlib.h
changes.
gcd.o: gcd.c mathlib.h
cc -g -Wall -c -o gcd.o gcd.c
Finally, to build the executable, we link the files together.
gcd: gcd.o mathlib-gcd.o
cc -o gcd gcd.o mathlib-gcd.o
Let’s see how well that works. If you put those lines into a file
called Makefile, you can create gcd
by typing make gcd
. Here’s
what I see when I do so.
$ make gcd
cc -g -Wall -c -o gcd.o gcd.c
cc -g -Wall -c -o mathlib-gcd.o mathlib-gcd.c
cc -o gcd gcd.o mathlib-gcd.o
Yay! It even figured out the order in which to do the instructions.
What about the tests? Well, let’s add two sets of rules, one for the
.o
file and one for the executable.
test-gcd.o: test-gcd.c mathlib.h
cc -g -Wall -c -o test-gcd.o test-gcd.c
test-gcd: test-gcd.o mathlib-gcd.o
cc -o test-gcd test-gcd.o mathlib-gcd.o
We’re even going to add a special kind of target, one that refers to
an action, rather than to a file. We want to run tests. What do we
need to do to run tests? Right now, we just need to generate test-gcd
and run it. Later, we might have other tests. So we’ll write a target
called test
.
test: test-gcd
./test-gcd
Let’s see how well that works.
$ make test
$ cc -g -Wall -c -o test-gcd.o test-gcd.c
$ cc -o test-gcd test-gcd.o mathlib-gcd.o
$ ./test-gcd
$
Make figured out that it didn’t need to recompile mathlib-gcd.c
, since
it hadn’t changed. So it compiled test-gcd.o
, linked it with
mathlib-gcd.o
, and then ran the result. Pretty awesome, isn’t it?
Now, let’s see what happens if we change a file. Suppose we change
test-gcd.c
. What do we need to rebuild? Let’s see if Make can tell.
(Note that we are using the touch
utility to update the modification
date of a file to simulate changing the file.)
$ touch test-gcd.c
$ make gcd
make: 'gcd' is up to date.
$ make test-gcd
cc -g -Wall -c -o test-gcd.o test-gcd.c
cc -o test-gcd test-gcd.o mathlib-gcd.o
Once again, Make figured out what work had to be done and what work didn’t have to be done. Let’s try a few more examples
$ make gcd test-gcd
make: 'gcd' is up to date.
make: 'test-gcd' is up to date.
$ touch mathlib-gcd.c
$ make gcd test-gcd
cc -g -Wall -c -o mathlib-gcd.o mathlib-gcd.c
cc -o gcd gcd.o mathlib-gcd.o
cc -o test-gcd test-gcd.o mathlib-gcd.o
$ touch mathlib.h
$ make gcd test-gcd
cc -g -Wall -c -o gcd.o gcd.c
cc -g -Wall -c -o mathlib-gcd.o mathlib-gcd.c
cc -o gcd gcd.o mathlib-gcd.o
cc -g -Wall -c -o test-gcd.o test-gcd.c
cc -o test-gcd test-gcd.o mathlib-gcd.o
Yup, it looks like things are going well, and we’ll leave it at that.
You now know the basics of Make. In designing a project, you should
think about the source files you will create, the target files you
will generate, the ways in which the targets depend upon the source
files (and upon each other), and the instructions to build the targets.
All of that should happen, whether or not you are using Make. Once you
start using Make, you take the additional step of expressing them in this
simple syntax, and type make TARGET
to generate a target. It doesn’t
take a lot of effort beyond what you’d have to do to figure out the magic
incantations in the first place and, after that, Make remembers the rest.
Are we done? No. While you can certainly get a lot out of Make with these basic ideas, there’s a lot more that you should know [5]. For example, at some point, you probably noticed that we were writing very similar instructions. The Don’t Repeat Yourself (DRY) principle suggests that that’s bad practice. How can we make our code more concise and less repetitions? We’ll consider those issues, and many more, in future sections.
[1] Yes, I realize that there are other sequences of instructions we could use, particularly if we want to put our library code in, well, a library. But we’ll stick with those for now.
[2] I think the answer is it depends
. If other files depend on the
source code in that file (e.g., if the file is a header file), then we
probably do need to recompile everything, or at least many things. But
if we’re just using that file as a utility, we probably just need to
recompile that one file and then relink everything.
[3] I recently met Feldman at a conference and thanked him for creating
Make [4]. He said something like I still regret that one really bad
design decision I made.
Of course, I’m clueless, and couldn’t think
of any really bad design decisions in Make, even though there’s one I
curse every time I use Make.
[4] If I’d seen him later, I would also have thanked him for his talk,
which was aimed mostly at students, and, in effect, was of the form If
you want to get an interesting programming job, particularly one at Google,
you better know how to do running-time analysis.
[5] In fact, no competent Make user would write the instructions I wrote above. But I think it’s worth seeing some basic instructions to help you understand what’s going on before you delve into complexities.
Version 1.0.1 of 2017-02-06.