Build Systems Suck

Matt Soucy ()

In the beginning...

...there were basic commands.

$CC -c main.c -o main.o
$CC -c lib.c -o lib.o
$CC -o prog lib.o main.o
# $CC is a variable that holds "C compiler of your choice"

More arguments

What happens when we want more arguments added?

$CC -c main.c -o main.o -I../somelib
$CC -c lib.c -o lib.o -I../somelib
$CC -o prog lib.o main.o

This is silly...

Variables

Can we store values in shell variables?

export CFLAGS=-I../somelib -DFOO=5
$CC -c main.c -o main.o $CFLAGS
$CC -c lib.c -o lib.o $CFLAGS
$CC -o prog lib.o main.o $CFLAGS

Loops?

More "condensing"

export cfiles=main lib
export CFLAGS=-I../somelib -DFOO=5
for i in $cfiles; do
    $CC -c ${i}.c -o ${i}.o ${CFLAGS}
done
$CC -o prog $(cfiles// /.o ).o
# Fails if multiple spaces are in cfiles, like 'a  b'

Still rebuilds everything EVERY TIME.

Rule-based

What rules to we have?

Introducing make!

Introducing make

Back to the "beginning"...so no variables (yet) or optimization

prog: main.o lib.o
    ${CC} -o prog main.o lib.o -I../somelib -DFOO=5

main.o: main.c
    ${CC} -o main.o -c main.c -I../somelib -DFOO=5

lib.o: lib.c
    ${CC} -o lib.o -c lib.c -I../somelib -DFOO=5

Using some variables

CFLAGS:=-I../somelib -DFOO=5

prog: main.o lib.o
    ${CC} -o prog main.o lib.o ${CFLAGS}

main.o: main.c
    ${CC} -o main.o -c main.c ${CFLAGS}

lib.o: lib.c
    ${CC} -o lib.o -c lib.c ${CFLAGS}

Rule matching

CFLAGS+=-I../somelib -DFOO=5

prog: main.o lib.o
    ${CC} -o prog main.o lib.o ${CFLAGS}

%.o: %.c
    ${CC} -o $@ -c $< ${CFLAGS}

Implicit Rules

Make actually has support for C, C++, Fortran, Lex, Yacc...

CFLAGS+=-I../somelib -DFOO=5
proc: main.o lib.o

Cleanup and tests

CFLAGS+=-I../somelib -DFOO=5
OBJS=main.o foo.o

all: prog

prog: ${OBJS}

# Test target depends on the program existing
test: prog
    ./prog 1 2 > test.out
    diff test.out actual.out

clean:
    rm -rf ${OBJS}

# Tell make that "clean" doesn't generate anything,
# so is always out of date
.PHONY: clean test

make is magic

If you have just a test file main.c, you can run make main without a makefile at all, and the built-in rules will operate on it!

make Awesomeness

make Problems

Alternatives?

Language specific?

Language Tool
D dub (JSON)
Java ant (xml)
Java Maven (xml)
Python Python (pip, setuptools, pbr...)
Go go build (builtin)
Haskell Cabal

Most of these also do some sort of dependency management, as well as build management.

tup

Graph-based system that promises that builds are always "as if from clean"

# Tupfile
CFLAGS += -I../somelib
CFLAGS += -DFOO=5
!cc = |> $(CC) $(CFLAGS) -c %f -o %o |> %B.o
: foreach main.c lib.c |> !cc |>
: main.o lib.o |> $(CC) %f -o %o |> prog

tup Awesomeness

tup Problems

scons

cons (perl) -> sccons (Python) -> scons (Python)

Basically a build-system-in-a-library

env = Environment(CPPPATH=["#../somelib"],
                  CPPDEFINES={"FOO": "5"})
prog = env.Program(target="prog", source=["main.c", "lib.c"])
Default(prog)

scons Awesomeness

scons Problems

ninja

cc = gcc
cflags = -I../somelib -DFOO=5
rule cc
  command = $cc -c $in -o $out $cflags
rule link
  command = $cc $in -o $out $cflags

build main.o: cc main.c
build lib.o: cc lib.c
build prog: link main.o lib.o

ninja Awesomeness

ninja Problems

cmake

A meta-build system that generates:

Write one bit of cross-platform code, build for all platforms

cmake Example

cmake_minimum_required(VERSION 2.8.12)

project(MyProgram)

include_directories(../somelib)
add_definitions(-DFOO=5)

add_executable(prog main.c lib.c)

cmake Awesomeness

cmake Problems

cmake Problems

Conclusion

All build systems have huge problems, but some have nice features

Find the system that works best for your use case

Desire Possible build system
Simple make, ninja
Fast tup
Portable cmake, SCons
for q in ${questions}; do
    try-answer $q
done