class: center, middle # Week 7 --- # Announcements * HW5, ADV5 due by 11:59 PM on Feb 26 * HW6, ADV6 due by 11:59 PM on Mar 4 * Will release ADV6 tonight or Saturday: will be easy and not require going to office hours --- class: center, middle # Lecture 7: Build Systems #### `gcc -I inc -o app $(find . -name *.c) -lsomelib` --- # Overview * Build systems * `make` * Other build systems --- # Build systems -- ## Q: Who has used a build system? --- ## What is a build system? * Typing out `gcc -o myapp file1.c file2.c file3.c main.c` every time you want to compile can be annoying... -- * Tool to automate building software * Compilation * Packaging * Testing --- ## A simple build system #### A throwback to HW2... ```bash --- build.sh --- #!/usr/bin/env bash gcc -o myapp src/file1.c src/file2.c src/file3.c src/main.c ``` ```bash --- build.sh --- #!/usr/bin/env bash gcc -o myapp $(find src -name "*.c") ``` ```bash ./build.sh ``` --- ## Some issues * Can be a bit of work to custom write a script, especially with larger projects * Will blindly compile everything, every time * What if we made a small change to one file and didn't want to recompile all the code? --- ## Aside: incremental building * It takes my lab computer about 30 minutes to do a clean build of LLVM and Clang while maxing out my CPU * (Hours if I restrict how many cores I give it...) -- * Imagine if I had to recompile _everything_ every time I made a small code change -- * Put together independent bits instead of compiling/building everything every time * Classic model: C/C++ programs * Compile individual C/C++ files into _object code_ (`.o` files) -- * _Link_ the object code files into the final output executable binary -- * Change only one C/C++ file? Just build the object code for that file, then link the object code -- * ...now a simple shell script doesn't seem to cut it --- # Make #### (We'll be focusing on [GNU Make](https://www.gnu.org/software/make/manual/) as it's probably the most popular) * Classic tool that helps with build automation -- * Provides more abstractions over a plain shell script -- * Invoke it by running `make` -- * Will look for a `Makefile` (or `makefile`) to run * (it's actually possible to run without a `Makefile`, but we won't really get into that) -- * The `Makefile` will specify __rules__ that have __prerequisites__ that have to be met/built before running its __recipe__ to build a __target__ ```make target: prerequisites recipe # <- actual tab character, not spaces! ``` * Make is able tell if the built __target__ file is newer than __prerequisite__ files to avoid unnecessarily performing the __recipe__ * The __recipe__ consists of shell commands * `make <target>` will build a specific __target__ --- ### Simple example ```make myapp: src/file1.c src/file2.c src/file3.c src/main.c gcc -o myapp src/file1.c src/file2.c src/file3.c src/main.c ``` -- ### A bit more sophisticated ```make myapp: src/file1.c src/file2.c src/file3.c src/main.c gcc -o $@ $^ ``` --- ## Philosophy * Overall idea is to have __rules__ that depend on other __rules__ that build smaller bits of the system * e.g. building the final executable depends on compiling individual source code files into corresponding object code -- * This composability means that we can incrementally build our project * Invaluable with enormous code bases: don't want to recompile _every_ file of the Linux kernel if you made a single line change to one file -- * Can have additional rules that run/test/debug the application and clean the directory of build output --- ## Make concepts #### Make gives us more abstractions to make our lives easier #### It's a pretty deep tool; we're going to look at the basics * Targets and rules * "Phony" targets * Powerful variable assignments * Functions and other expansions * Automatic variables * Pattern matching --- # Targets and rules ```make target: prerequisites recipe # <- actual tab character, not spaces! ``` * __Prerequisite__ targets will be built before the __target__'s __recipe__ will be able to be run -- * The __recipe__ consists of shell commands -- * `make <target>` will build a specific __target__ -- * The __target__ is assumed to be some actual file that gets produced -- * Make is able tell if the built __target__ file is newer than __prerequisite__ files to avoid unnecessarily performing the __recipe__ -- * If there are no __prerequisites__ and the __target__ file is present, the __recipe__ won't be run again --- ## "Phony" targets * What if you have a __target__ that you want to be a word/concept? * e.g. `clean`, `all`, `test` -- * If a file called `clean` or `all` is present, the target won't ever be run -- * The `.PHONY` target can specify phony targets that don't have actual files ```make .PHONY: all clean test ``` --- ## Variable assignments * You can define variables in Makefiles as well (you can put spaces around the '='!) -- * Often times are things like compilation flags, compiler selection, directories, files etc. -- * To use Makefile variables and "expand" them, you use `$(varname)` or `${varname}` * Parentheses are more common -- #### Two flavors of variables * Define how they get assigned and how they get expanded -- * `varA = $(varB)` recursively expands the right hand side whenever the variable gets expanded * If `varB` gets reassigned after this, an expanded `varA` will expand the current value of `varB` -- * `varA := $(varB)` performs a simple expansion, taking on the value of the right hand side at assignment time * If `varB` gets reassigned after this, an expanded `varA` will expand to the value of `varB` when `varA` got assigned -- #### Another operator * `varA ?= bar` will assign variable `varA` if it hasn't been assigned before --- ## Functions and other expansions * There are some functions that can be expanded to help with text manipulation * `$(wildcard <pattern>)` can expand to a file list for files that match the pattern * `$(wildcard *.c)` -- * `$(<var>:<pattern>=<replacement>)`, known as a substitution reference, can perform replacements * `$(SOURCES:%.c=%.o)` -- * `$(shell <commands>)` can run a shell command and expand to the command's output * `$(shell find . -name "*.c")` --- ## Automatic variables * In a rule, Make has some automatically assigned variables * `$@`: __target__'s file name * `$<`: First __prerequisite__ * `$?`: All __prerequisites__ newer than the target * `$^`: All __prerequisites__ * ...and more -- #### Using what we've learned so far... ```make CC := gcc BIN := myapp SRCS := $(shell find src -name *.c) $(BIN): $(SRCS) $(CC) -o $@ $^ ``` --- ## Pattern matching ### Pattern rules ```make %.o : %.c $(CC) -c -o $@ $< ``` * Uses `%` to match file names * This example compiles `.c` files into `.o` files * Note that this is a general rule that applies to all `.o` files * This is known as an __implicit rule__ -- ### Static pattern rules ```make OBJS := $(SRCS:src/%.c=obj/%.o) $(OBJS): obj/%.o : src/%.c $(CC) -c -o $@ $< ``` * Can narrow down a pattern rule to a particular list of targets --- ## Make * This is a brief overview of some of the features of Make * This is by no means a comprehensive look at Make: refer to the manual for more features and details --- # Other build systems * Make is a fairly general build system, but other build systems have more abstractions and may be tailored towards a particular language -- * General: Ninja, CMake (actually more of a Makefile generator) * Java: Ant, Maven, Gradle * Ruby: Rake * Continuous integration: Jenkins, Travis CI --- class: center, middle # Demo time --- class: center, middle # Questions