Lecture 11: Testing
--- class: center, middle # Week 12 --- # Announcements * Lecture 8 assignments due tonight (Nov 20) * Lecture 9 assignments due Nov 30 * Lecture 10 assignments due Dec 2 * Lecture 11 assignments, when released, will be due Dec 8 * Please fill out teaching evaluations when they're out! * I take these very seriously and they help the class evolve for future semesters * Also think about whether or not you want to be an IA for this class in WN 2021 * Details about this will come out soon --- class: center, middle # Lecture 11: Testing --- # Overview * What is and why testing? * Kinds of testing * Unit testing * Test-driven development --- # Foreword * We won't go into the finer details about software testing * It's very deep and evolving topic * If you want to know more, try taking EECS 481: Software Engineering --- ## What is and why testing? * Wikipedia: "Software testing is an investigation conducted to provide stakeholders with information about the quality of the software product or service under test" * Broad definition: includes checking for correctness, quality of service, etc. * We'll focus on the correctness _checking_ * _Checking_ to see if the right outputs are produced for the given inputs -- * Testing __does not necessarily__ guarantee or prove correctness * Testing helps give confidence that the implementation follows specifications and __helps uncover bugs/defects__ * Failing tests tells us something's broken * Passing tests tells us our code should work as far as tests go * It's still up to us to design good tests --- ## Some kinds of testing ### Hierarchy * __Unit testing__: testing a __unit__: individual component of code e.g. function, class, etc. * __Integration testing__: testing the interactions between components/subsystems * The line between integration and unit testing gets hazy when a class depends on another class... * __System testing__: testing your final application -- ### Other terms * __Regression testing__: testing to see if anything old breaks from new changes * __White-box testing__: testing that is aware of internals of the component being tested * __Black-box testing__: testing that is blind to the internals of the component being tested --- ## Unit testing * Testing of individual __units__: individual component of code such as a function or class * Write test cases that follow along with the specification * By keeping the scope small, we can more easily locate bugs when a test fails * Test cases provide inputs and check outputs for the particular unit * Test cases should be independent of each other: they should not keep state between tests -- * Test cases tend to have a typical structure: * __Setup__: sets up the "unit under test" (UUT) and its inputs * __Execution__: runs the UUT * __Validation__: checks to see if the outputs/behavior of the UUT is correct * __Cleanup__: restore the test system to a clean state --- ## Unit testing ```python import unittest class Foo: def __init__(self, name): self.name = name def bar(self, num): return self.name + str(num) class TestFoo(unittest.TestCase): def test_bar(self): uut = Foo('test') out = uut.bar(42) self.assertEqual('test 42', out) if __name__ == '__main__': unittest.main() ``` #### Does `Foo.bar()` work? --- ### Unit testing frameworks * Most languages have some sort of framework to test in * Provides an environment to generate a special executable to run tests * Many are based off of the xUnit paradigm influenced by Kent Beck's SUnit * "S" for "Smalltalk" * Examples * Java: JUnit * Python: `unittest` * C/C++: Google Test --- ## Test-driven development * Development process where you turn specifications for new features into tests before you code -- 1. Add tests for new feature 2. Run tests, new tests should fail 3. Write the minimum code to pass the new tests 4. Run tests, they should pass 5. Refactor the code while passing tests 6. Repeat for new features -- * In simple terms: add tests, write code to pass tests, make your code nicer, repeat --- ## Test-driven development * This process allows you to have some confidence that your code works * By minimizing your implementation you allow fewer avenues for things to go wrong * More tests != better testing * This process can tunnel-vision on small, simple tests; can fail to see bigger picture * More tests = more maintenance * Tests can take time to write: development may seem slower * Countered by time saved when debugging --- class: center, middle ## Live TDD + unit testing demo feat. Python `unittest` ### Feel free to follow along! --- ## Reverse polish notation calculator * Infix notation: (5 - 3) * (1 + 2) * Polish notation (PN): * - 5 3 + 1 2 * Also known as "prefix notation" * Reverse polish notation (RPN): 5 3 - 1 2 + * * Also known as "postfix notation" --- ## Reverse polish notation calculator * RPN lends itself to being implemented as a "stack machine" * Numbers get pushed onto the stack * Operators pop numbers off the stack and push the result * Example: 5 3 - 1 2 + * * Push 5 * Push 3 * -: pop 3, pop 5, perform 5-3, push 2 * Push 1 * Push 2 * +: pop 2, pop 1, perform 1+2, push 3 * \*: pop 3, pop 2, perform 2\*3, push 6 --- ## Before we start * Starter files ```bash https://www.eecs.umich.edu/courses/eecs201/ |-- files/examples/tdd/rpn.py |-- files/examples/tdd/test_rpn.py ``` * [Python 3 unittest documentation](https://docs.python.org/3/library/unittest.html) -- * Let's create a Makefile to run the tests ```make test: python3 -m unittest test_rpn ``` --- ## Spec * Let's start simple then add more features to illustrate TDD * Implement a read-evaluate-print-loop (REPL) to get input from user * Implement a Calculator class that encapsulates the stack * Numbers are all floating point * `size()` function to return size of the stack * `result()` function to return top of stack * `input()` function to pass in commands, returns top of stack -- 1. Handles numbers by pushing them on the stack -- 2. Handles addition -- 3. Handles unknown operator -- 4. Handles subtraction -- 5. Handles multiplication 6. Handles division --- ## Refactoring * Maybe we can go back and clean things up a bit * Are we repeating ourselves? How can we make this nicer? --- ## Conclusion * Testing isn't a panacea: tests are only as intelligent as their designers * Unit tests can tell you something is wrong and what unit is failing * TDD is a solid methodology, just beware of shortcomings -- * This lecture's assignment will look at C/C++ and Google Test :) * You may find unit testing to be helpful in your 281 or 370 endeavors ;) -- ### Additional resources * [EECS 481 slides about testing](https://web.eecs.umich.edu/~weimerw/481/lectures/se-04-qatesting.pdf) * One of the sources I drew upon: goes deeper than I did here * If that sort of stuff interests you, I encourage that you take the class * [Kent Beck's original work on Smalltalk testing](https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm) * [Test Driven Development: By Example, Kent Beck](https://learning.oreilly.com/library/view/test-driven-development/0321146530/) * [Free O'Reilly access provided by UMich](https://apps.lib.umich.edu/database/link/10263) (kudos to Arav) --- class: center, middle # Questions? ### Have a good Thanksgiving break and stay safe!