Who am I
Maciej Maciaszek
- Django/Python Developer at BlackCherry Software
- Frontend Developer (JS, CSS3, HTML5)
- Graphic Designer
- Trumpeter
Contact
maciej.maciaszek@gmail.com
twitter: https://twitter.com/maciejmaciaszek
phone: +48 666 813 283
Test Driven Development¶
to do or not to do?¶
Unittests - ABC
Python docs
https://docs.python.org/2/library/unittest.html
Dive into Python
http://www.diveintopython.net/unit_testing/index.html
Django
Everyone tests¶
- manual testing
printstatement
asserts
- unit tests, system tests
- QA
Test Driven Development in brief
It's software development method which originated from Extreme Programming - created by Kent Beck.
In his book "TDD by example" Beck states:¶
- Don’t write a line of new code unless you first have a failing automated test.
- Eliminate duplication.
Some of the technical implications are:¶
- You must design organically, with running code providing feedback between decisions
- You must write your own tests, since you can’t wait twenty times a day for someone else to write a test
- Your development environment must provide rapid response to small changes
- Your designs must consist of many highly cohesive, loosely coupled components, just to make testing easy
TDD Cycle
Which translates to:¶
write a little test that doesn’t work, perhaps doesn’t even compile at first
make the test work quickly, committing whatever sins necessary in the process
eliminate all the duplication created in just getting the test to work
Why "not to do"?¶
Criticism
- Big time investment
- Additional Complexity
- Design Impacts
- Continuous Tweaking
- TDD doesn't scale
- TDD is shortsighted it misses overall design perspective
- TDD does not consider worst-case scenarios
- TDD doesn't give you confidence that the code works
- TDD freezes the API too early
Why "to do"?¶
Acclaim
We develop without fear and gain confidency
We spend less time debugging.
- The tests act as accurate, precise, and unambiguous documentation at the lowest level of the system.
- Writing tests first requires decoupling that other testing strategies do not; and we believe that such decoupling is beneficial.
- Fast Feedback Loop (developing in small steps)
- TDD leads to better design and code quality
- Increased protection from defects, especially regressions
Big time investment
Django Project¶
.
├── manage.py
├── prefix
│ ├── __init__.py
│ ├── admin.py
│ ├── migrations
│ │ └── __init__.py
│ ├── models.py
│ ├── tests.py
│ └── views.py
├── requirements.txt
└── tdd
├── __init__.py
├── settings.py
├── urls.py
├── views.py
└── wsgi.py
%load prefix/tests.py
from lxml import html
from django.test import TestCase, RequestFactory
from prefix.views import prefix_list
class PrefixViewTest(TestCase):
def setUp(self):
self.factory = RequestFactory()
def test_root_url_resolves_to_prefix_list_view(self):
view = resolve('/')
self.assertEquals(view.func, prefix_list)
!./manage.py test
Let's make it ...
%load prefix/views.py
from django.shortcuts import render
# Create your views here.
prefix_list = None
SIC!¶
!./manage.py test
%load tdd/urls.py
from django.conf.urls import patterns, include, url
from django.contrib import admin
urlpatterns = patterns('',
url(r'^$', 'prefix.views.prefix_list', name='prefix-list'),
)
!./manage.py test
%load prefix/views.py
from django.shortcuts import render
# Create your views here.
def prefix_list(request):
pass
!./manage.py test
Let's add another failing test¶
from django.core.urlresolvers import resolve
from lxml import html
from django.test import TestCase, RequestFactory
from prefix.views import prefix_list
class PrefixViewTest(TestCase):
def setUp(self):
self.factory = RequestFactory()
[...]
def test_view_returns_proper_html(self):
request = self.factory.get('xxx')
html_tree = html.fromstring(prefix_list(request))
self.assertIn('Prefix List', html_tree.xpath('//title/text()'))
prefix_table = html_tree.xpath('//table#prefix-list')[0]
self.assertTrue(len(prefix_table))
self.assertEquals(len(prefix_table.xpath('.//tr/td')), 1)
!./manage.py test
Make it green ... and so on ...¶
Is it a time waste or an investment?¶
Writing tests for small functions and easy things must be time wasting
Without TDD / UnitTests I can get things done in half time
Unit Tests are a measure of completion.
Found on Twitter #TDD:
You think you're done without TDD but it's just the beginning
Maintaining test code is time consuming
so is manual testing, debugging and regression
Programming by wishful thinking
Isolation is a "design damage"
# parser.py
def strip_multiple_whitespaces(text):
[...]
class Parser(object):
[...]
def _log(self, *args):
[...]
def _get_comments_date_body_xpaths(self):
[...]
def _get_comments_date_body(self, comment_tree):
comment_xpaths = self._get_comments_date_body_xpaths()
try:
comment_date = comment_tree.xpath(comment_xpaths[0])[0]
comment_body = comment_tree.xpath(comment_xpaths[1])[0]\
.text_content()
except IndexError:
self._log('wystąpił błąd z xpathami')
raise
return comment_date, strip_multiple_whitespaces(comment_body)
Mocks
Objects that mimic behaviour of other objects
For Python 3: https://docs.python.org/3/library/unittest.mock.html
For Python 2: http://www.voidspace.org.uk/python/mock/
from unittest.mock import MagicMock
from prefix.models import Comment
comment = Comment()
Comment.objects.filter = MagicMock(return_value=comment)
retrieved_comment = Comment.objects.filter(hash_='asdf')
Comment.objects.filter.assert_called_with(hash_='asdf')
Using mocks does tie you to specific ways of using an API. This is one of the many trade-offs involved in the use of mock objects.
# tests.py
import unittest
from lxml import html
from unittest import TestCase, mock
class ParserTestCase(TestCase):
def setUp(self):
self.parser = Parser()
@mock.patch('__main__.strip_multiple_whitespaces') # we have test for that
@mock.patch.object(Parser, '_get_comments_date_body_xpaths') # xpaths may change
def test__get_comments_date_body(self, MockClass, mock_strip):
MockClass.return_value = '//span[@itemprop="datePublished"]/text()', \
'//div[@class="patient-text"]'
body = 'ASDF'
comment_html = """
<html>
<div class="comment clearfix" itemprop="review" itemscope=""
itemtype="http://schema.org/Review">
<div class="patient-info">
<h4 itemprop="author">Anonim</h4>b
<span class="date-info"
itemprop="datePublished">11.02.2011<br><em>18:17</em></span>
</div>
<div class="patient-points">
<div class="patient-text" itemprop="description"><p>
%s
</p></div>
</div>
</div>
</html>
""" % body
comment_tree = html.fromstring(comment_html)
mock_strip.return_value = body
date, body = self.parser._get_comments_date_body(comment_tree)
self.assertEquals(date, '11.02.2011')
self.assertEquals(body, 'ASDF')
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(ParserTestCase)
unittest.TextTestRunner().run(suite)
More isolated test
# tests.py
import unittest
from lxml import html
from unittest import TestCase, mock
class ParserTestCase(TestCase):
def setUp(self):
self.parser = Parser()
@mock.patch('__main__.strip_multiple_whitespaces') # we have test for that
@mock.patch.object(Parser, '_get_comments_date_body_xpaths') # xpaths may change
def test__get_comments_date_body(self, MockClass, mock_strip):
[...]
@mock.patch('__main__.Parser._log')
@mock.patch.object(Parser, '_get_comments_date_body_xpaths')
def test__get_comments_date_body_log_when_bad_xpaths(self, mock_xpaths, mock_log):
mock_xpaths.return_value = ('abc', 'def')
comment_tree = mock.MagicMock()
comment_tree.xpath.return_value.__getitem__.side_effect = IndexError
try:
self.parser._get_comments_date_body(comment_tree)
except IndexError:
self.assertTrue(mock_log.called)
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(ParserTestCase)
unittest.TextTestRunner().run(suite)
TDD guaratees fast feedback loop
- You can discover early wheter an implementation is correct
- You can detect bugs at early stage
# models.py
from django.db import models
from django.contrib.auth.models import User
# Create your models here.
class Comment(models.Model):
comment_hash = models.CharField(max_length=32)
author = models.ForeignKey(User)
body = models.TextField()
comment_date = models.DateTimeField()
# parser.py
class Parser(object):
def _check_comment_exists(self, comment_hash):
raise NotImplementedError()
def _get_comment_data(self, comment):
raise NotImplementedError()
def insert_comment(self, comment_hash, comment_body, comment_date):
comment = Comment.objects.create(
comment_hash=comment_hash, body=comment_body,
comment_date=comment_date, # we miss author
)
def parse_comment(self, request, comment_tree):
hash_, body_, date_ = self._get_comment_data(comment_tree)
if not self._check_comment_exists(hash_):
return self.insert_comment(hash_, body_, date_)
else:
self._debug('Komentarz istnieje i nie zostanie dodany')
# tests.py
import unittest
from unittest.mock import patch, MagicMock
from lxml import html
class ParserTestCase(unittest.TestCase):
def setUp(self):
self.parser = Parser()
[...]
@patch('__main__.Parser.insert_comment')
@patch('__main__.Parser._get_comment_data')
@patch('__main__.Parser._check_comment_exists', new=MagicMock(return_value=False))
def test_parse_comment_insert_comment_comment_not_exists(
self, mock_get_data, mock_insert):
request = None
hash_ = 'hash'
some_date = '2014-02-01'
body = 'comment body'
mock_get_data.return_value = hash_, body, some_date
self.parser.parse_comment(request, comment_tree=None)
self.assertTrue(mock_insert.called)
mock_insert.assert_called_with(hash_, body, some_date)
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(ParserTestCase)
unittest.TextTestRunner().run(suite)
$ ./manage.py test
Traceback (most recent call last):
[...]
django.db.utils.IntegrityError: NOT NULL constraint failed: prefix_comment.author_id
OR¶
# tests.py
import unittest
from unittest.mock import patch, MagicMock
from lxml import html
class ParserTestCase(unittest.TestCase):
def setUp(self):
self.parser = Parser()
[...]
@patch('__main__.Parser.insert_comment')
@patch('__main__.Parser._get_comment_data')
@patch('__main__.Parser._check_comment_exists', new=MagicMock(return_value=False))
def test_parse_comment_insert_comment_comment_not_exists(
self, mock_get_data, mock_insert):
request = None
hash_ = 'hash'
some_date = '2014-02-01'
body = 'comment body'
mock_get_data.return_value = hash_, body, some_date
comment = self.parser.parse_comment(request, comment_tree=None)
self.assertEquals(comment, mock_insert.return_value)
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(ParserTestCase)
unittest.TextTestRunner().run(suite)
Isolation
good for individual layers, doesn't verify itegration between them
Contracts
thinking in terms of objects colaboration
We missed one test
# tests.py
import unittest
from unittest.mock import patch, MagicMock
from lxml import html
class ParserTestCase(unittest.TestCase):
def setUp(self):
self.parser = Parser()
[...]
@patch('__main__.Parser.insert_comment')
@patch('__main__.Parser._get_comment_data')
@patch('__main__.Parser._check_comment_exists', new=MagicMock(return_value=False))
def test_parse_comment_insert_comment_comment_not_exists(
self, mock_get_data, mock_insert):
[...]
def test_insert_comment_returns_comment_object(self): # placeholder
self.fail('Fixme')
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(ParserTestCase)
unittest.TextTestRunner().run(suite)
Test could look like:¶
class Parser(object):
[...]
def insert_comment(self, comment_hash, comment_body, comment_date):
comment = Comment.objects.create(
comment_hash=comment_hash, body=comment_body,
comment_date=comment_date, # we miss author
)
return comment
# tests.py
import unittest
from unittest.mock import patch, MagicMock
from lxml import html
class ParserTestCase(unittest.TestCase):
def setUp(self):
self.parser = Parser()
[...]
@patch('__main__.Comment.save')
def test_insert_comment_returns_comment_object(self, mock_save):
expected = hash_, body_, date_ = 'hash', 'body', '2013-12-12'
comment = self.parser.insert_comment(hash_, body_, date_)
actual = comment.comment_hash, comment.body, comment.comment_date
self.assertEquals(actual, expected)
if __name__ == '__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(ParserTestCase)
unittest.TextTestRunner().run(suite)
ATDD comes in¶
Example¶
from django.test import LiveServerTestCase
class PrefixTest(LiveServerTestCase):
def setUp(self):
self.browser = Browser('chrome')
self.driver = self.browser.driver
# expensive setup
[...]
def tearDown(self):
self.patcher.stop()
self.browser.quit()
@patch('interface.modelforms.Prefix.is_unique', return_value=True)
def test_can_add_new_prefix_on_frontpage(self, MockClass):
"""
Can add prefix through frontpage form
"""
# Jarek wants to add prefix
PREFIX = '1234'
PREFIX_LENGTH = 9 - 2 # minus area
# Jarek visits index page but he gets login page
self.browser.visit(self.live_server_url)
# he types his username and password
usr_inp = self.browser.find_by_id('id_username')
pass_inp = self.browser.find_by_id('id_password')
usr_inp.type('login')
pass_inp.type('pass')
pass_inp.type(Keys.RETURN)
# now he tries to add new prefix
p_number_inp = self.browser.find_by_id('id_number')
p_number_inp.type(PREFIX)
prefix_form_id = '#prefix_add_form'
p_area = Select(self.driver.find_element_by_css_selector(
'%s [name="area"]' % prefix_form_id))
p_area.select_by_visible_text('%(name)s (%(wsn)s)' % self.area_config)
self.browser.find_by_css('%s button' % prefix_form_id).click()
# he's redirected to extended form
extended_form_id = '#colright form'
prefix_inp = self.driver.find_element_by_css_selector(
'%s [name="prefix"]' % extended_form_id)
# he should see prefix in first input
self.assertEquals(prefix_inp.get_attribute('value'), PREFIX)
# he should see proper area name below
area_name = self.driver.execute_script("""return $();""")
[...]
OUTSIDE-IN Methodology¶
breaking ...
YAGNI¶
You ain't gonna need it
To have 100% coverage you should have:
- Acceptance/Functional tests for full-stack testing (top layer)
- Integration tests (usually middle layer)
- Unittests (bottom layer)
Some Continuous Integration System is needed
TDD leads to better design and code quality
- can we have good quality code without it?
- decoupling with easy testing in mind leads to good design?
The tests act as documentation at the lowest level of the system.
TDD freezes the API too early
Fluid code. Uncertainty
XP Spike Solution¶
Increased protection from defects, especially regressions
- The user interface is really hard to test.
- You may be working on a legacy system.
TDD does not consider worst-case scenarios
- What do you do with tests which are expected to pass?
- TDD aimed only on tests to implement the code you think is right
TDD creates unit tests which are used to develop and refactor code.
One of the ironies of TDD is that it isn't a testing technique (the Cunningham Koan). It's an analysis technique, a design technique, really a technique for structuring all the activities of development.
Kent Beck, "Test-driven development: by example"
Can we estimate the costs and benefits of TDD?¶
Author studied academic articles and came to conlusions
Conclusions
There is a correlation between more tests and a reduced number of defects.
Any increase in a team performance through the use of TDD is either very difficult or illusionary.
Erdogmus et al reported in [1] that there was an improvement of productivity while Bhat et al [5] reported that TDD took an estimated 25%-35% longer. Erdogmus et al also note in section 2.1
“George and Williams [12] later conducted a formal TDD experiment with professional pair programmers. They reported that the TDD pairs’ product quality was on average 18 percent higher than that of the non-TDD pairs, while their productivity was 14 percent lower.”
[...] no empirical material reporting on the design quality effects TDD is proposed to have impact on
Claims of improved quality are supported by all the articles that I reviewed. However, there is very little data to support claims of improved productivity or software design.
Conclusions¶
- Try it!
- TDD is difficult. Remember about a learning curve!
- Whole team must do it!
- You'll get better code quality (TDD as learning tool?)
- You'll slow down about 30% and become less productive
- You'll get less complaints from customers
Reading
Books
Test Driven Development: By Example Paperback – November 18, 2002 by Kent Beck
Growing Object-Oriented Software, Guided by Tests Paperback – October 22, 2009 by Steve Freeman
Internet
http://blog.8thlight.com/uncle-bob/2014/04/30/When-tdd-does-not-work.html
http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html
http://chimera.labs.oreilly.com/books/1234000000754/index.html
http://en.wikipedia.org/wiki/SOLID_(object-oriented_design)
http://en.wikipedia.org/wiki/Design_by_contract
http://agiledata.org/essays/tdd.html
http://blog.8thlight.com/uncle-bob/2014/04/25/MonogamousTDD.html
https://www.youtube.com/watch?v=z9quxZsLcfo
http://www.agile-itea.org/public/deliverables/ITEA-AGILE-D2.7_v1.0.pdf
http://scrumology.com/the-benefits-of-tdd-are-neither-clear-nor-are-they-immediately-apparent/