Continuous Behavior - BDD in Continuous Delivery (CoDers Who Test, Gothenburg, 15/5/2018)

Continuous
Behavior
BDD in Continuous Delivery
CoDers Who Test
15th May 2018
Gáspár Nagy
coach • trainer • bdd addict • creator of specflow
“The BDD Books: Discovery” • http://bddbooks.com
@gasparnagy • gaspar@specsolutions.eu

Copyright © Gaspar NagyCopyright © Gaspar Nagy
bdd addict
given.when.then
CAUTION!
on the stage
Gáspár Nagy
coach, trainer and bdd addict
creator of SpecFlow
gaspar@specsolutions.eu
https://specsolutions.eu
@gasparnagy

What makes you nervous on
the day of the release?

A long time ago* in a galaxy
far, far away**…
* 1995
** Veszprém, Hungary

My dream-projects in 1995
• Working software that was in use
• Exciting technical solutions
• Continuous learning
• And no conflicts in the team…
…because I was alone

My first “real” project…

Practicing team work

Practicing team work
And by a
modem for
“A”

Integration is more than merging…
changed
foo.cpp
changed
bar.cpp
?
void ProcessOrder()
{
…
+ account->Withdraw(sum);
…
}
void FinalizeOrder()
{
…
+ account->Withdraw(sum);
…
}

What is integration, then?
Ensuring that the improvements developed independently still form a working unit
Resolving file-
level conflicts
Can compile,
can build
Can be
deployed
There is no
regression
[Enter your
specific step
here]

Proper integration is difficult…

Frequency reduces difficulty (Fowler)
• „if it hurts, do it more often”

Continuous Integration, CI
Ensures that the improvements developed independently still form a working unit
• Resolving file-level conflicts
• Compilation, build
• Unit tests
• ...

Continuous integration revealed the next
problems…
• Manual regression testing does not scale…
• Manual deployments are error-prone…

Continuous Delivery, CD
Ensures that the application is always in a potentially deployable state (whether we
deploy is a manual decision)
• CI +
• Deployable
• Automated regression tests
(Acceptance Tests)
• ...

CD vs. CD
Continuous Delivery vs. Continuous Deployment

Continuous Delivery, Continuous Deployment

“Continues delivery can be achieved by anyone,
Continues deployment is only for some specific
products…”
/some people/

Continuous Deployment, CD
Ensures that the application is always actual
• CD +
• Fully* automated verification
• Automated non-functional tests
• Appropriate context (eg. SaaS)
• ...
• ???
„live”

Continuous Deployment is the key to understand the
“CD” mentality…
…so let’s try!

What are your fears?
• I am afraid of pushing the “release” button!
• The feedback I get from my tests are not useful!

More testing and more automation!
Source: World Quality Report 2015/16
increased by 35%
increased by 60%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Amount of IT budget allocated to QA Ratio of test case automation
2014
2015

Small tests = small problems;
Big tests = big problems
• They are SLOOOOOOW!
• They are BRITTLE!
• They need awful lot of MAINTENANCE!
• They FAIL!!!

Our typical automated tests are
not good enough for CD…

What’s wrong with our tests?

Episode #1
PO: We have far too many prod issues. We have
to improve our CD pipeline…
Write more tests!
Team: How much more?

Episode #2
PO: Are your tests balanced according to the test
automation pyramid?
Team: … not fully…, more like an ice-cream cone
PO: Then write more unit tests!

Episode #3
PO: You should increase your test coverage!
Team: You mean line coverage? How much?
PO: Well… what is exactly a code line doing?

Episode #4
PO: You should cover the risks better!
Team: How do we know if we have covered
them?

PO controlling testing activities…
Problem Domain Solution Domain
Lightbulb by Unknown Author is licensed under CC BY-SA
PO
REQ APP
TESTS

You cannot control something that is
outside of your control…

Tests should be part of shared ownership…
REQ APP
TESTS

How could the entire team
own the tests?

Tests should be part of shared ownership…
The old PIN
should be
correct
public void VerifyOldPin(string enteredOldPin)
{
var db = new CardRespoitory();
var oldPin = db.LoadOldPin(CurrentCard);
if (oldPin != enteredOldPin)
{
throw new InvalidPinException();
}
}
Implements
[TestMethod]
public void ShuldFailWhenOldPinIncorrect()
{
var cardStoreMock = new CardStoreMock();
cardStoreMock.SetCard("1234678", "1234");
var sut = new CardService();
var result = sut.ChangePin("1234678", "1111",
"1212", "1212");
Assert.IsFalse(result);
}

Specification by example
The old PIN
should be
correct
{
{
}
}
Implements
[TestMethod]
{
"1212", "1212");
}
Old PIN is incorrect
• PIN is 1234
• Enter old PIN: 1111
• Change PIN
⇒ PIN change rejected
Illustrates
Automates

Codified tests are hard to maintain
• What was the example?
• Why were these
numbers used?
• What did we want to
verify?
[TestMethod]
{
"1212", "1212");
}

BDD is here to help?

Formulated BDD scenarios
The old PIN
should be
correct
{
{
}
}
Implements
Old PIN is incorrect
• PIN is 1234
• Enter old PIN: 1111
• Change PIN
⇒ PIN change rejected
Illustrates
Scenario: Old PIN is incorrect
Given my old pin is 1234
And I enter 1111 as old PIN
When I try to change my PIN
Then the PIN change should be rejected [Then]
public void I_enter_PIN_as_old_pin(string pin)
{
changePinContext.SetOldPin(pin);
}
[When]
{
}
[Given]
{
}

Behavior Driven Development
REQ APP
Given
When
Then
SPECS
Validates Verifies

Behavior Driven Development is about
understanding & validating
business requirements
through illustrative examples

BDD is not about tests!

Functional quality is not only about whether
IT WORKS as expected…
but also about
whether the EXPECTATIONS ARE GOOD and
DOCUMENTED…
Functional quality

Episode in a later season…
PO: We have got a strange prod issue. Could we
look into that?
Team: Sure. Let’s have a look at the related BDD
scenarios…
PO: I see. I think there is an important case we
did not detail enough…

Happy ending

Finally I’ve got a dream-project again
• It was a complex service used by hundreds of apps and through them millions of
users all over the world
• Processed several thousands of requests per second
• Was critical in the client’s service chain
• We rolled it out without any severe issues…
• And we knew that there will be no problems…
• Because we had shared control over the specs (BDD scenarios)
• And this gave us confidence to push the release button

Continuous deployment is about CONFIDENCE
…and you need to keep the link to the problem
domain to gain that

Why was my first project working?
• Short cycles
• Instant release
• Close to “business” but also
close to users

Think about YOUR tests…
Are they also linking the solution with the problem?

Source: https://shashinki.com/

Thank you!
Gáspár Nagy
coach • trainer • bdd addict • creator of specflow
“The BDD Books: Discovery” • http://bddbooks.com
@gasparnagy • gaspar@specsolutions.eu

Continuous Behavior - BDD in Continuous Delivery (CoDers Who Test, Gothenburg, 15/5/2018)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Continuous Behavior - BDD in Continuous Delivery (CoDers Who Test, Gothenburg, 15/5/2018)

Similar to Continuous Behavior - BDD in Continuous Delivery (CoDers Who Test, Gothenburg, 15/5/2018) (20)

More from Gáspár Nagy

More from Gáspár Nagy (11)

Recently uploaded

Recently uploaded (20)

Continuous Behavior - BDD in Continuous Delivery (CoDers Who Test, Gothenburg, 15/5/2018)