writing test and BDD

this post will focus on no side effect environment module testing, and the code is not in any programming language :P

intro

let's first take a look at simple how-to write a test that I sees people usually do now
suppose we have module (or class) Cart
we will follow the OO concept of encapsulating state as private and have a public method

module Cart
  private items

  public fn isEmpty()
  public fn addItem(item)
  public fn checkOut()

just to be simple we will use item as string "A", "B" and the checkout will just print format list of purchased items

> cart.addItem("A")
> cart.checkout()
Order: A
> cart.addItem("B")
> cart.checkout()
Order: A, B

now let's start writing a test on each function, start with the very easy first test on isEmpty()

module CartTest
  describe "isEmpty()"
    test "it return true when no item"
      cart = new Cart()
      assert cart.isEmpty() == true

this is so simple. let's see the next method addItem()

describe "addItem()"
  test "it return ..."

more complicate

we already stuck at the name of test. this method just add the item to cart without returning any result
you can make a method return boolean to represent the success/fail result but it still not what we expect a method to do, to actually put item in the cart
instead of checking the result, we want to check the state of the cart for it's items but the items property is private
the quick easy way would be

module Cart
-  private items
+  public items

or

module Cart
+  public fn getItems()

we change the items to be exposed to public scope. now we can test

describe "addItem()"
  test "it add item to cart's items list"
    cart = new Cart()
    assert cart.items == []   
    cart.addItem("A")
    assert cart.items == ["A"]

but this break a concept of encapsulation and make a test become a white box testing. we would prefer black box approach because we don't want the detail and logic of module to grow outside of itself and become hard to understand

let's just keep it like this and see the next test
the isEmpty() test we wrote is not covered all the cases it can be. also we want to test addItem() with multiple item.
we need to setup test with already added item. someone may come up with this solution

describe "isEmpty()"
  test "it return true when no item"
    ...

  test "it return false when cart have items"
    cart = new Cart()
    cart.items = ["A"]
    assert cart.isEmpty() == false

describe "addItem()"
  test "it add item to cart"
    ...

  test "it can add more item to cart"
    cart = new Cart()
    cart.items = ["A"]
    cart.addItem("B")
    assert cart.items == ["A", "B"]

by setting items directly (or create public fn setItems to set it). you can isolate a test to just it's testing method.
also if some method require more pre-setup state, setting up state directly often be the quick way to do.

the problem is, this is not the actual usage of your module. by directly setting state, it can happen to be a "shouldn't exists state" which mean you are testing the thing that not actually exists in real code

  module Cart
    public isActive
    public isDelete

    public fn activate()
      this.isActive = true
      this.isDelete = false

  public fn delete()
      this.isActive = false
      this.isDelete = true

cart = new Cart()
cart.isActive = true
cart.isDelete = true

also as mentioned before, the state shouldn't be exposed out of module. so this kind of setup is not a way we want to do.

taking a step back

if you just want code and howto, skip this part :P

for now we have 2 problems

• we want to test the fn that modify state but don't want to expose the state
• we want to setup pre-state but also don't want to expose the state

this may feel like a paradox that cannot be solved. But let's take a step back and look at the bigger picture of module with methods and state.

imagine if we build a boiler machine.
we may not want to wire the power into the heater part to test that this machine can boil the water without testing the turn on button.
for this context, we want to test the complete machine not a component

the same with module, many times that it is designed that each method cannot live by itself, but work together as a state diagram

look at this example state machine diagram
not all transition is drawn out from the initial state so it shouldn't work without previous transition and you shouldn't jump to any state without transition

apply this with code module, not all method will work without setup by some other method and you can't set the state directly.

for this I want you to forget about testing individual method and instead focus on testing the Behavior of entire module or BDD

yes BDD is not just forcing test name with awkward Given When Then and don't need to be on the top level end-to-end or feature test

Let's do BDD

from the example above we may see that testing single method is not work well with encapsulated black box logic because the whole module isn't mean to be used by just that single method alone.
let's look at the problem we found again

describe "addItem()"
  test "it add item to cart's items list"
    cart = new Cart()
    assert cart.items == []   
    cart.addItem("A")
    assert cart.items == ["A"]

because we can't verify the result of addItem() alone but we can look in another way, the addItem() is actually the passage to the checkout(). So we will test both of them together

describe "checkout()"
  test "it print order with single item"
    cart = new Cart()
    cart.addItem("A")
    assert cart.checkout == "Order: A"

we don't care to describe what addItem() do behind the scene. we only know that we need it before checkout(). so there is nothing like list or something internal in the test at all

the next problem was about setup pre-state, again we can just use the addItem together

describe "isEmpty()"
  test "it return false when cart have items"
    cart = new Cart()
    cart.addItem("A")
    assert cart.isEmpty() == false

describe "checkout()"
  test "it print order with single item"
    ...

  test "it print order with multiple item"
    cart = new Cart()
    cart.addItem("A")
    cart.addItem("B")
    assert cart.checkout == "Order: A, B"

and because we don't focus a test to just single method, let's remove a describe with the method name from tests and rename some tests

module CartTest
- describe "isEmpty()"
    test "new cart is empty"
    test "cart with single item is not empty"
    test "cart with multiple items is not empty"

- describe "checkout()"
    test "new cart cannot checkout"
    test "print order from cart with single item"
    test "print order from cart with multiple items"

we can see some pattern here. there are 3 tests per (removed) describe scope with the new cart, single item and multiple items in both of them. let's try group them together

describe "new cart"
  cart = new Cart()
  test "should be empty"
  test "cannot checkout"

describe "cart with single item"
  cart = new Cart()
  cart.addItem("A")
  test "should not be empty"
  test "should print order with single item"

describe "cart with multiple items"
  cart = new Cart()
  cart.addItem("A")
  cart.addItem("B")
  test "should not be empty"
  test "should print order with multiple items"

by grouping these together, not only we get the shorter name. we can also share the setup code in each describe block because it is tested on the same state

and if you try hard enough, you can phrase it to the given-when-then pattern.

given new cart
  when check empty
    then it return true 
  when checkout
    then it return error

given map directly to the setup state
when is the method or action we execute
then map expected result

I think this test pattern is a lot more easier to map to these GWT format than the previous per-method pattern because we test the module by actual Behavior thus BDD

Summary

• testing per method is hard because some method is not self-completed
• instead we want to test behavior of entire module using only public method (Black Box)
• we group test by pre setup state so we can share setup code
• the test is more reflect to real world use-case

Extra

I recommend to have multiple it or then per test if need. In this way it is easier to describe the behavior of your module
for example

when cancel an order, mark status as 'canceled', update timestamp and fire event order canceled

even grouping by state and describe logic in the test name, it can make test name and body too long

describe "created order"
  it "cancel order mark status as 'canceled', update timestamp and fire event"

  // and because its very long, we most likely do
  it "can cancel successfully"

better example: test by behavior, descriptive name, separate each assertion

describe "created order"
  describe "when cancel"
    test "it mark status as 'canceled"
    test "it update timestamp"
    test "it fire event"

separate each topic in it's own test/it/then is easier to add new requirement or adjust existing one and test report will looks very clean and can better describe behavior of the code