[C++] Start Using Cucucmber Part 2. This Time With CWT-Cucumber

Feb 26

Written By Thomas

In this article we will go through CWT-Cucumber: A C++ Cucumber interpreter. This started as a side project and turned into a nice project for me. I've read Crafting Interpreters by Robert Nystrom and it's one of the best books I've ever read. In his book he creates a general purpose scripting language and he guides you through two different implementations.

Once I finished his book with his scripting language, I needed a real-world example where I could try to make one of my own. But writing another general purpose language would just be a copy of his.

So I thought of a good project and I really like using Cucumber. There is a Cucumber C++ project by Cucucmber himself on GitHub, but it's not natively implemented in C++ and there are some difficulties:

You have to install ruby to parse Cucumber files.
Dependencies: Building is often difficult and has a lot of dependencies
Missing Conan package, currently there is no official Conan support (if there is, please let me know)

And that means that applying the knowledge from Crafting Interpreters to start a Cucumber C++ interpreter seemed like a reasonable and worthwhile project. And well, here we are, I have created a CWT-Cucumber: A C++20 Cucumber interpreter, without dependencies that addresses the problems I mentioned above. So let's get started.

Before We Start ...

... here is a short overview of my GitHub repos to this. I separated CWT-Cucumber, the conan recipe and the documentation in three different GitHub repos:

CWT-Cucumber: A C++ Cucumber interpreter. This is the interpreter itself, you can star it for updates.

CWT Cucumber Conan recipe. I'll use this directory within this article.

Documentation: This repo builds the documentation and pushes it to GitHub pages.

Setup

I created a simple Conan file to build the package and use Conan 2.x to integrate it. In the long run it makes sense to publish this package to the Conancenter, but for now we just have to build it locally. It isn't a big deal.

Fire up a terminal and clone my github repo and we build the package:

git clone https://github.com/ThoSe1990/cwt-cucumber-conan.git
cd ./cwt-cucumber-conan/package
conan create . --version 1.2.1 --user cwt --channel stable
cd ../examples

And now we're ready to build the examples:

conan install . -of ./build 
cmake -S . -B ./build -DCMAKE_TOOLCHAIN_FILE="./build/conan_toolchain.cmake"
cmake --build ./build

Note: If you're using Windows, your build directory with the executables is ./build/bin/Debug (or Release) and not ./build/bin. Adapt the paths accordingly if needed.

Box as Example

Before we start with the example, let us have a look at the box class we are using:

class box
{
public:
  void add_item(const std::string &item) { m_items.push_back(item); }
  void add_items(const std::string &item, std::size_t count)
  {
    for ([[maybe_unused]] int i = 0; i < count; i++)
    {
      add_item(item);
    }
  }
  [[nodiscard]] std::size_t items_count() const noexcept
  {
    return m_items.size();
  }
  void close() noexcept { m_is_open = false; }

private:
  bool m_is_open{true};
  std::vector<std::string> m_items;
};

I think this is pretty self-explanatory. We have a simple box where we can put items, we can close the box and either the box is open or closed, and we can check how many items are in the box.

First Scenario

Now we are going to create our first test. To do this, we will create our first scenario in the first feature file:

Feature: My first feature

  Scenario: First Scenario
    Given An empty box
    When I place 2 x "apple" in it
    Then The box contains 2 item(s)

And to execute that step, we need to implement the appropriate functions to execute on each step. Therefore we use the macros provided by CWT-Cucumber:

GIVEN(function_name, "your step goes here")
WHEN(function_name, "your step goes here")
THEN(function_name, "your step goes here")
Alternatively just use STEP(function_name, "your step goes here")

Note: Each step results in a simple function. Therefore, we need to specify a function name.

GIVEN(init_box, "An empty box")
{
  const box& my_box = cuke::context<box>();
  cuke::equal(my_box.items_count(), 0);
}

WHEN(add_item, "I place {int} x {string} in it")
{
  const std::size_t count = CUKE_ARG(1);
  const std::string item = CUKE_ARG(2);

  cuke::context<box>().add_items(item, count);
}

THEN(check_box_size, "The box contains {int} item(s)")
{
  const int items_count = CUKE_ARG(1);
  const box& my_box = cuke::context<box>();
  cuke::equal(my_box.items_count(), items_count);
}

And now we can build and run our first scenario (of course after the build which we already did) and we get the results:

./build/bin/box ./features/1_first_scenario.feature

Feature: My first feature  ./features/1_first_scenario.feature:2

Scenario: First Scenario  ./features/1_first_scenario.feature:5
[   PASSED    ] An empty box  ./features/1_first_scenario.feature:6
[   PASSED    ] I place 2 x "apple" in it  ./features/1_first_scenario.feature:7
[   PASSED    ] The box contains 2 item(s)  ./features/1_first_scenario.feature:8

1 Scenarios (1 passed)
3 Steps (3 passed)

And cool, the first scenario passes. So let's break down the components of this test into their implementation details.

Steps

With macros we can structure the code with keywords. And their purpose is only to structure the code better, there is no difference between them. Alternative: We could also use STEP(..,..), which is a more general macro.

Values in a Step

To define the values we need to access, we use Cucumber expressions. That is, between the braces we define the type we expect. Currently CWT supports Cucumber: {byte} , {short}, {int}, {long}, {float}, {double} and {string}.

Access Values

Now we need to access these values in our step definition. So we use the define CUKE_ARG(..) to get a value. The index starts with 1 from the left. That is, if (as above) the first value is an integer and the second is a string, we use

const std::size_t count = CUKE_ARG(1);
const std::string item = CUKE_ARG(2);

Note: I use the implicit conversion operator underneith, this means auto type deduction will not work here.

Object lifetime management with `cuke::context`

Let us take a quick look at object lifetime. During a scenario we need to store an object to be able to access it in each step. For this we have cuke::context. Here we can insert a type and its lifetime is guaranteed as long as a scenario is running. For each object type we can add one to the context. See this short example:

// forwards 1,2,3 to your object: 
cuke::context<some_object>(1,2,3);
// access or default initialize your object: 
cuke::context<some_object>();

// if you have multiple objects, no problem just put the next type into cuke::context:
cuke::context<std::string>("we'll need this later");
cuke::context<some_other_obj>();

// To modify the object or call member function we can do this direct:
cuke::context<some_object>().do_some_work();
// or take the reference:
some_object& obj = cuke::context<some_object>();

And if you go back and look at our first test, you'll notice that I always used box in cuke::context to run the first test.

Scenario Outline

Often test cases are pretty similar to each other or exactly the same but with different values. To do so we can create a Scenario Outline:

  Scenario Outline: First Scenario Outline
    Given An empty box
    When I place <count> x <item> in it
    Then The box contains <count> item(s)

    Examples:
      | count | item      |
      | 1     | "apple"   |
      | 2     | "bananas" |

      

And now we have a sort of template for a scenario that runs twice, for each row in the examples table. First, we run the scenario with the first row of values, followed by the second. When we run it, we get this output:

./build/bin/box ./features/2_scenario_outline.feature

Feature: My first feature  ./features/2_scenario_outline.feature:2

Scenario Outline: First Scenario Outline  ./features/2_scenario_outline.feature:5
[   PASSED    ] An empty box  ./features/2_scenario_outline.feature:6
[   PASSED    ] I place  x  in it  ./features/2_scenario_outline.feature:7
[   PASSED    ] The box contains  item(s)  ./features/2_scenario_outline.feature:8
  With Examples:
  | count | item      |
  | 1     | "apple"   |

Scenario Outline: First Scenario Outline  ./features/2_scenario_outline.feature:5
[   PASSED    ] An empty box  ./features/2_scenario_outline.feature:6
[   PASSED    ] I place  x  in it  ./features/2_scenario_outline.feature:7
[   PASSED    ] The box contains  item(s)  ./features/2_scenario_outline.feature:8
  With Examples:
  | count | item      |
  | 2     | "bananas" |


2 Scenarios (2 passed)
6 Steps (6 passed)

Background

If we have scenarios that require all the same steps at the beginning, we can define a background. Say we want a box with an apple already placed in it by default:

Feature: We always need apples!
  
  Background: Add an apple 
    Given An empty box
    When I place 1 x "apple" in it

  Scenario: Apples Apples Apples
    When I place 1 x "apple" in it
    Then The box contains 2 item(s)

  Scenario: Apples and Bananas
    When I place 1 x "apple" in it
    And I place 1 x "banana" in it
    Then The box contains 3 item(s)

This means that for all Scenarios under this Feature we first run the Background, as you can see in the terminal output:

./build/bin/box ./features/3_background.feature

Feature: We always need apples!  ./features/3_background.feature:1

Scenario: Apples Apples Apples  ./features/3_background.feature:7
[   PASSED    ] An empty box  ./features/3_background.feature:4
[   PASSED    ] I place 1 x "apple" in it  ./features/3_background.feature:5
[   PASSED    ] I place 1 x "apple" in it  ./features/3_background.feature:8
[   PASSED    ] The box contains 2 item(s)  ./features/3_background.feature:9

Scenario: Apples and Bananas  ./features/3_background.feature:11
[   PASSED    ] An empty box  ./features/3_background.feature:4
[   PASSED    ] I place 1 x "apple" in it  ./features/3_background.feature:5
[   PASSED    ] I place 1 x "apple" in it  ./features/3_background.feature:12
[   PASSED    ] I place 1 x "banana" in it  ./features/3_background.feature:13
[   PASSED    ] The box contains 3 item(s)  ./features/3_background.feature:14


2 Scenarios (2 passed)
9 Steps (9 passed)

Tables

Data tables or tables in Cucumber are very handy to provide values in one step. There are three types of tables in CWT Cucumber 1.2.1:

Raw tables
Hashes
Key/Value pairs or rows hash

The following subchapters cover all three types.

Raw tables

A raw table is just an arbitrary table, as the name implies. Consider following Scenario:

Scenario: Adding items with raw
  Given An empty box
  When I add all items with raw():
    | apple      | 2 |
    | strawberry | 3 |
    | banana     | 5 |
  Then The box contains 10 item(s)

And here we can now iterate over each row and access their values, see this step implementation:

WHEN(add_table_raw, "I add all items with raw():")
{
  // create a table 
  const cuke::table& t = CUKE_TABLE();

  // with raw() you iterate over all rows 
  for (const cuke::table::row& row : t.raw())
  {
    // and with the operator[] you get access to each cell in each row
    cuke::context<box>().add_items(row[0].to_string(), row[1].as<long>());
  }
}

And when it comes to arrays this can be pretty useful.

Note: The underlying value is a cuke::value that we are accessing here. We can access it with as where we pass the dedicated type or we can use to_string to convert any value to a string here.

Alternatively, there is another raw access. You can use the operator[]:

const cuke::table& t = CUKE_TABLE();
t[0][0].to_string();
t[0][1].as<int>();

And with this getter functions you have some meta information about the given table:

// returns the number of rows.
t.row_count();
// returns the number of cols.
t.col_count();
// returns the total count of cells:
t.cells_count();

Hashes

With an additional header in the table we can make this table more descriptive:

Scenario: Adding items with hashes
  Given An empty box
  When I add all items with hashes():
    | ITEM   | QUANTITY |
    | apple  | 3        |
    | banana | 6        |
  Then The box contains 9 item(s)

You can now iterate over the table using hashes() and access the elements with string literals:

WHEN(add_table_hashes, "I add all items with hashes():")
{
  const cuke::table& t = CUKE_TABLE();
  for (const auto& row : t.hashes())
  {
    cuke::context<box>().add_items(row["ITEM"].to_string(), row["QUANTITY"].as<long>());
  }
}

Key/Value Pairs or Rows Hash

Another descriptive way works for key value pairs, or rows hash. Note, that this works only for tables with two columns. The first column describes the element, the second holds the element:

Scenario: Adding items with rows_hash
  Given An empty box
  When I add the following item with rows_hash():
    | ITEM     | really good apples |
    | QUANTITY | 3                  |
  Then The box contains 9 item(s)

And with cuke::table::pair hash_rows = t.rows_hash(); you can create this hash map. The access to each element is again by the string literal.

WHEN(add_table_rows_hash, "I add the following item with rows_hash():") 
{
  const cuke::table& t = CUKE_TABLE();
  // cuke::table::pair is just an alias for a std::unordered_map which gets created in rows.hash()
  cuke::table::pair hash_rows = t.rows_hash();

  cuke::context<box>().add_items(hash_rows["ITEM"].to_string(), hash_rows["QUANTITY"].as<long>());
}

Hooks

Hooks are executed before and after each scenario or step. The implementation is pretty straightforward. You can have multiple hooks of the same type. All of them are executed at their time. The hook implementation is with the according define, similar to a STEP:

// ./examples/hooks.cpp: 
BEFORE(before)
{
  // this runs before every scenario
}
AFTER(after)
{
  // this runs after every scenario
}
BEFORE_STEP(before_step)
{
  // this runs before every step
}
AFTER_STEP(after_step)
{
  // this runs after every step
}

I don't have any default hooks prepared for my box showcase. But just implement some and you will see the execution. But there is a showcase for hooks in the next subchapter, for tagged hooks.

Tagged Hooks

You can add a tag expression to your hook. Use

BEFORE_T(name, "tags come here") for a tagged hook before a scenrio
AFTER_T(name, "tags come here") for a tagged hook after a scenario

This means a tagged hook is executed, when a scenario fulfills the given condition. You can pass in any logical expression to a tagged hook:

AFTER_T(dispatch_box, "@ship or @important"){
  std::cout << "The box is shipped!" << std::endl;
}

Feature: Scenarios with tags

  @ship
  Scenario: We want to ship cucumbers
    Given An empty box
    When I place 1 x "cucumber" in it
    Then The box contains 1 item(s)
   
  @important
  Scenario: Important items must be shipped immediately
    Given An empty box
    When I place 1 x "important items" in it
    Then The box contains 1 item(s)

And now we can see that our box has been shipped, the terminal print now comes from the hook. We don't have to specify a hook in the terminal command because the hook is executed according to the hooks in the feature file:

./build/bin/box ./features/5_tagged_hooks.feature

Feature: Scenarios with tags  ./features/5_tagged_hooks.feature:1

Scenario: We want to ship cucumbers  ./features/5_tagged_hooks.feature:4
[   PASSED    ] An empty box  ./features/5_tagged_hooks.feature:5
[   PASSED    ] I place 1 x "cucumber" in it  ./features/5_tagged_hooks.feature:6
[   PASSED    ] The box contains 1 item(s)  ./features/5_tagged_hooks.feature:7
The box is shipped!

Scenario: Important items must be shipped immediately  ./features/5_tagged_hooks.feature:10
[   PASSED    ] An empty box  ./features/5_tagged_hooks.feature:11
[   PASSED    ] I place 2 x "important items" in it  ./features/5_tagged_hooks.feature:12
[   PASSED    ] The box contains 2 item(s)  ./features/5_tagged_hooks.feature:13
The box is shipped!

2 Scenarios (2 passed)
6 Steps (6 passed)

Running Single Scenarios / Full Directories

If you only want to run single scenarios, you can append the appropriate line to the feature file:

This runs a Scenario in Line 6:

./build/bin/box ./features/box.feature:6

This runs each Scenario in line 6, 11, 14:

./build/bin/box ./examples/features/box.feature:6:11:14

If you want to execute all feature files in a directory (and subdirectory), just pass the directory as argument:

./build/bin/box ./examples/features

Conclusion

Well, here we are. It was and will be a fun and interesting project for me. I have learned a lot and enjoyed it. Lets see how this project continues.

If there are things unclear, you need support, you found bugs or you need features, please feel free to contact me or join me in contributing to this project.

I separated CWT-Cucumber, the conan recipe and the documentation in three different GitHub repos:

CWT-Cucumber: A C++ Cucumber interpreter. This is the interpreter itself, you can star it for updates.

CWT Cucumber Conan recipe. I'll use this directory within this article.

Documentation: This repo builds the documentation and pushes it to GitHub pages.

c++cucumbertestingbdd

Thomas

[C++] Start Using Cucucmber Part 2. This Time With CWT-Cucumber

Before We Start ...

Setup

Box as Example

First Scenario

Steps

Values in a Step

Access Values

Object lifetime management with `cuke::context`

Scenario Outline

Background

Tables

Raw tables

Hashes

Key/Value Pairs or Rows Hash

Tags

Only with tag banana:

Bananas or apples:

Hooks

Tagged Hooks

Running Single Scenarios / Full Directories

Conclusion

[C++] Start Using Cucucmber Part 2. This Time With CWT-Cucumber

Before We Start ...

Setup

Box as Example

First Scenario

Steps

Values in a Step

Access Values

Object lifetime management with cuke::context

Scenario Outline

Background

Tables

Raw tables

Hashes

Key/Value Pairs or Rows Hash

Tags

Only with tag banana:

Bananas or apples:

Hooks

Tagged Hooks

Running Single Scenarios / Full Directories

Conclusion

How to Write a C++ Test Framework

[C++] Lets Review my Article on Type Erasure

Object lifetime management with `cuke::context`