Commit 2b29f617 authored by Rodolfo Marinho's avatar Rodolfo Marinho
Browse files

Catch dependency

# This sets the default behaviour, overriding core.autocrlf
* text=auto
# All source files should have unix line-endings in the repository,
# but convert to native line-endings on checkout
*.cpp text
*.h text
*.hpp text
# Windows specific files should retain windows line-endings
*.sln text eol=crlf
\ No newline at end of file
language: cpp
sudo: false
ccache: true
- $HOME/.ccache
# 1/ Linux Clang Builds
- os: linux
compiler: clang
addons: &clang35
sources: ['llvm-toolchain-precise-3.5', 'ubuntu-toolchain-r-test']
packages: ['clang-3.5']
env: COMPILER='ccache clang++-3.5' BUILD_TYPE='Release'
- os: linux
compiler: clang
addons: *clang35
env: COMPILER='ccache clang++-3.5' BUILD_TYPE='Debug'
- os: linux
compiler: clang
addons: &clang36
sources: ['llvm-toolchain-precise-3.6', 'ubuntu-toolchain-r-test']
packages: ['clang-3.6']
env: COMPILER='ccache clang++-3.6' BUILD_TYPE='Release'
- os: linux
compiler: clang
addons: *clang36
env: COMPILER='ccache clang++-3.6' BUILD_TYPE='Debug'
- os: linux
compiler: clang
addons: &clang37
sources: ['llvm-toolchain-precise-3.7', 'ubuntu-toolchain-r-test']
packages: ['clang-3.7']
env: COMPILER='ccache clang++-3.7' BUILD_TYPE='Release'
- os: linux
compiler: clang
addons: *clang37
env: COMPILER='ccache clang++-3.7' BUILD_TYPE='Debug'
- os: linux
compiler: clang
addons: &clang38
sources: ['llvm-toolchain-precise-3.8', 'ubuntu-toolchain-r-test']
packages: ['clang-3.8']
env: COMPILER='ccache clang++-3.8' BUILD_TYPE='Release'
- os: linux
compiler: clang
addons: *clang38
env: COMPILER='ccache clang++-3.8' BUILD_TYPE='Debug'
# 2/ Linux GCC Builds
- os: linux
compiler: gcc
addons: &gcc48
sources: ['ubuntu-toolchain-r-test']
packages: ['g++-4.8']
env: COMPILER='ccache g++-4.8' BUILD_TYPE='Release'
- os: linux
compiler: gcc
addons: *gcc48
env: COMPILER='ccache g++-4.8' BUILD_TYPE='Debug'
- os: linux
compiler: gcc
addons: &gcc49
sources: ['ubuntu-toolchain-r-test']
packages: ['g++-4.9']
env: COMPILER='ccache g++-4.9' BUILD_TYPE='Release'
- os: linux
compiler: gcc
addons: *gcc49
env: COMPILER='ccache g++-4.9' BUILD_TYPE='Debug'
- os: linux
compiler: gcc
addons: &gcc5
sources: ['ubuntu-toolchain-r-test']
packages: ['g++-5']
env: COMPILER='ccache g++-5' BUILD_TYPE='Release'
- os: linux
compiler: gcc
addons: *gcc5
env: COMPILER='ccache g++-5' BUILD_TYPE='Debug'
# 3/ OSX Clang Builds
- os: osx
osx_image: xcode6.4
compiler: clang
env: COMPILER='ccache clang++' BUILD_TYPE='Debug'
- os: osx
osx_image: xcode6.4
compiler: clang
env: COMPILER='ccache clang++' BUILD_TYPE='Release'
- os: osx
osx_image: xcode7
compiler: clang
env: COMPILER='ccache clang++' BUILD_TYPE='Debug'
- os: osx
osx_image: xcode7
compiler: clang
env: COMPILER='ccache clang++' BUILD_TYPE='Release'
- mkdir -p ${DEPS_DIR} && cd ${DEPS_DIR}
- |
if [[ "${TRAVIS_OS_NAME}" == "linux" ]]; then
mkdir cmake && travis_retry wget --quiet -O - ${CMAKE_URL} | tar --strip-components=1 -xz -C cmake
export PATH=${DEPS_DIR}/cmake/bin:${PATH}
elif [[ "${TRAVIS_OS_NAME}" == "osx" ]]; then
brew install cmake ccache
- export CXX=${COMPILER}
- cmake -Hprojects/CMake -BBuild -DCMAKE_BUILD_TYPE=${BUILD_TYPE}
- cd Build
- make -j 2
- ctest -V -j 2
Boost Software License - Version 1.0 - August 17th, 2003
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:
The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.
![catch logo](catch-logo-small.png)
Build status (on Travis CI) [![Build Status](](
<a href="">The latest, single header, version can be downloaded directly using this link</a>
## What's the Catch?
Catch stands for C++ Automated Test Cases in Headers and is a multi-paradigm automated test framework for C++ and Objective-C (and, maybe, C). It is implemented entirely in a set of header files, but is packaged up as a single header for extra convenience.
## How to use it
This documentation comprises these three parts:
* [Why do we need yet another C++ Test Framework?](docs/
* [Tutorial](docs/ - getting started
* [Reference section](docs/ - all the details
## More
* Issues and bugs can be raised on the [Issue tracker on GitHub](
* For discussion or questions please use [the dedicated Google Groups forum](!forum/catch-forum)
These are the currently documented areas of the framework. There is more to come.
Before looking at this material be sure to read the [tutorial](
* [Assertion macros](
* [Logging macros](
* [Test cases and sections](
* [Test fixtures](
* [Command line](
* [Build systems](
* [Supplying your own main()](
* [Configuration](
* [String Conversions](
* [Why are my tests slow to compile?](
* [Why Catch?](
* [Contributing](
# Assertion Macros
Most test frameworks have a large collection of assertion macros to capture all possible conditional forms (```_EQUALS```, ```_NOTEQUALS```, ```_GREATER_THAN``` etc).
Catch is different. Because it decomposes natural C-style conditional expressions most of these forms are reduced to one or two that you will use all the time. That said there are a rich set of auxilliary macros as well. We'll describe all of these here.
Most of these macros come in two forms:
## Natural Expressions
The ```REQUIRE``` family of macros tests an expression and aborts the test case if it fails.
The ```CHECK``` family are equivalent but execution continues in the same test case even if the assertion fails. This is useful if you have a series of essentially orthogonal assertions and it is useful to see all the results rather than stopping at the first failure.
* **REQUIRE(** _expression_ **)** and
* **CHECK(** _expression_ **)**
Evaluates the expression and records the result. If an exception is thrown it is caught, reported, and counted as a failure. These are the macros you will use most of the time
CHECK( str == "string value" );
CHECK( thisReturnsTrue() );
REQUIRE( i == 42 );
* **REQUIRE_FALSE(** _expression_ **)** and
* **CHECK_FALSE(** _expression_ **)**
Evaluates the expression and records the _logical NOT_ of the result. If an exception is thrown it is caught, reported, and counted as a failure.
(these forms exist as a workaround for the fact that ! prefixed expressions cannot be decomposed).
REQUIRE_FALSE( thisReturnsFalse() );
### Floating point comparisons
When comparing floating point numbers - especially if at least one of them has been computed - great care must be taken to allow for rounding errors and inexact representations.
Catch provides a way to perform tolerant comparisons of floating point values through use of a wrapper class called ```Approx```. ```Approx``` can be used on either side of a comparison expression. It overloads the comparisons operators to take a tolerance into account. Here's a simple example:
REQUIRE( performComputation() == Approx( 2.1 ) );
By default a small epsilon value is used that covers many simple cases of rounding errors. When this is insufficent the epsilon value (the amount within which a difference either way is ignored) can be specified by calling the ```epsilon()``` method on the ```Approx``` instance. e.g.:
REQUIRE( 22/7 == Approx( 3.141 ).epsilon( 0.01 ) );
When dealing with very large or very small numbers it can be useful to specify a scale, which can be achieved by calling the ```scale()``` method on the ```Approx``` instance.
## Exceptions
* **REQUIRE_THROWS(** _expression_ **)** and
* **CHECK_THROWS(** _expression_ **)**
Expects that an exception (of any type) is be thrown during evaluation of the expression.
* **REQUIRE_THROWS_AS(** _expression_, _exception type_ **)** and
* **CHECK_THROWS_AS(** _expression_, _exception type_ **)**
Expects that an exception of the _specified type_ is thrown during evaluation of the expression.
* **REQUIRE_NOTHROW(** _expression_ **)** and
* **CHECK_NOTHROW(** _expression_ **)**
Expects that no exception is thrown during evaluation of the expression.
## Matcher expressions
To support Matchers a slightly different form is used. Matchers will be more fully documented elsewhere. *Note that Matchers are still at early stage development and are subject to change.*
* **REQUIRE_THAT(** _lhs_, _matcher call_ **)** and
* **CHECK_THAT(** _lhs_, _matcher call_ **)**
\ No newline at end of file
# Integration with build systems
Build Systems may refer to low-level tools, like CMake, or larger systems that run on servers, like Jenkins or TeamCity. This page will talk about both.
# Continuous Integration systems
Probably the most important aspect to using Catch with a build server is the use of different reporters. Catch comes bundled with three reporters that should cover the majority of build servers out there - although adding more for better integration with some is always a possibility (as has been done with TeamCity).
Two of these reporters are built in (XML and JUnit) and the third (TeamCity) is included as a separate header. It's possible that the other two may be split out in the future too - as that would make the core of Catch smaller for those that don't need them.
## XML Reporter
```-r xml```
The XML Reporter writes in an XML format that is specific to Catch.
The advantage of this format is that it corresponds well to the way Catch works (especially the more unusual features, such as nested sections) and is a fully streaming format - that is it writes output as it goes, without having to store up all its results before it can start writing.
The disadvantage is that, being specific to Catch, no existing build servers understand the format natively. It can be used as input to an XSLT transformation that could covert it to, say, HTML - although this loses the streaming advantage, of course.
## JUnit Reporter
```-r junit```
The JUnit Reporter writes in an XML format that mimics the JUnit ANT schema.
The advantage of this format is that the JUnit Ant schema is widely understood by most build servers and so can usually be consumed with no additional work.
The disadvantage is that this schema was designed to correspond to how JUnit works - and there is a significant mismatch with how Catch works. Additionally the format is not streamable (because opening elements hold counts of failed and passing tests as attributes) - so the whole test run must complete before it can be written.
## TeamCity Reporter
```-r teamcity```
The TeamCity Reporter writes TeamCity service messages to stdout. In order to be able to use this reporter an additional header must also be included.
```catch_reporter_teamcity.hpp``` can be found in the ```include\reporters``` directory. It should be included in the same file that ```#define```s ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER```. The ```#include``` should be placed after ```#include```ing Catch itself.
#include "catch.hpp"
#include "catch_reporter_teamcity.hpp"
Being specific to TeamCity this is the best reporter to use with it - but it is completely unsuitable for any other purpose. It is a streaming format (it writes as it goes) - although test results don't appear in the TeamCity interface until the completion of a suite (usually the whole test run).
# Low-level tools
## CMake
You can use the following CMake script to automatically fetch Catch from github and configure it as an external project:
cmake_minimum_required(VERSION 2.8.8)
project(catch_builder CXX)
find_package(Git REQUIRED)
# Expose required variable (CATCH_INCLUDE_DIR) to parent scope
ExternalProject_Get_Property(catch source_dir)
set(CATCH_INCLUDE_DIR ${source_dir}/include CACHE INTERNAL "Path to include folder for Catch")
If you put it in, e.g., `${PROJECT_SRC_DIR}/${EXT_PROJECTS_DIR}/catch/`, you can use it in your project by adding the following to your root CMake file:
# Includes Catch in the project:
include_directories(${CATCH_INCLUDE_DIR} ${COMMON_INCLUDES})
enable_testing(true) # Enables unit-testing.
\ No newline at end of file
Catch works quite nicely without any command line options at all - but for those times when you want greater control the following options are available.
Click one of the followings links to take you straight to that option - or scroll on to browse the available options.
<a href="#specifying-which-tests-to-run"> ` <test-spec> ...`</a><br />
<a href="#usage"> ` -h, -?, --help`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -l, --list-tests`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -t, --list-tags`</a><br />
<a href="#showing-results-for-successful-tests"> ` -s, --success`</a><br />
<a href="#breaking-into-the-debugger"> ` -b, --break`</a><br />
<a href="#eliding-assertions-expected-to-throw"> ` -e, --nothrow`</a><br />
<a href="#invisibles"> ` -i, --invisibles`</a><br />
<a href="#sending-output-to-a-file"> ` -o, --out`</a><br />
<a href="#choosing-a-reporter-to-use"> ` -r, --reporter`</a><br />
<a href="#naming-a-test-run"> ` -n, --name`</a><br />
<a href="#aborting-after-a-certain-number-of-failures"> ` -a, --abort`</a><br />
<a href="#aborting-after-a-certain-number-of-failures"> ` -x, --abortx`</a><br />
<a href="#warnings"> ` -w, --warn`</a><br />
<a href="#reporting-timings"> ` -d, --durations`</a><br />
<a href="#input-file"> ` -f, --input-file`</a><br />
<a href="#list-test-names-only"> ` --list-test-names-only`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-reporters`</a><br />
<a href="#order"> ` --order`</a><br />
<a href="#rng-seed"> ` --rng-seed`</a><br />
<a id="specifying-which-tests-to-run"></a>
## Specifying which tests to run
<pre>&lt;test-spec> ...</pre>
Test cases, wildcarded test cases, tags and tag expressions are all passed directly as arguments. Tags are distinguished by being enclosed in square brackets.
If no test specs are supplied then all test cases, except "hidden" tests, are run.
A test is hidden by giving it any tag starting with (or just) a period (```.```) - or, in the deprecated case, tagged ```[hide]``` or given name starting with `'./'`. To specify hidden tests from the command line ```[.]``` or ```[hide]``` can be used *regardless of how they were declared*.
Specs must be enclosed in quotes if they contain spaces. If they do not contain spaces the quotes are optional.
Wildcards consist of the `*` character at the beginning and/or end of test case names and can substitute for any number of any characters (including none).
Test specs are case insensitive.
If a spec is prefixed with `exclude:` or the `~` character then the pattern matches an exclusion. This means that tests matching the pattern are excluded from the set - even if a prior inclusion spec included them. Subsequent inclusion specs will take precendence, however.
Inclusions and exclusions are evaluated in left-to-right order.
Test case examples:
<pre>thisTestOnly Matches the test case called, 'thisTestOnly'
"this test only" Matches the test case called, 'this test only'
these* Matches all cases starting with 'these'
exclude:notThis Matches all tests except, 'notThis'
~notThis Matches all tests except, 'notThis'
~*private* Matches all tests except those that contain 'private'
a* ~ab* abc Matches all tests that start with 'a', except those that
start with 'ab', except 'abc', which is included
Names within square brackets are interpreted as tags.
A series of tags form an AND expression wheras a comma-separated sequence forms an OR expression. e.g.:
This matches all tests tagged `[one]` and `[two]`, as well as all tests tagged `[three]`
<a id="choosing-a-reporter-to-use"></a>
## Choosing a reporter to use
<pre>-r, --reporter &lt;reporter></pre>
A reporter is an object that formats and structures the output of running tests, and potentially summarises the results. By default a console reporter is used that writes, IDE friendly, textual output. Catch comes bundled with some alternative reporters, but more can be added in client code.<br />
The bundled reporters are:
<pre>-r console
-r compact
-r xml
-r junit
The JUnit reporter is an xml format that follows the structure of the JUnit XML Report ANT task, as consumed by a number of third-party tools, including Continuous Integration servers such as Hudson. If not otherwise needed, the standard XML reporter is preferred as this is a streaming reporter, whereas the Junit reporter needs to hold all its results until the end so it can write the overall results into attributes of the root node.
<a id="breaking-into-the-debugger"></a>
## Breaking into the debugger
<pre>-b, --break</pre>
In some IDEs (currently XCode and Visual Studio) it is possible for Catch to break into the debugger on a test failure. This can be very helpful during debug sessions - especially when there is more than one path through a particular test.
In addition to the command line option, ensure you have built your code with the DEBUG preprocessor symbol
<a id="showing-results-for-successful-tests"></a>
## Showing results for successful tests
<pre>-s, --success</pre>
Usually you only want to see reporting for failed tests. Sometimes it's useful to see *all* the output (especially when you don't trust that that test you just added worked first time!).
To see successful, as well as failing, test results just pass this option. Note that each reporter may treat this option differently. The Junit reporter, for example, logs all results regardless.
<a id="aborting-after-a-certain-number-of-failures"></a>
## Aborting after a certain number of failures
<pre>-a, --abort
-x, --abortx [&lt;failure threshold>]
If a ```REQUIRE``` assertion fails the test case aborts, but subsequent test cases are still run.
If a ```CHECK``` assertion fails even the current test case is not aborted.
Sometimes this results in a flood of failure messages and you'd rather just see the first few. Specifying ```-a``` or ```--abort``` on its own will abort the whole test run on the first failed assertion of any kind. Use ```-x``` or ```--abortx``` followed by a number to abort after that number of assertion failures.
<a id="listing-available-tests-tags-or-reporters"></a>
## Listing available tests, tags or reporters
<pre>-l, --list-tests
-t, --list-tags
```-l``` or ```--list-tests`` will list all registered tests, along with any tags.
If one or more test-specs have been supplied too then only the matching tests will be listed.
```-t``` or ```--list-tags``` lists all available tags, along with the number of test cases they match. Again, supplying test specs limits the tags that match.
```--list-reporters``` lists the available reporters.
<a id="sending-output-to-a-file"></a>
## Sending output to a file
<pre>-o, --out &lt;filename>
Use this option to send all output to a file. By default output is sent to stdout (note that uses of stdout and stderr *from within test cases* are redirected and included in the report - so even stderr will effectively end up on stdout).
<a id="naming-a-test-run"></a>
## Naming a test run
<pre>-n, --name &lt;name for test run></pre>
If a name is supplied it will be used by the reporter to provide an overall name for the test run. This can be useful if you are sending to a file, for example, and need to distinguish different test runs - either from different Catch executables or runs of the same executable with different options. If not supplied the name is defaulted to the name of the executable.
<a id="eliding-assertions-expected-to-throw"></a>
## Eliding assertions expected to throw
<pre>-e, --nothrow</pre>
Skips all assertions that test that an exception is thrown, e.g. ```REQUIRE_THROWS```.
These can be a nuisance in certain debugging environments that may break when exceptions are thrown (while this is usually optional for handled exceptions, it can be useful to have enabled if you are trying to track down something unexpected).
Sometimes exceptions are expected outside of one of the assertions that tests for them (perhaps thrown and caught within the code-under-test). The whole test case can be skipped when using ```-e``` by marking it with the ```[!throws]``` tag.
When running with this option any throw checking assertions are skipped so as not to contribute additional noise. Be careful if this affects the behaviour of subsequent tests.
<a id="invisibles"></a>
## Make whitespace visible
<pre>-i, --invisibles</pre>
If a string comparison fails due to differences in whitespace - especially leading or trailing whitespace - it can be hard to see what's going on.
This option transforms tabs and newline characters into ```\t``` and ```\n``` respectively when printing.
<a id="warnings"></a>
## Warnings
<pre>-w, --warn &lt;warning name></pre>
Enables reporting of warnings (only one, at time of this writing). If a warning is issued it fails the test.
The ony available warning, presently, is ```NoAssertions```. This warning fails a test case, or (leaf) section if no assertions (```REQUIRE```/ ```CHECK``` etc) are encountered.
<a id="reporting-timings"></a>
## Reporting timings
<pre>-d, --durations &lt;yes/no></pre>
When set to ```yes``` Catch will report the duration of each test case, in milliseconds. Note that it does this regardless of whether a test case passes or fails. Note, also, the certain reporters (e.g. Junit) always report test case durations regardless of this option being set or not.
<a id="input-file"></a>
## Load test names to run from a file
<pre>-f, --input-file &lt;filename></pre>
Provide the name of a file that contains a list of test case names - one per line. Blank lines are skipped and anything after the comment character, ```#```, is ignored.
A useful way to generate an initial instance of this file is to use the <a href="#list-test-names-only">list-test-names-only</a> option. This can then be manually curated to specify a specific subset of tests - or in a specific order.
<a id="list-test-names-only"></a>
## Just test names
This option lists all available tests in a non-indented form, one on each line. This makes it ideal for saving to a file and feeding back into the <a href="#input-file">```-f``` or ```--input-file```</a> option.
<a id="order"></a>
## Specify the order test cases are run
<pre>--order &lt;decl|lex|rand&gt;</pre>
Test cases are ordered one of three ways:
### decl
Declaration order. The order the tests were originally declared in. Note that ordering between files is not guaranteed and is implementation dependent.