tests.rst 2.77 KB
Newer Older
BorjaEst's avatar
BorjaEst committed
1
Tests
BorjaEst's avatar
BorjaEst committed
2
3
==================================

4
5
6

Tests should run using tox_, an automation is used to simplify the 
testing process.
BorjaEst's avatar
BorjaEst committed
7
8
9
10
11
12
13
14
15
16

.. _tox: https://tox.readthedocs.io/en/latest/


To install it with pip use:

.. code-block:: bash
    
    $ pip install tox

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
Tests are divided into two types:

- Black-Box tests: Based on pytest_ framework, test the functionality 
  of the application without peering into its internal structures or 
  workings.
- White tests: Based on unittest_ framework, test the internal 
  structures of the application.

.. _pytest: https://docs.pytest.org/en/stable/
.. _unittest: https://tox.readthedocs.io/en/latest/

To run White and Black-Box tests use:

.. code-block:: bash

    $ tox tests o3skim/*.py
    ...
    py36: commands succeeded
    ...

This command generates a complete test report together with a 
coverage and pep8 report.


Black-Box tests:
----------------

The framework used is pytest_ to provide a simple syntax to 
test all possible combinations from the user point of view.

Pytest detects directly all tests following the test_discovery_
naming conventions. Therefore all Black-Box tests should be 
located on the **tests** folder at the package root and start 
with **test**. For example *test_sources.py*.

More than 500 test combinations are generated using which otherwise 
might not be feasible using other python test frameworks. 

.. _pytest: https://docs.pytest.org/en/stable/
.. _test_discovery: https://docs.pytest.org/en/reorganize-docs/new-docs/user/naming_conventions.html

To run only Black-Box tests simply call tox followed by the 
folder with the test location:
BorjaEst's avatar
BorjaEst committed
60

61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
.. code-block:: bash

    $ tox tests
    ...
    py36: commands succeeded
    ...


White tests:
------------

The framework used is unittest_, a simple and extended test framework
which ships by default together with python.

The usage is very simple and straight forward for simple test, but
the difficulty to parametrize and combine multiple test fixtures 
makes it not suitable for Black-Box testing without a very complex
customization.

To simplify code usage and testing, the white tests should be located
on the same file than the function / class are supposed to test.

To run only White tests simply call tox followed by the module files
you would like to test. You can also use the wildcard '*' to selected
and test all python modules:
BorjaEst's avatar
BorjaEst committed
86
87
88

.. code-block:: bash

89
    $ tox o3skim/*.py
BorjaEst's avatar
BorjaEst committed
90
    ...
BorjaEst's avatar
BorjaEst committed
91
    py36: commands succeeded
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
    ...


Coverage and Pep8 reports:
--------------------------

One of the benefits of tox test automation is the capability to 
generate code reports during testing.

The last coverage report output is produced at **htmlcov** which 
can be displayed in html format accessing to **index.html**.


The last Pep8 report produced by flake8 at the output file
**flake8.log**.
BorjaEst's avatar
BorjaEst committed
107