Improve the documentation and add cli again

* Write documentation and restructure it to have a consistent
documentation with examples and just a brief readme.
* Make the comandline utility work again
* Remove the not working setup alias `docs`

CONTRIBUTING.rst

@@ -50,10 +50,18 @@ So please write your docstrings following `Google's docstring convention`_.
 
 Since Sphinx does not handle Python 2's type annotations (`PEP 484 <https://www.python.org/dev/peps/pep-0484/#suggested-syntax-for-python-2-7-and-straddling-code>`_) well, it is still necessary to write the type in the docstring.
 
+If you create a new module you need to include it in the `index.rst` at least so the automatically generated documentation is also included in the documentation.
+
 Write Tests
 ~~~~~~~~~~~
 
-Try to increase the test coverage by writing tests.
+Try to increase the test coverage and code quality by writing tests.
+I'm sorry for that, but please write your tests in a way that they are still Python 2.7 compatible.
+Though there are many doctests_  and some unittests_ fragments, I switched to pytest_.
+
+.. _`doctests`: https://docs.python.org/3.6/library/doctest.html
+.. _unittests: https://docs.python.org/3.6/library/unittest.html
+.. _pytest: https://docs.pytest.org/en/latest/
 
 Submit Feedback
 ~~~~~~~~~~~~~~~
@@ -93,13 +101,14 @@ Ready to contribute? Here's how to set up `cassandra` for local development.
 
 5. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox::
 
-    $ flake8 cassandra tests
     $ python setup.py test
     $ tox
 
-   To get flake8 and tox, just pip install them into your virtualenv.
+flake8 and tox, etc. are all installed by installing the `test-requirements.txt`
+
+Change things marked by the `pre-commit`_-hooks if there are some.
 
-   Change things marked by the pre-commit-hooks if there are some.
+.. _`pre-commit`: http://pre-commit.com/
 
 6. Check for upstream updates::
 

README.rst

@@ -2,73 +2,28 @@
 Cassandra
 =========
 
-Cassandra_ — ANKAs aktuelles (Stand 2016) Archivierungsdatenbanksystem. Es existiert eine JSON-API, die aus dem ANKA-Netz (las-bernhard.anka.kit.edu) verfügbar ist.
-
-EPICS — ANKAs neuestes Kontrollsystem (Stand 2016), in welchem u. a. der CLIC-Dämpfungswiggler integriert ist. Aus diesem wird Cassandra befüllt. Für weitere Informationen siehe z. B. Arbeitsgruppenvortrag von Walter Werner, ANKA-Confluence, sowie die `offizielle EPICS Seite`_
-
-PV — EPICS-Variable, die hier abgefragt werden kann.
-
-.. _Cassandra: https://cassandra.apache.org/
-.. _`offizielle EPICS Seite`: https://www.aps.anl.gov/epics/
-
-Prerequists
-------------
-* `gcc`_
-* opensslv.h
-
-.. _`gcc`: https://gcc.gnu.org
-
-Installation
-------------
-
-::
-
-    python -m pip install "git+ssh://git@git.scc.kit.edu:/las-software/15-3-DataProcessing/cassandra.git#egg=cassandra"
-
-
-Script
--------
-Das Script `./fetch.py` läd Daten eines PV und plottet sie
-
-./scw02_clic.pv — List of all CLIC-dw-PVs saved in Cassandra
-
-cassandra.py Python-Klasse zum abstrahieren von Cassandra (basiert auf dem fetch.py-Skript)
-
-Unter `bin` sind ein beispielhaftes Skript zur Verwendung der Cassandra-Klasse (`plot_two.py`).
-Auch wird das Kommandozeilen-Programm zum anzeigen von gemittelten Werten aus der Cassandra-Datenbank:`get_mean_values.py` installiert.
-
-JSON-Dateien
-------------
-./\*json sind aus Cassandra abgefragte Dateien im JSON-Format `JavaScript-Object-Notation`_
-The URI scheme is described in the `ANKA-Confluence`_.
-
-.. _`JavaScript-Object-Notation`: https://de.wikipedia.org/wiki/JSON
-.. _`ANKA-Confluence`: https://ankawiki.anka.kit.edu/display/MACHINE/Cassandra+Web+Readout
-
 Documentation
 -------------
-If you want to read the docu::
+If you want to read the docu, you may find a PDF at `docs/_build/latex/cassandra.pdf` or a HTML documentation at
+`docs/_build/html/index.html`. In doubt the HTML version is newer/more accurate.
+In principle the docu should also be available as the `pages` artifact of the `GitLab-CI-Pipeline <https://git.scc.kit.edu/las-software/15-3-DataProcessing/cassandra/pipelines>`_.
 
-    $ okular docs/_build/latex/cassandra.pdf
+If you want or need to create the docu on your own you can find some general note about writing docu.
+The docu is generated using `Sphinx <https://sphinx-doc.org>`_, is written in `ReStructuredText <https://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html>`_ and not in Markdown (like e. g. many other README files).
 
-or::
-
-    $ firefox docs/_build/html/index.html
-
-If you want to create your own docu you here are some general note about writing docu.
-The docu is generated using `Sphinx <https://sphinx-doc.org>`_, is written in `ReStructuredText <https://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html>`_ and not in Markdown (like e.g. many other README files).
-
-You can compile the docu either by using the setup.py::
+You can compile the docu either by using the `setup.py` if you have got the necessary dependencies installed::
 
+    $ python -m pip install -r requirements.txt
     $ python setup.py build_sphinx
 
-In detail this dose something similar to the following steps.
+Or doing the steps by your-self.
 Run the sphinx-apidoc::
 
     $ cd docs && sphinx-apidoc -f -o ./_source ../src/cassandra
 
-Include the filenames (ls `*.rst >> index.rst`) into `index.rst` at a appropriate place and add three spaces before them (e.\,g. care about the indention!)
-and finally by run::
+..  Include the filenames (ls `*.rst >> index.rst`) into `index.rst` at a appropriate place and add three spaces before them (e.\,g. care about the indention!)
+
+And finally by running::
 
     $ make latexpdf
     $ okular _build/latex/cassandra.pdf
@@ -78,19 +33,19 @@ or::
     $ make html
     $ firefox _build/html/index.html
 
-To setup your own documentation if you're not using PyScaffold you need to
-install Sphinx and napoleon::
+.. To setup your own documentation if you're not using PyScaffold you need to
+   install Sphinx and napoleon::
 
-    $ pip install Sphinx
-    $ pip install sphinxcontrib-napoleon
+       $ pip install Sphinx
+       $ pip install sphinxcontrib-napoleon
 
-Create the docs directory and initialize the project::
+   Create the docs directory and initialize the project::
 
-    $ mkdir docs
-    $ cd docs
-    $ sphinx-quickstart (project path: ../src/cassandra, source-build seperated)
+       $ mkdir docs
+       $ cd docs
+       $ sphinx-quickstart (project path: ../src/cassandra, source-build seperated)
 
-Configure its configuration by editing `source/conf.py` and adding `shpinx.ext.napoleon` to the list of extensions and
- uncommenting `import sys` and `import os` and modifying `abspath('../..')`.
+   Configure its configuration by editing `source/conf.py` and adding `shpinx.ext.napoleon` to the list of extensions and
+    uncommenting `import sys` and `import os` and modifying `abspath('../..')`.
 
-Create some required directories `mkdir _source _static`.
+   Create some required directories `mkdir _source _static`.

bin/plot_two.py

 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
-
-"""
-   :Authors: Julian Gethmann
-   :Contact: phd@gethmann.org
-   :Date: 2016-03-07
-   :Version: 0.1
-"""
-
-from cassandra import Pvs, Cassandra
 import matplotlib.pyplot as plt
+from cassandra import Cassandra, Pvs
 
 
 def main():
     # configuration
-    start_time = "2016/01/27 09:00:00"
-    end_time = "2016/02/12 18:00:00"
-    start_time = "2016/02/03 09:00:00"
-    end_time = "2016/02/03 19:00:00"
     start_time = "2016/04/07 13:30:00"
     end_time = "2016/04/07 14:00:00"
     count = 1000
 
     pvs = (
-        # Pvs.pv["sul_field"],
-        # Pvs.pv["energy"],
-        # Pvs.pv["clic_field"],
-        # Pvs.pv["nu_y"],
-        # Pvs.pv["nu_x"],
         Pvs.pv["q1"],
         Pvs.pv["q2"],
         Pvs.pv["q3"],
@@ -37,7 +20,7 @@ def main():
         Pvs.pv["sh"],
     )
 
-    # data aquisition
+    # data acquisition
     cassandra_objects = []
     for pv in pvs:
         obj = Cassandra(start_time, end_time, pv, count, directory="/tmp/")
@@ -56,9 +39,8 @@ def main():
         plt.plot(timestamp, value)
 
     plt.legend(pvs)
-    plt.show(block=True)
+    plt.show()
 
 
 if __name__ == "__main__":
     main()
-# vim: tabstop=8 expandtab shiftwidth=4 softtabstop=4

docs/background.rst

+=========================================
+Background information and general remark
+=========================================
+
+Cassandra_ is the current (2018) archiving solution used at KARA. There is a `JavaScript-Object-Notation`_-API available, that can be accessed from within the IBPT-LAN (CN, VPN or las-bernhard.anka.kit.edu).
+
+Terminology
+-----------
+
+EPICS — KARA's currently used control system, in which e. g. CLIC-damping wiggler is integrated into. This system feeds its values to the Cassandra-db. More information on EPICS can be found in the working group presentation done by Walter Werner at CS or in the ANKA-Confluence, as well as the `official EPICS website`_
+
+PV — EPICS-Variable, (Process variable) are  the variables you can ask EPICS or Cassandra-db for. They look something like `A:SR:S1:What:Not:01`. A list of oftenly used ones is also included in the :class:`Pvs`.
+
+CSS — The Control System Studio is the graphical user interface used for the EPICS control system. It can export data which can be imported using function of the :mod:`css_read` module.
+
+.. _Cassandra: https://cassandra.apache.org/
+.. _`official EPICS website`: https://www.aps.anl.gov/epics/
+
+
+JSON-Files
+-----------
+./\*json are files that are requested via the webfront end of the Cassandra-db. They are formated in the `JavaScript-Object-Notation`_
+The URI scheme is described in the `ANKA-Confluence`_.
+
+.. _`JavaScript-Object-Notation`: https://de.wikipedia.org/wiki/JSON
+.. _`ANKA-Confluence`: https://ankawiki.anka.kit.edu/display/MACHINE/Cassandra+Web+Readout

docs/examples.rst

@@ -6,14 +6,24 @@ Examples
 
 Here are some examples how you can use the Classes as well as how you can use the script.
 
-If I write Cassandra I am speaking of this Python class, if I write cassanda I am referencing to the package and if I am writing Cassandra-DB I am speaking of the `Apache Cassandra <https://cassandra.apache.org/>`_ instance at KARA.
+Beside the examples given here, *please* also have look at the examples given in the Module reference.
+
+.. toctree::
+   :maxdepth: 2
+
+   Module reference <api/cassandra>
+
 
 Scripts
 -------
 
+At the moment there is just the `get_mean_values` script that you can call with the `-h` or `--help` option flag to get further information how to use it. Please feel free to provide an example.
+
 Classes
 -------
 
+Context manager (with Cassandra(...))
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Let's say we want to have the time a specific fill started, then we can ask the Cassandra-DB to provide us the fill numbers for the time range of the beginning of the data logging (let's say 1st Jan, 2005) till today.
 
 This could be done with the following function that uses the Cassanda's context manager (with statement) to also care about exceptions.
@@ -78,3 +88,10 @@ We end up with a code that might look like:
 
 
 The `data` object we get from the handler is a :obj:`tuple` of two :obj:`list` s that contain the date stamps as :obj:`datetime.datetime` returned from Cassandra and the corresponding data. In our case the latter are the fill numbers.
+
+Cassandra objects
+~~~~~~~~~~~~~~~~~
+
+One example for a script that fetches the data for the main magnets and plots them can be written like this.
+
+.. literalinclude:: ../bin/plot_two.py

docs/index.rst

 .. include:: ../README.rst
 
+A list of all PVs that are saved in the Cassandra-db that are relevant for the CLIC damping wiggler prototype can be found in the file `./scw02_clic.pv`.
+
 
 Contents
 ========
 
+If I write Cassandra I am speaking of this Python class, if I write cassanda I am referencing to the package and if I am writing Cassandra-DB I am speaking of the `Apache Cassandra <https://cassandra.apache.org/>`_ instance at KARA.
+
 .. toctree::
    :maxdepth: 2
 
+   Background information <background>
+   Installation <install>
    Examples <examples>
+   (Command line) Scripts <scripts>
    Module Reference <api/cassandra>
    Changelog <changelog>
    License <license>

docs/install.rst

+Installation
+============
+
+This package is compatible with the currently supported Python's, namely Python 2.7, 3.4-3.7.
+Because Python 2.7 is going to die very soon and I do not depend on it any longer, there might be more errors with that version which I don't know, yet.
+Because this package uses TLS you need to have a compiler and the openssl library.
+
+Prerequists
+------------
+For example on Fedora you need to have the following dependencies fullfilled
+
+* `gcc`_
+* opensslv.h
+
+.. _`gcc`: https://gcc.gnu.org
+
+Installation
+------------
+
+The installation itself can be done using Python's pip, that ships with Python >= 3.4 and is also available and common on Python 2.7.
+Because this library is not on PyPI, you cannot find it via pip's search, but have to install it from SCC's GitLab where you have to be able to use SSH.
+
+::
+
+    python -m pip install "git+ssh://git@git.scc.kit.edu:/las-software/15-3-DataProcessing/cassandra"
+
+
+Installing this package this way also installs all dependencies.
+
+requirements.txt or Pipfile
+---------------------------
+
+If you are using this package as a library in your project you probably want to include it into your `requirements.txt` or your `Pipfile`, but it cannot just be named, because is not on PyPI.
+
+The solution for `requirements.txt` or `Pipfile` is to edit them by hand and change your `cassandra` from the ``pip freeze`` to
+
+::
+
+    git+ssh://git@git.scc.kit.edu:/las-software/15-3-DataProcessing/cassandra.git@0.7.1#egg=cassandra
+
+without the quotation marks!
+Because you want to stick to a version, you have to add this `@0.7.1#egg=cassandra` where `0.7.1` is the actual version or git commit hash.

docs/scripts.rst

+Scripts
+=======
+
+Get mean values
+---------------
+
+This package ships with a comand line program that is called `get_mean_values` and that can print you the mean values for a given PV and time range.
+
+.. warning::
+    At the time of writing this docu there is a bug in the script. It calculates the mean values of the returned values, which is not the actual mean, because the values are not equaly spaced in time. See issue #13 in the issue tracker for more info on this.
+
+
+Legacy script with proxy functionality
+--------------------------------------
+
+Furthermore there is a script located at `src/cassandra/fetch.py` which downloads data for a PV and plots it.
+This package is based on that script, though it improved massively.
+
+The reason why it is still there is that one can use it if one just has got an account on `las-bernhard.anka.kit.edu`, but no access to the IBPT-LAN. In that case it downloads the data to this computer and then to your computer.
+To use this script you need to modify the `main()` function and adjust the `start_time`, `end_time` and `pvi` variable. For further information read the docstring and search for `Edit this variables` in the script.
+
+Anyhow I would suggest to ask for IBPT-LAN access.

setup.cfg

@@ -54,7 +54,6 @@ norecursedirs =
 
 [aliases]
 release = sdist bdist_wheel upload
-docs = build_sphinx
 
 [bdist_wheel]
 # Use this option if your package is pure-python

setup.py

@@ -15,9 +15,7 @@ from setuptools import setup
 # Add here console scripts and other entry points in ini-style format
 entry_points = """
 [console_scripts]
-# script_name = cassandra.module:function
-# For example:
-# fibonacci = cassandra.skeleton:run
+get_mean_values = cassandra.tools:get_mean_values
 """
 if sys.version_info <= (3,):
     requires = ['pyscaffold==2.5.10']