Commit fcc910d8 authored by julian.gethmann's avatar julian.gethmann

Update documentation

parent abf84efe
......@@ -4,7 +4,7 @@ Cassandra
Cassandra_ — ANKAs aktuelles (Stand 2016) Archivierungsdatenbanksystem. Es existiert eine JSON-API, die aus dem ANKA-Netz (las-bernhard.anka.kit.edu) verfügbar ist.
EPICS — ANKAs neuestes Kontrollsystem (Stand 2016), in welchem u. a. der CLIC-Dämpfungswiggler integriert ist. Aus diesem wird Cassandra befüllt. Für weitere Informationen siehe z. B. Arbeitsgruppenvortrag von Walter Werner, ANKA-Confluence, sowie die `offizielle EPICS Seite`_
EPICS — ANKAs neuestes Kontrollsystem (Stand 2016), in welchem u. a. der CLIC-Dämpfungswiggler integriert ist. Aus diesem wird Cassandra befüllt. Für weitere Informationen siehe z. B. Arbeitsgruppenvortrag von Walter Werner, ANKA-Confluence, sowie die `offizielle EPICS Seite`_
PV — EPICS-Variable, die hier abgefragt werden kann.
......@@ -23,7 +23,7 @@ Installation
::
python -m pip install "git+ssh://git@git.scc.kit.edu:/las/py/cassandra.git#egg=cassandra"
python -m pip install "git+ssh://git@git.scc.kit.edu:/las-software/15-3-DataProcessing/cassandra.git#egg=cassandra"
Script
......@@ -34,7 +34,7 @@ Das Script `./fetch.py` läd Daten eines PV und plottet sie
cassandra.py Python-Klasse zum abstrahieren von Cassandra (basiert auf dem fetch.py-Skript)
Unter `bin` sind ein beispielhaftes Skript zur Verwendung der Cassandra-Klasse (`plot_two.py`).
Unter `bin` sind ein beispielhaftes Skript zur Verwendung der Cassandra-Klasse (`plot_two.py`).
Auch wird das Kommandozeilen-Programm zum anzeigen von gemittelten Werten aus der Cassandra-Datenbank:`get_mean_values.py` installiert.
JSON-Dateien
......@@ -63,11 +63,12 @@ You can compile the docu either by using the setup.py::
$ python setup.py docs
or by hand by running::
$ cd docs && sphinx-apidoc -f -o ./_source ../cassandra
including filenames (ls `*.rst >> index.rst`) into `index.rst` at a apropiate place and add three spaces before them (e.\,g. care about the indention!)
including filenames (ls `*.rst >> index.rst`) into `index.rst` at a appropriate place and add three spaces before them (e.\,g. care about the indention!)
and finally by running::
$ make latexpdf
$ okular _build/latex/cassandra.pdf
......@@ -78,6 +79,7 @@ or::
To setup your own documentation if you're not using PyScaffold you need to
install Sphinx and napoleon::
$ pip install Sphinx
$ pip install sphinxcontrib-napoleon
......
.. _changes:
.. include:: ../CHANGELOG.rst
.. _examples:
========
Examples
========
Here are some examples how you can use the Classes as well as how you can use the script.
If I write Cassandra I am speaking of this Python class, if I write cassanda I am referencing to the package and if I am writing Cassandra-DB I am speaking of the `Apache Cassandra <https://cassandra.apache.org/>`_ instance at KARA.
Scripts
-------
Classes
-------
Let's say we want to have the time a specific fill started, then we can ask the Cassandra-DB to provide us the fill numbers for the time range of the beginning of the data logging (let's say 1st Jan, 2005) till today.
This could be done with the following function that uses the Cassanda's context manager (with statement) to also care about exceptions.
The context manager needs at least the start and end date and time for which it should ask for data and the process variable (PV) name.
So in our case this could something like :obj:`datetime.datetime(2005, 1, 1, 0, 0, 1)` as `start` and `datetime.datetime.now()` as `end`. The PV name is `'A:SR:OperationStatus:01:FillNumber'`.
Putting it together this gives us
.. code:: Python
from cassandra.cassandra import Cassandra
import datetime
with Cassandra(start=datetime.datetime(2005, 1, 1, 0, 0, 1),
end=datetime.datetime.now(),
pv="A:SR:OperationStatus:01:FillNumber") as data:
times, fills = data
Now the `times` list has got all the datetimes of the fill numbers that are in the `fills` list as integers.
Since we did not provide any number for the optional argument `count` the number of entries is not known and we need not to have one entry for each fill.
As it is not always convenient to provide :obj:`datetime.datetime` objects, you can also write the `start` and `end` date as strings in some common formats, like ISO-8601 or CSS-exports' format or the format used by the THz group.
For an explanation of the `count` argument see the documentation of Cassandra-DB's web-interface in the Confluence wiki.
The also optional argument `directory` does not make sense for the context manager, and is there for creating objects.
Now we can create another simple example that utilises the above mentioned string format, the count argument and uses the helper class :class:`cassandra.cassandra.Pvs`.
* The starting time, which we will provide as the string `"2005/01/01 00:00:01"`,
* the end time, which we may give as a :obj:`datetime.datetime` object,
* the PV name for which we can use the `fill` of the `Pvs.pv` dictionary, and
* in this case we want to limit the counts to `1`, because the frequency of the fills is very low.
We end up with a code that might look like:
.. code:: Python
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from cassandra import Cassandra, Pvs
import datetime
def fill_to_time(fill: int) -> datetime.datetime:
"""Return datetime of the KARA fill
Args:
fill: Fill number
Returns:
Fill time
"""
now_str = datetime.strftime(datetime.now(), "%Y/%m/%d %H:%M:%S")
with Cassandra("2005/01/01 00:00:01", now_str, Pvs.pv["fill"], 1) as data:
times, fills = data
return times[fills.index(fill)]
fill_to_time(6000)
The `data` object we get from the handler is a :obj:`tuple` of two :obj:`list` s that contain the date stamps as :obj:`datetime.datetime` returned from Cassandra and the corresponding data. In our case the latter are the fill numbers.
......@@ -7,6 +7,7 @@ Contents
.. toctree::
:maxdepth: 2
Examples <examples>
License <license>
Authors <authors>
Changelog <changelog>
......
......@@ -54,6 +54,7 @@ norecursedirs =
[aliases]
release = sdist bdist_wheel upload
docs = build_sphinx
[bdist_wheel]
# Use this option if your package is pure-python
......
......@@ -3,8 +3,10 @@
:Authors: Julian Gethmann
:Contact: atb@gethmann.org
:Date: 2017-09-15
.. versionadded:: 0.5.5
.. versionchanged:: 0.7.0
"""
from datetime import datetime
from typing import Iterable, List, Optional, Tuple # flake8: noqa
......@@ -18,11 +20,11 @@ def pvs2pd(
start, # type: datetime.datetime
end, # type: datetime.datetime
pv_names, # type: Iterable[str]
count=None, # type: Optional[int]
upsample=None, # type: Optional[str]
save_local=False, # type: Optional[bool]
): # type (...) -> pd.DataFrame
# Optional[int], Optional[str], Optional[bool]) -> pd.DataFrame
count=None, # type: int
upsample=None, # type: str
save_local=False, # type: bool
):
# type (...) -> pd.DataFrame
"""Return a `pd.DataFrame` with data for all `pv_names` and `time` as index
Missing data points are filled with the last value (like in CSS).
......@@ -71,22 +73,19 @@ def pvs2pd(
pv = Pvs.pv[pv_name] if ":" not in pv_name else pv_name
if save_local:
cas = Cassandra(start, end, pv=pv, count=None, directory=".")
data = cas.get_json_local(
) # type: Tuple[List[datetime.datetime], List[float]]
data = cas.get_json_local() # type: Tuple[List[datetime.datetime], List[float]]
collected_data = collected_data.join(
pd.DataFrame({
"time": data[0],
pv_name: data[1]
}).set_index("time"),
how="outer")
}).set_index("time"), how="outer")
else:
with Cassandra(start, end, pv=pv) as cas:
collected_data = collected_data.join(
pd.DataFrame({
"time": cas[0],
pv_name: cas[1]
}).set_index("time"),
how="outer")
}).set_index("time"), how="outer")
collected_data = collected_data.ffill()
collected_data = collected_data[collected_data.first_valid_index():]
# if start in collected_data.index:
......
......@@ -4,13 +4,12 @@
:Authors: Julian Gethmann
:Contact: phd@gethmann.org
.. lastupdated:: 2017-01-25
Return the mean and standard deviation for the PVs in the given time range.
"""
import argparse
import logging
from math import sqrt
from typing import Optional, Tuple, Union # flake8: noqa
from .cassandra import Cassandra, Pvs
......@@ -19,11 +18,12 @@ logging.getLogger('cassandra.tools').addHandler(logging.NullHandler())
def _get_mean_values(
start_time="2016/04/07 13:30:00",
end_time="2016/04/07 14:00:00",
pvs=("q1", "q2"),
output_format="full",
start_time="2016/04/07 13:30:00", # type: Optional[Union[datetime.datetime,str]]
end_time="2016/04/07 14:00:00", # type: Optional[Union[datetime.datetime,str]]
pvs=("q1", "q2"), # type: Tuple[str, str, ...]
output_format="full", # type: str
):
# type: (...) -> None
"""
.. versionchanged:: 0.3.0
"""
......@@ -43,37 +43,29 @@ def _get_mean_values(
"str": "{val}",
}
for pv in pvs:
with Cassandra(
start_time, end_time, Pvs.pv.get(pv, pv), count,
directory="/tmp/") as obj:
with Cassandra(start_time, end_time, Pvs.pv.get(pv, pv), count, directory="/tmp/") as obj:
if "str" in output_format:
_logger.info("Raw string for PV: {}".format(pv))
try:
print(obj[1][0])
except KeyError:
raise KeyError(
"No %s as output_format available!" % output_format)
raise KeyError("No %s as output_format available!" % output_format)
else:
_logger.info("Values for PV: {}".format(pv))
try:
mean = sum(obj[1]) / len(obj[1])
std = sqrt(
sum([(xi - mean)**2
for xi in obj[1]]) / (len(obj[1]) - 1))
print(output_str[output_format].format(
name=pv, val=mean, std=std))
std = sqrt(sum([(xi - mean)**2 for xi in obj[1]]) / (len(obj[1]) - 1))
print(output_str[output_format].format(name=pv, val=mean, std=std))
except ValueError:
raise ValueError(
"Probably too short time to get enough points")
raise ValueError("Probably too short time to get enough points")
except KeyError:
raise KeyError(
"No %s as output_format available!" % output_format)
raise KeyError("No %s as output_format available!" % output_format)
def get_mean_values():
parser = argparse.ArgumentParser(
description="Return the mean value "
"and standard deviation for the PVs in the given time range.")
# type: () -> None
parser = argparse.ArgumentParser(description="Return the mean value "
"and standard deviation for the PVs in the given time range.")
parser.add_argument(
"--start",
"-s",
......@@ -99,12 +91,7 @@ def get_mean_values():
required=False,
help="short name of the PV (see PVs class from cassandra module)")
parser.add_argument(
"--rawpv",
dest="rawpvs",
nargs="?",
type=str,
required=False,
help="full name of *one* PV")
"--rawpv", dest="rawpvs", nargs="?", type=str, required=False, help="full name of *one* PV")
parser.add_argument(
"--raw",
"-r",
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment