Commit 3f28b67e authored by BorjaEst's avatar BorjaEst
Browse files

First version ready for getting started

parent 8feb91b2
Build
===================
Download the repository at the **Build machine** using git.
Download the code from the o3skim_ repository at the **Build machine**.
For example, using git_:
.. code-block:: bash
......@@ -9,8 +11,13 @@ Download the repository at the **Build machine** using git.
Cloning into 'o3skim'...
...
.. _o3skim: https://git.scc.kit.edu/synergy.o3as/o3skim
.. _git: https://git-scm.com/
Build the container image at the **Build machine** using **docker**.
Build the container image at the **Build machine**.
For example, using docker_:
.. code-block:: bash
......@@ -19,8 +26,10 @@ Build the container image at the **Build machine** using **docker**.
Successfully built 69587025a70a
Successfully tagged o3skim:latest
.. _docker: https://docs.docker.com/engine/reference/commandline/build
If the build process succeded, you should see the image name on the container images list:
If the build process succeded, then you should see the image name on the container images list:
.. code-block:: bash
......@@ -29,3 +38,30 @@ If the build process succeded, you should see the image name on the container im
o3skim latest 69587025a70a xx seconds ago 557MB
...
To use your new generated image on the **Runtime machine**, the easiest way is to
push to a dockerhub repository. For example, with docker_:
.. code-block:: bash
$ docker push <repository>/o3skim:<tag>
The push refers to repository [docker.io/........./o3skim]
...
7e84795fccac: Preparing
7e84795fccac: Layer already exists
ffaeb20d9e23: Layer already exists
4cdd6a90e552: Layer already exists
3e0762bebc71: Layer already exists
1e441fe06d90: Layer already exists
98ff2784e9f5: Layer already exists
2b99e2403063: Layer already exists
d0f104dc0a1f: Layer already exists
...: digest: sha256:...................... size: 2004
If you do not have internet access from the **Build machine** or **Runtime machine**
it is also possible to use `docker save`_ to export your images.
.. _`docker save`: https://docs.docker.com/engine/reference/commandline/save/
Command Line Interface
=======================
Finally, run the container. Note the described `data`, `output` and
`sources.yaml` have to be provided. Also it is needed to specify the
Usage:
.. code-block:: bash
usage: main [-h] [-f SOURCES_FILE] [-s {year,decade}]
[-v {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
To run the the application using **udocker** at the **Runtime machine**
you need to provide the following volumes to the container:
- --volume, mount `/app/data`: Input path with data to skim.
- --volume, mount `/app/output`: Output path for skimmed results.
- --volume, mount `/app/sources.yaml`: Configuration file with a data structure
desctiption at the input path in YAML_ format.
See sources_example_ for a configuration example.
.. _YAML: https://yaml.org/
.. _sources_example: ../_static/sources_example.yaml
Also, in the specific case of udocker_, it is needed to specify that the
user `application` should run inside the container:
.. _udocker: https://indigo-dc.gitbook.io/udocker
For example,to run the container using udocker_ use the following:
.. code-block:: bash
$ udocker run \
......@@ -26,6 +50,15 @@ For the main function description and commands help you can call:
.. code-block:: bash
$ udocker run --user=application o3skim --help
...
As optional arguments, it is possible to indicate:
- -h, --help: show this help message and exit
- -f, --sources_file SOURCES_FILE: Custom sources YAML configuration. (default: ./sources.yaml)
- -s, --split_by {year,decade}: Period time to split output (default: None)
- -v, --verbosity {DEBUG,INFO,WARNING,ERROR,CRITICAL}: Sets the logging level (default: ERROR)
Note that SOURCES_FILE is only modifyed for development purposes as usually any
file from host can be mounted using the container directive '--volume'.
......@@ -2,36 +2,50 @@ Deployment
==================================
To deploy the the application using **udocker** at the **Runtime machine**
you need:
you need the o3skim container image.
- Input path with data to skim, to be mounter on `/app/data` inside the
container.
- Output path for skimmed results, to be mounted on `/app/output` inside
the container.
- Configuration file with a data structure desctiption at the input path
in YAML_ format. This configuration file has to be mounted on
`/app/sources.yaml` inside the container.
See [sources_example.yaml](/sources_example.yaml) for a configuration
example.
The easiest way to deploy in your **Runtime machine** is by pulling the image
from a remote registry. You can use the official registry at synergyimk_ or use
the instructions at :doc:`build` to create your image and uploaded at your own registry.
.. _YAML: https://yaml.org/
.. _synergyimk: https://hub.docker.com/r/synergyim
Once the requierement are needed, pull the image from the image registry.
Once you decide from which registry download, pull the image that image registry.
For example, to pull it from the synergy-imk official registry use:
.. code-block:: bash
$ udocker pull synergyimk/o3skim
...
Downloading layer: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Downloading layer: sha256:......
...
Note it is also possible to use `udocker load`_ to import images generated by
`docker save`_.
.. _`udocker load`: https://indigo-dc.gitbook.io/udocker/user_manual#1-4-basic-flow
.. _`docker save`: https://docs.docker.com/engine/reference/commandline/save
Once the repository is added and the image downloaded, create the local container:
Once the image is downloaded or imported, create the local container.
For example, if it was downloaded from synergyimk registry you can use:
.. code-block:: bash
$ udocker create --name=o3skim synergyimk/o3skim
fa42a912-b0d4-3bfb-987f-1c243863802d
Check the containers
available at the **Runtime machine**:
.. code-block:: bash
$ udocker ps
CONTAINER ID P M NAMES IMAGE
...
fa42a912-b0d4-3bfb-987f-1c243863802d . W ['o3skim'] synergyimk/o3skim:latest
Now you are ready to start using the container as `o3skim`. Read how to use the
:doc:`cli` as first steps to skim your data.
......@@ -3,12 +3,14 @@ Prerequisites
To run the project as container, you need the following systems and container technologies:
- **Build machine** with docker_
- **Runtime machine** with udocker_
- **Build machine** with docker_ in case you want/need to build your own image.
- **Runtime machine** with udocker_ and access to the data to skim.
In case you do not want to create your image, last images are uploaded in dockerhub
at synergyimk_.
.. rubric:: Note udocker_ cannot be used to build containers, only to run them.
.. rubric:: Note udocker_ cannot build containers but run them.
.. _docker: https://docs.docker.com/engine/install/
.. _udocker: https://indigo-dc.gitbook.io/udocker/installation_manual
.. _synergyimk: https://hub.docker.com/r/synergyimk
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment