README.md 4.23 KB
Newer Older
BorjaEst's avatar
BorjaEst committed
1
2
3
4
5
<p align="center">
  <a href="" rel="noopener">
 <img width=200px height=200px src="https://i.imgur.com/6wj0hh6.jpg" alt="Project logo"></a>
</p>

BorjaEst's avatar
BorjaEst committed
6
<h3 align="center">o3skim</h3>
BorjaEst's avatar
BorjaEst committed
7
8
9

<div align="center">

10
11
12
13
  [![pipeline status](https://git.scc.kit.edu/synergy.o3as/o3skim/badges/master/pipeline.svg)](https://git.scc.kit.edu/synergy.o3as/o3skim/-/commits/master)
  [![coverage status](https://git.scc.kit.edu/synergy.o3as/o3skim/badges/master/coverage.svg)](https://git.scc.kit.edu/synergy.o3as/o3skim/-/commits/master)
  [![License](https://img.shields.io/badge/license-GPL-blue.svg)](https://git.scc.kit.edu/synergy.o3as/o3skim/-/commits/master)
  [![Status](https://img.shields.io/badge/status-building-blue.svg)](https://git.scc.kit.edu/synergy.o3as/o3skim/-/commits/master) 
BorjaEst's avatar
BorjaEst committed
14
15
16
17
18

</div>

---

BorjaEst's avatar
BorjaEst committed
19
<p align="center"> Data pre-processing for ozone models 
BorjaEst's avatar
BorjaEst committed
20
21
22
    <br> 
</p>

BorjaEst's avatar
BorjaEst committed
23
# 📝 Table of Contents
BorjaEst's avatar
BorjaEst committed
24
- [About](#about)
25
26
27
- [Build using docker](#build)
- [Run using udocker](#deployment)
- [Documentation](#doc)
BorjaEst's avatar
BorjaEst committed
28
29
- [Authors](#authors)
- [Acknowledgments](#acknowledgement)
BorjaEst's avatar
BorjaEst committed
30
- [TODO](https://git.scc.kit.edu/synergy.o3as/o3skim/-/issues)
BorjaEst's avatar
BorjaEst committed
31

32
# About <a name = "about"></a>
33
This project provides the tools to preprocess, standardize and reduce ozone data for later transfer and plot. 
BorjaEst's avatar
BorjaEst committed
34

BorjaEst's avatar
BorjaEst committed
35
## Prerequisites
36
37
38
To run the project as container, you need the following systems and container technologies:
- __Build machine__ with [docker](https://docs.docker.com/engine/install/) 
- __Runtime machine__ with [udocker](https://indigo-dc.gitbook.io/udocker/installation_manual)
BorjaEst's avatar
BorjaEst committed
39

BorjaEst's avatar
BorjaEst committed
40
> Note udocker cannot be used to build containers, only to run them. 
BorjaEst's avatar
BorjaEst committed
41
42


43
# Built using docker <a name = "build"></a>
44
Download the repository at the __Build machine__ using git.
BorjaEst's avatar
BorjaEst committed
45
46
```sh
$ git clone git@git.scc.kit.edu:synergy.o3as/o3skim.git
47
48
Cloning into 'o3skim'...
...
BorjaEst's avatar
BorjaEst committed
49
```
50
Build the docker image at the __Build machine__ using docker.
BorjaEst's avatar
BorjaEst committed
51
52
53
54
55
```sh
$ docker build --tag o3skim .
...
Successfully built 69587025a70a
Successfully tagged o3skim:latest
BorjaEst's avatar
BorjaEst committed
56
```
57
If the build process succeeded, you can list the image on the docker image list:
BorjaEst's avatar
BorjaEst committed
58
59
60
61
62
```sh
$ docker images
REPOSITORY                         TAG                 IMAGE ID            CREATED              SIZE
o3skim                             latest              69587025a70a        xx seconds ago      557MB
...
BorjaEst's avatar
BorjaEst committed
63
64
```

65
# Run using udocker <a name = "deployment"></a>
66
67
68
To deploy the the application using __udocker__ at the __Runtime machine__ you need:
 - Input path with data to skim, to be mounter on `/app/data` inside the container.
 - Output path for skimmed results, to be mounted on `/app/output` inside the container.
69
70
 - Configuration file with a data structure description at the input path in [YAML](https://yaml.org/) format.
   This configuration file has to be mounted on `/app/sources.yaml` inside the container.
71

72
Once the requirement are completed, pull the image from the image registry.
73
74
75
76
77
78
79
80
81
For example, to pull it from the synergy-imk official registry use:
```sh
$ udocker pull synergyimk/o3skim
...
Downloading layer: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
...
```

Once the repository is added and the image downloaded, create the local container: 
BorjaEst's avatar
BorjaEst committed
82
```sh
83
84
85
86
87
88
89
90
91
92
93
$ udocker create --name=o3skim synergyimk/o3skim
fa42a912-b0d4-3bfb-987f-1c243863802d
```

Finally, run the container. Note the described _data_, _output_ and _sources.yaml_ have to be provided. Also it is needed to specify the user _application_ should run inside the container:
```sh
$ udocker run \
  --user=application \
  --volume=${PWD}/sources.yaml:/app/sources.yaml \
  --volume=${PWD}/data:/app/data \
  --volume=${PWD}/output:/app/output \
BorjaEst's avatar
BorjaEst committed
94
95
  o3skim --verbosity INFO
...
96
97
98
99
100
101
102
103
104
105
106
executing: main
...
2020-08-25 12:42:34,151 - INFO     - Configuration found at: './sources.yaml'
2020-08-25 12:42:34,152 - INFO     - Loading data from './data' 
2020-08-25 12:42:34,261 - INFO     - Skimming data to './output' 
```

For the main function description and commands help you can call:
```sh  
$ udocker run --user=application o3skim --help
...
BorjaEst's avatar
BorjaEst committed
107
108
109
```


110
111
112
113
# Documentation <a name = "doc"></a>
- [TODO]()


114
# Authors <a name = "authors"></a>
115
116
- [@V.Kozlov](https://git.scc.kit.edu/eo9869) - TBD
- [@T.Kerzenmacher](https://git.scc.kit.edu/px5501) - TBD
BorjaEst's avatar
BorjaEst committed
117
- [@B.Esteban](https://git.scc.kit.edu/zr5094) - TBD
BorjaEst's avatar
BorjaEst committed
118

119

120
# Acknowledgements <a name = "acknowledgement"></a>
BorjaEst's avatar
BorjaEst committed
121
- 
BorjaEst's avatar
BorjaEst committed
122