Commit ed9b0661 authored by uoega's avatar uoega
Browse files

readme minor change

parent 8a38216e
......@@ -8,8 +8,8 @@ Gesture Recognition** by David Heiming, Hannes Uhl and Jonas Linkerhaegner from
This module is located in the folder **st-gcn_original** and can be used like the original from the git repo [st-gcn](https://github.com/yysijie/st-gcn). Additional files:
- **/config/st_gcn** contains two new folders with the config files for our splits
- **/tools** contains three python scripts "ntu_gendata.py", "ntu_gendata_zsasr.py" and "ntu_gendata_zsar_nearest_cos.py" to generate the training splits for the ST-GCN.
- **/processor** contains the python file "feature_extraction.py" to generate the 256 dimensional features of the the classes the ST-GCN was not trained on
- **/tools** contains three python scripts `ntu_gendata.py`, `ntu_gendata_zsasr.py` and `ntu_gendata_zsar_nearest_cos.py` to generate the training splits for the ST-GCN.
- **/processor** contains the python file `feature_extraction.py` to generate the 256 dimensional features of the the classes the ST-GCN was not trained on
## SBERT
......@@ -17,11 +17,11 @@ This module is located in the folder **Bert_language_embeddings**.
### Language Embeddings
The most important file to generate the [SBERT mean class label embedding](https://huggingface.co/sentence-transformers/bert-base-nli-mean-tokens) is the python file "class_label_embedding_bert_mean.py".
The most important file to generate the [SBERT mean class label embedding](https://huggingface.co/sentence-transformers/bert-base-nli-mean-tokens) is the python file `class_label_embedding_bert_mean.py`.
We additionally experimented with the [cls token](https://huggingface.co/sentence-transformers/bert-base-nli-cls-token) and [mpnet](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) from [sentence transformers](https://www.sbert.net/).
You can find the different versions of the manually annotated descriptive labels in the .txt files "class_sentences_40_verx.txt" with x beeing the version between one and five.
You can find the different versions of the manually annotated descriptive labels in the .txt files `class_sentences_40_verx.txt` with x beeing the version between one and five.
- Version 1: First version of the visually focussed descriptions.
- Version 2: Corrected Version 1
......@@ -37,27 +37,28 @@ The files for the automatic augmentation can be found in the subfolder **/augmen
This module is located in the folder **LearningToCompare_ZSL**. Our Version is based on the implementation of the [original paper](https://arxiv.org/abs/1711.06025) from [LearningToCompare_ZSL](https://github.com/lzrobots/LearningToCompare_ZSL).
The main file to train the Zero-Shot part of the architecture is "NTU_RelationNet_copy.py". There are different input arguments most importantly the unseen classes (from 1 to 40) and the used label embedding. For example:
The main file to train the Zero-Shot part of the architecture is `NTU_RelationNet_copy.py`. There are different input arguments most importantly the unseen classes (from 1 to 40) and the used label embedding. For example:
```python:
python NTU_RelationNet_copy.py -u 2 9 11 18 38 -s sentence_40_mean_ver1_norm
```
This uses the classes 2,9,11,18 and 38 as unseen classes and the ver1 descriptive embedding as label embedding.
### Multiple Labels
Here the file "NTU_RelationNet_random_multi_label.py" is used for training. The input arguements are the same like for the single label approach but now more than one label embedding can be used, for example:
Here the file `NTU_RelationNet_random_multi_label.py` is used for training. The input arguements are the same like for the single label approach but now more than one label embedding can be used, for example:
```python:
python NTU_RelationNet_random_multi_label.py -u 2 9 11 18 38 -s sentence_40_mean_ver1_norm sentence_40_mean_ver2_norm sentence_40_mean_ver5_norm
```
## Additional files
As usual more experiments than the ones presented in the paper were performed. The corresponding files are put together here.
As usual more experiments than the ones presented in the paper were performed. The corresponding files are put together here.
### Simase Networks
Located in the folder **siamese-triplet** are all files used for experiments using a siamese net to cluster the visual features from the ST-GCN.
## Results
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment