Files and containers

Search…
GenomeHubs tools and services are designed to run in separate containers. While the overall system can be described in terms of the links between containers through shared files and the Ensembl databases, it is important to understand the interaction between the containers and the underlying filesystem.
The Docker containers used by GenomeHubs are ephemeral - if any container is stopped or removed, it can be replaced by another container of the same type without loss of data. The actual data are stored in directories on the host machine that mounted onto the containers while they are running. This approach means that GenomeHubs data files can always be accessed directly for further analyses or hosting in an alternative tool, and greatly simplifies versioning and backup.
The standard pattern is for each container to mount two directories, one containing configuration files that are specific to the software inside the container and one containing data to be imported/analysed/hosted:
1
container
2
├── conf
3
└── data
Copied!
The folders for each container can be nested inside a directory for the version that they relate to:
the MySQL data folder can be kept outside of the versioned directory to allow databases to be shared between versions
analysis containers are wrappers around specific bioinformatics software so they typically mount download/data subdirectories (see Run Analyses for details) to simplify the process of hosting the input and output files for bulk downloads
1
genomehubs
2
├── v1
3
│   ├── blast      # SequenceServer container
4
│   │   ├── conf
5
│   │   └── data
6
│   ├── download   # h5ai container
7
│   │   ├── conf
8
│   │   └── data
9
│   ├── ensembl    # EasyMirror container
10
│   │   ├── conf
11
│   │   └── data
12
│   ├── import     # EasyImport container
13
│   │   ├── conf
14
│   │   └── data
15
└── mysql          # MySQL container
16
    └── data
Copied!
An example of this directory structure with default configuration files is avalaible in the genomehubs/template github repository:
this template is used in the Quick Start guide, which contains more details of the roles of the individual configuration files
1
template
2
├── blast
3
│   ├── conf
4
│   │   ├── custom.css
5
│   │   ├── links.rb
6
│   │   ├── Masthead.html
7
│   │   └── img
8
│   │       ├── download-icon.png
9
│   │       ├── genomehubs-icon.png
10
│   │       ├── help-icon.png
11
│   │       └── tools-icon.png
12
│   └── data
13
├── download
14
│   ├── conf
15
│   │   ├── _h5ai.headers.html
16
│   │   ├── custom.css
17
│   │   ├── Masthead.html
18
│   │   └── img
19
│   │       ├── genomehubs-icon.png
20
│   │       ├── help-icon.png
21
│   │       └── tools-icon.png
22
│   └── data
23
├── ensembl
24
│   ├── conf
25
│   │   ├── database.ini
26
│   │   └── setup.ini
27
│   └── data
28
└── import
29
    ├── conf
30
    │   ├── default.ini
31
    │   ├── genus_species_asm_core_32_85_1.ini
32
    │   └── setup.ini
33
    └── data
Copied!
Introduction - Previous
GenomeHubs
Next - Introduction
Demo
Last modified 1yr ago
Copy link