Commit 3e1d0732 authored by Boris Baldassari's avatar Boris Baldassari
Browse files

WIP: add image, move repo description.

parent ffcb2d1f
Pipeline #2868 passed with stage
......@@ -61,33 +61,16 @@ More information:
* Temple university dataset homepage: https://isip.piconepress.com/projects/tuh_eeg/html/downloads.shtml#c_tusz
* Temple university dataset reference: Obeid I., Picone J. (2016). The temple university hospital EEG data corpus. Front. Neurosci. 10:196. 10.3389/fnins.2016.00196.
## Areas of improvement
### Industrialisation: Cleaning/Refactoring of the repository
## Areas of improvement
The work done by the AURA researchers and data scientists on ECGs had been organised in a bunch of GitHub repositories, with different people using different tools and structures. The first step was to identify the parts required to run the complete workflow, and extract them from the various repositories and branches to build a unified structure. The requirements of this repository structure are:
* Re-use common scripts into each process automatically (no redundancy).
* Provide up-to-date documentation and passing tests.
* Set up a process to automatically build the Docker images to allow multiple execution methods: Airflow/pure python, Docker/Compose, Kubernetes or Eclipse Graphene.
We worked
Building upon the current resources in use at AURA for Ai workflow, the following directory structure was adopted:
{{< grid/div isMarkdown="false" >}}
<img src="/images/articles/aice_aura_demonstrator/ecg_workflow.png" alt="The AURA AI process - before" class="img-responsive">
{{</ grid/div >}}
```
├── data => Data samples for tests
├── graphene => All Docker images
│ ├── aura_dataprep => - Data processing Docker image
│ ├── aura_ml_trainer => - ML training Docker image
│ └── ...
├── resources => Documentation, images..
├── scripts => Repo-related scripts for builds, integration..
├── src => AI-related scripts and source code
│ ├── domain
│ ├── infrastructure
│ └── usecase
└── tests => Tests for scripts and source code
```
We defined and enforced a contributing guide, making tests and documentation mandatory in the repository. We also set up a Travis job to execute the full python test suite at every commit, and made it visible through a badge in the repository's README. Regarding Git we used a simple Git workflow to maintain a clean branching structure. The newly agreed development process definitely helped clean up the repository. Each time a set of scripts was added, we knew exactly where they should go and how to reuse them in the overall workflow.
### Portability: Building the AURA Containers
......@@ -153,6 +136,33 @@ The process of importing the rr-intervals and annotations is time- and resource-
It is also very important to interpret visually and discuss the outcomes of the AI-based seizure detector with the healthcare professionals in order to build trust and assess limitation of the algorithm. Having an easy way to import ECGs to easily visualise and annotate them is a major benefit in this context, especially in healthcare centers where teams do not always have the resources and knowledge to set up a complex software stack. We are now working on a database dump that will enable end-users to import specific datasets into their own Postgres / Grafana instance in a few clicks, thus fostering the usage and research on open datasets.
### Industrialisation: Cleaning/Refactoring of the repository
The work done by the AURA researchers and data scientists on ECGs had been organised in a bunch of GitHub repositories, with different people using different tools and structures. The first step was to identify the parts required to run the complete workflow, and extract them from the various repositories and branches to build a unified structure. The requirements of this repository structure are:
* Re-use common scripts into each process automatically (no redundancy).
* Provide up-to-date documentation and passing tests.
* Set up a process to automatically build the Docker images to allow multiple execution methods: Airflow/pure python, Docker/Compose, Kubernetes or Eclipse Graphene.
Building upon the current resources in use at AURA for Ai workflow, the following directory structure was adopted:
```
├── data => Data samples for tests
├── graphene => All Docker images
│ ├── aura_dataprep => - Data processing Docker image
│ ├── aura_ml_trainer => - ML training Docker image
│ └── ...
├── resources => Documentation, images..
├── scripts => Repo-related scripts for builds, integration..
├── src => AI-related scripts and source code
│ ├── domain
│ ├── infrastructure
│ └── usecase
└── tests => Tests for scripts and source code
```
We defined and enforced a contributing guide, making tests and documentation mandatory in the repository. We also set up a Travis job to execute the full python test suite at every commit, and made it visible through a badge in the repository's README. Regarding Git we used a simple Git workflow to maintain a clean branching structure. The newly agreed development process definitely helped clean up the repository. Each time a set of scripts was added, we knew exactly where they should go and how to reuse them in the overall workflow.
## Benefits
### Industrialisation of the solution
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment