Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
b7196f6
Deprecate old tools and black format
BioWilko Jun 20, 2024
841a0ef
Support fragmented data
BioWilko Jun 20, 2024
0b5628d
use new trim primer arg in minion
BioWilko Jun 20, 2024
3c156bd
reverse the (revolting) primer position logic
BioWilko Jun 20, 2024
569e6c9
Fix find primer logic
BioWilko Jun 20, 2024
ca9e839
Improve the cursed report writer logic
BioWilko Jun 20, 2024
3f6ecaf
explicitly named function args, what a novel idea
BioWilko Jun 20, 2024
c0331d8
add temp debugging prints to find_primer
BioWilko Jun 20, 2024
04196a3
Add logic to handle segments outside of primer scheme
BioWilko Jun 20, 2024
ee7b8d1
fix typo
BioWilko Jun 20, 2024
284fa64
General bugfixes, test fix for overlapping variant issue
BioWilko Jun 21, 2024
0c69bbd
add min mapq argument
BioWilko Jun 25, 2024
a56c591
properly provide min_mapq to handle_segment
BioWilko Jun 25, 2024
c806e8e
Fix unmatched read group issue
BioWilko Jun 25, 2024
8beee12
Fix 8 bit int error
BioWilko Jun 26, 2024
3fadfc8
First attempt to integrate clair3 as optional
BioWilko Aug 9, 2024
bd3555e
Slightly change env yaml format
BioWilko Aug 9, 2024
e6ea9da
Specify env name in mamba gh action
BioWilko Aug 9, 2024
eb496f3
Checkout repo before installing env
BioWilko Aug 9, 2024
c6f15dd
python 3.6.15
BioWilko Aug 9, 2024
75bc7dc
Run on ubuntu 20.04
BioWilko Aug 9, 2024
93749c3
unpin python
BioWilko Aug 9, 2024
1ee5afa
Unpin python entirely
BioWilko Aug 9, 2024
bce1af8
Separate out tests
BioWilko Aug 9, 2024
16eb5cb
correct unit test path
BioWilko Aug 9, 2024
d2f0ec2
remove retvar hints for compatibility
BioWilko Aug 9, 2024
db6ed57
Fix silly relative imports
BioWilko Aug 9, 2024
321d99c
Switch to cyvcf and fix unit tests
BioWilko Aug 9, 2024
63abec7
Fix pipeline parse unit test
BioWilko Aug 9, 2024
f26eb49
Fix fstring
BioWilko Aug 9, 2024
2aaa76d
Large changes to make clair3 work, various unit test updates
BioWilko Aug 12, 2024
6ca6b92
Fix fuzzy primer site match logic
BioWilko Aug 12, 2024
968782b
Fix minion_validator path
BioWilko Aug 12, 2024
f21404c
Fix vcf merge bug with empty VCF files
BioWilko Aug 12, 2024
3e1a4cb
Increase fuzzy match default threshold to 20
BioWilko Aug 12, 2024
382f5e9
Fix CHROM bug in vcf merge properly
BioWilko Aug 12, 2024
77e50cb
Report failed workflow when variants are missed
BioWilko Aug 12, 2024
b3e7d2f
Fix vcf merge again
BioWilko Aug 12, 2024
2089061
Separate out tests
BioWilko Aug 12, 2024
31c4def
No counter anymore
BioWilko Aug 12, 2024
b69da37
Add C241T to SP1 variants
BioWilko Aug 12, 2024
c5f2a7b
Add limit on ref pileup
BioWilko Aug 13, 2024
bef2f72
Single copy of CVR1 12733 var
BioWilko Aug 13, 2024
031dedb
re-enable other consensus tests
BioWilko Aug 13, 2024
0b14011
First attempt to make minion validator less fragile
BioWilko Aug 13, 2024
825c826
Unify testvariants, improve test logic
BioWilko Aug 13, 2024
ba1d58b
medaka has to be different smh
BioWilko Aug 13, 2024
cb16e8b
Index ref fasta when using the clair3 workflow
BioWilko Aug 14, 2024
92bd24f
Add mean amplicon depth TSV output
BioWilko Aug 15, 2024
b5832d5
Update CLI and scheme fetcher
BioWilko Aug 17, 2024
cc59756
Update align trim unit test
BioWilko Aug 17, 2024
ae19deb
Use proper version of V3 bedfile
BioWilko Aug 17, 2024
8520466
Fix pipeline unit test
BioWilko Aug 17, 2024
f7dae3c
Fix align trim amp depth bug
BioWilko Aug 17, 2024
a5da480
Call variants on non primertrimmed bam
BioWilko Aug 17, 2024
c288416
Don't test MT vs clair
BioWilko Aug 17, 2024
00d45e9
properly disable test
BioWilko Aug 17, 2024
e752bd5
Add docker build push action on release
BioWilko Aug 17, 2024
70c2f63
Grab wget in dockerfile
BioWilko Aug 17, 2024
dc217f8
Update docs and readme
BioWilko Aug 18, 2024
c068d67
docker push on release only
BioWilko Aug 18, 2024
02e2565
Remove debugging print statement
BioWilko Aug 18, 2024
bb596b9
Make fuzzy primer match logic clearer
BioWilko Aug 18, 2024
d7d9bb9
Install procps for nxf compatibility
BioWilko Aug 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/docker-build-push.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: quay_build_push

on:
release:
types: [published]

jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: quay.io/artic-network/fieldbioinformatics
flavor: |
latest=true
tags: |
type=semver,pattern={{version}}
- name: Checkout
uses: actions/checkout@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
45 changes: 45 additions & 0 deletions .github/workflows/unittests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: fieldbioinformatics_unit_tests

on: [push, pull_request]

jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
steps:
- uses: actions/checkout@v3
- uses: mamba-org/setup-micromamba@v1
name: Setup dependencies
with:
environment-file: environment.yml
init-shell: >-
bash
cache-environment: true
post-cleanup: "all"

- name: Pip install artic
run: |
python3 -m pip install -e .
shell: micromamba-shell {0}

- name: Run unit tests
run: |
pytest -s tests/*_unit_test.py
shell: micromamba-shell {0}

- name: Run medaka test
run: |
./test-runner.sh medaka
shell: micromamba-shell {0}

- name: Run clair3 test
run: |
./test-runner.sh clair3
shell: micromamba-shell {0}

- name: Run minion_validator tests
run: |
pytest -s tests/minion_validator.py
shell: micromamba-shell {0}

14 changes: 11 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,14 @@ dist/

# test data
*.index*
CVR1/
NRW01/
SP1/
CVR1.*
NRW01.*
SP1.*
MT007544*
tmp/
test-data/CVR1/
test-data/NRW01/
test-data/SP1/
test-data/MT007544/
rg_*
primer-schemes/
38 changes: 0 additions & 38 deletions .travis.yml

This file was deleted.

28 changes: 28 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM mambaorg/micromamba:1.5.8

COPY . ./fieldbioinformatics/

USER root

RUN apt-get update && apt-get install -y --no-install-recommends build-essential wget procps

USER $MAMBA_USER

COPY --chown=$MAMBA_USER:$MAMBA_USER environment.yml /tmp/env.yml

RUN sed -i 's/name: artic/name: base/' /tmp/env.yml

RUN micromamba install --yes --file /tmp/env.yml && \
micromamba clean --all --yes

ARG MAMBA_DOCKERFILE_ACTIVATE=1

USER root

RUN python3 -m pip install ./fieldbioinformatics

USER $MAMBA_USER

ENTRYPOINT ["/usr/local/bin/_entrypoint.sh"]

CMD ["/bin/bash"]
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,34 +27,34 @@ Features include:

There are **2 workflows** baked into this pipeline, one which uses signal data (via [nanopolish](https://github.com/jts/nanopolish)) and one that does not (via [medaka](https://github.com/nanoporetech/medaka)).

## Installation
<!-- ## Installation

### Via conda

```sh
conda install -c bioconda -c conda-forge artic
```

If conda reports that nothing provides particular packages when running the above command ensure your `channel_priority` is set to `flexible` using the following command:

```sh
conda config --set channel_priority false
```
Please make sure you are using either mamba or conda version >= 23.10.0 where libmamba solver was made the default conda solver. -->

### Via source

#### 1. downloading the source:

Download a [release](https://github.com/artic-network/fieldbioinformatics/releases) or use the latest master (which tracks the current release):
<!-- Download a [release](https://github.com/artic-network/fieldbioinformatics/releases) or use the latest master (which tracks the current release): -->

Clone the repository then checkout the 1.4.0-dev branch to test the 1.4.0 pre-release.

```sh
git clone https://github.com/artic-network/fieldbioinformatics
cd fieldbioinformatics
git checkout 1.4.0-dev
```

#### 2. installing dependencies:

The `artic pipeline` has several [software dependencies](https://github.com/artic-network/fieldbioinformatics/blob/master/environment.yml). You can solve these dependencies using the minimal conda environment we have provided:
The `artic pipeline` has several [software dependencies](https://github.com/artic-network/fieldbioinformatics/blob/master/environment.yml). You can solve these dependencies using the minimal conda environment we have provided, Please make sure you are using either mamba or conda version >= 23.10.0 where libmamba solver was made the default conda solver.
:

```sh
conda env create -f environment.yml
Expand All @@ -64,7 +64,7 @@ conda activate artic
#### 3. installing the pipeline:

```sh
python setup.py install
pip install .
```

#### 4. testing the pipeline:
Expand Down
3 changes: 3 additions & 0 deletions artic/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# pixi environments
.pixi
*.egg-info
Loading
Loading