Steps for successfully running an open source GitHub repo workflow on the Form Bio platform
- Create a Private Fork of Public repository
- Update Your Forked Repo
- Create correct structure
- Convert json schema
- Update nextflow.config
- Import Workflow
Create a Private Fork of Public repository
Make a private fork of a public github repo e.g.:
HiFi-16S-workflow
Download and install
- https://github.com/cli/cli - github's official cli tool
Or create a new repository from the Github web UI.
GITHUB_REPO_ORG=PacificBiosciences
GITHUB_REPO_NAME=HiFi-16S-workflow
gh repo create --private formbio/$GITHUB_REPO_NAME
git clone --bare https://github.com/$GITHUB_REPO_ORG/$GITHUB_REPO_NAME.git
cd $GITHUB_REPO_NAME.git
git push --mirror https://github.com/formbio/$GITHUB_REPO_NAME.git
Then you can just check it out and you can add the original repo as an upstream to pull changes from and keep in sync:
git clone https://github.com/formbio/$GITHUB_REPO_NAME.git
cd $GITHUB_REPO_NAME
git remote add upstream https://github.com/$GITHUB_REPO_ORG/$GITHUB_REPO_NAME.git
git remote set-url --push upstream DISABLE
# Pull (fetch & merge) from latest upstream (e.g. merge in changes from original repo)
git pull upstream main --rebaseUpdate Your Forked Repo
Create correct structure
- Ensure
main.nfandnextflow.configlive in the root of the repo - Create a “workflows” directory inside the repo, if one does not already exist.
- Inside that directory, create another directory with same name as the ID of the workflow (if one does not already exist)
- Inside that directory, create a
workflow.jsonfile and symlink any documentation that already exists in the repo here under the namesoverview.mdcitations.mdinputs.mdoutputs.md - You can also create your own documentation files if you’d like
Convert json schema
- Convert the repo’s existing json schema to a Form Bio workflow schema in
workflow.json, using our Schema Guide to help you - There is a script inside our workflow-schema repo that will help with this process if you have node installed on your machine. To use it:
- Clone the repo to your local machine
- Cd into repo → workflow-schema/
- Run
sh scripts/json-schemaToV3.sh <path to json-schema file> - This will output a
workflow.jsonin our format inside the json-schema file’s parent directory - It is important to then go through the output and fill in missing fields, like
id, and fix any field types that the conversion script may have missed, and verify everything looks okay - It can then be moved to the correct
workflows/[id]directory - Some important notes:
- Ensure the value you give the
idproperty matches the parent directory’s name - If the schema includes a field for specifying an output directory, make sure to hide it
- Keep the
idof the output param in the back of your mind for future steps
Update nextflow.config
- Beneath the section where default
paramsare defined, include statement that remaps our output param to theirs: params.out_dir = "${params.output}"where “out_dir” is their param name
Import Workflow
See