Getting Started
Creating New Projects
Adding New Members To Projects
Adding Files to Projects
Workflows
Toolbox
Form Bench
Reference
FAQs
Form Bio CLI SDK
Downloading Workflow Results
Roles and Permissions
Form Bio CLI/SDK
formbio
is a high-level command line tool for managing Form Bio workflows and projects.
- Form Bio CLI/SDK
- Installing and Updating the formbio CLI tool
- Windows
- MacOS
- Linux
- Commands
- Authentication
- Login
- Logout
- Whoami
- Data Upload and Download
- List
- Upload / Download
- Create Directory
- Run Workflows on Form Bio
- Legal Notices
The Form Bio CLI SDK is under active development and subject to change.
Installing and Updating the formbio
CLI tool
To install the latest version of the formbio
CLI tool or update your existing installation with the latest you can run the following depending on your operating system.
Windows
# Install via PowerShell
iwr https://storage.googleapis.com/formbio-go-releases/public/latest/install/windows | iex
MacOS
# Install via command line
curl -sSL https://storage.googleapis.com/formbio-go-releases/public/latest/install/macos | bash
# Make it available on your $PATH (only need to do this the first time you install)
sudo ln -s $HOME/.local/bin/formbio /usr/local/bin/formbio
Linux
# Install via command line
curl -sSL https://storage.googleapis.com/formbio-go-releases/public/latest/install/linux | bash
# Make it available on your $PATH (only need to do this the first time you install)
sudo ln -s $HOME/.local/bin/formbio /usr/local/bin/formbio
Commands
‣
formbio --help
$ formbio --help
manages Formbio workflows, storage and resources.
Usage:
formbio [command]
Available Commands:
auth Runs authentication subcommands
completion Generate the autocompletion script for the specified shell
help Help about any command
storage Runs storage subcommands
version Version will output the current build information
workflow Runs workflow subcommands
Flags:
--config string Select your config file (default is $HOME/.formbio.yaml).
--debug Print debug log lines in the CLI console.
-h, --help help for formbio
Use "formbio [command] --help" for more information about a command.
‣
formbio auth --help
$ formbio auth --help
Runs authentication subcommands
Usage:
formbio auth [command]
Available Commands:
login Authenticates with the Form Bio platform for running CLI commands
logout Logs the current user out
whoami Prints the current user's email
Flags:
-h, --help help for auth
Global Flags:
--config string Select your config file (default is $HOME/.formbio.yaml).
--debug Print debug log lines in the CLI console.
Use "formbio auth [command] --help" for more information about a command.
‣
formbio storage --help
$ formbio storage --help
Runs storage subcommands
Usage:
formbio storage [command]
Available Commands:
cp Copies and object or tree of objects
ls Lists the stored objects for a specific project
Flags:
-h, --help help for storage
Global Flags:
--config string Select your config file (default is $HOME/.formbio.yaml).
--debug Print debug log lines in the CLI console.
Use "formbio storage [command] --help" for more information about a command.
‣
formbio workflow run --help
$ formbio workflow run --help
Runs a Nextflow workflow through the Form Bio platform
Usage:
formbio workflow run --org org --project project --repo repo --version version --run-name runName -- [--paramName <param-value>] [--paramName2 <param-value>] [flags]
Examples:
formbio workflow run --org=org-production --project=projectA --repo=formbio/nf-examplepipeline --version=main --run-name="testInputs" -- --input "formbio://org-slug/project-slug/sequencing" --batchFile "formbio://org-slug/project-slug/design.batch"
Flags:
--config string If --workflow-container is true, a stringified JSON object containing container config values (e.g. '{"inputs": ["filePath1"], "outputs": ["outputs/"]}')
--config-file string If --workflow-container is true, the location of a .json file containing container-config options
-h, --help help for run
--nextflow-params string Nextflow pipeline params JSON e.g. --nextflow-params='{"input": "gs://test-bucket/test.txt", "batchInput": "gs://test-bucket/test.batch"}'
--nextflow-version string The version of Nextflow that you want to use for your workflow run.
--org string The slug of the organization you are accessing.
--project string The slug of the project you are accessing.
--repo string The repository you want to source for your Nextflow script (e.g. formbio/nf-proteingetinfo).
--require-schema-validation If set to true, a workflow run will not be started unless the inputs can be validated against the workflow schema. (default true)
--run-name string The name of the pipeline run that it will be indexed under.
--set-default-inputs If set to true, empty workflow inputs will be auto-filled with default values (if available). (default true)
--skip-workflow-results string Skip running workflow, copy provided workflow results to output folder. Must be in "org://project/workflow_run_id" format.
--timeout duration Timeout for running Nextflow on google life sciences. (default 720h0m0s)
--version string Github branch or tag to run (e.g. main or v1.0.2).
--workflow string The name of the workflow you are running.
--workflow-container If set to true, workflow container will run directly rather than use an uploaded workflow. Must include --config or --config-file
Global Flags:
--debug Print debug log lines in the CLI console.
Authentication
Login
‣
formbio auth login --help
$ formbio auth login --help
Authenticates with the Form Bio platform for running CLI commands
Usage:
formbio auth login [flags]
Flags:
-h, --help help for login
Global Flags:
--config string Select your config file (default is $HOME/.formbio.yaml).
--debug Print debug log lines in the CLI console.
Example usage:
# logs the user in
formbio auth login
‣
Authenticating Form Bio CLI...
Log in at https://form.bio/auth/login-token/, then copy/paste your Form Bio CLI API Token below...
Enter the CLI API Token:
// User's browser will open to the Form Bio platform where they can
// 1. Log in with email and password
// 2. Log in with Google Signin
// Once user pastes in the CLI API Token, they will see this message:
Authenticated as [user@email.com]
To switch users, use:
formbio auth logout
# logs the user in when they are already authenticated
formbio auth login
‣
Already authenticated as [user@email.com] - to switch users, use:
formbio auth logout
Logout
‣
formbio auth logout --help
$ formbio auth logout --help
Logs the current user out
Usage:
formbio auth logout [flags]
Flags:
--env string The Bantha environment you want to access [local|dev|sbx|stg|prod].
-h, --help help for logout
Global Flags:
--config string Select your config file (default is $HOME/.formbio.yaml).
--debug Print debug log lines in the CLI console.
Example usage:
# logs the user out
formbio auth logout
‣
You have successfully logged out.
Whoami
‣
formbio auth whoami --help
$ formbio auth whoami --help
Prints the current user's email
Usage:
formbio auth whoami [flags]
Aliases:
whoami, who
Flags:
--env string The Bantha environment you want to access [local|dev|sbx|stg|prod].
-h, --help help for whoami
Global Flags:
--config string Select your config file (default is $HOME/.formbio.yaml).
--debug Print debug log lines in the CLI console.
Example usage:
# list currently authenticated user
formbio auth whoami
‣
You are currently authenticated as Form Bio user user@email.com
Data Upload and Download
List
‣
formbio storage ls --help
$ formbio storage ls --help
Lists the stored objects for a specific organization and project. The path should be in the format "formbio://<organization>/<project>/<prefix>".
Usage:
formbio storage ls [-r | --recursive] <path> [flags]
Examples:
# list the root level objects for a project
formbio storage ls --env=prod formbio://my-organization/my-project/
# recursively list the objects in a specific directory
formbio storage ls --env=prod -r formbio://my-organization/my-project/workflow-input-files/
Flags:
--env string The Bantha environment you want to access [local|dev|sbx|stg|prod].
-h, --help help for ls
-r, --recursive List objects recursively, default is non-recursive.
Global Flags:
--config string Select your config file (default is $HOME/.formbio.yaml).
--debug Print debug log lines in the CLI console.
Example usage:
# list files in top level of project space
formbio storage ls \
formbio://my-organization/my-project/
‣
Authenticating on the "prod" environment
Authenticated as [my_user@formbio.com] on the prod environment
Looking up stored objects
total objects: 3
formbio://my-organization/my-project/:
2022-06-23T23:54:20Z formbio://my-organization/my-project/logs/
2022-10-07T05:18:53Z formbio://my-organization/my-project/my_file.txt
# list files recursively in the "logs" directory
formbio storage ls -r \
formbio://my-organization/my-project/logs/
‣
Authenticating on the "prod" environment
Authenticated as [my_user@formbio.com] on the prod environment
Looking up stored objects
total objects: 2
formbio://my-organization/my-project/logs/:
formbio://my-organization/my-project/logs/end/
formbio://my-organization/my-project/logs/end/:
formbio://my-organization/my-project/logs/end/of/
formbio://my-organization/my-project/logs/end/of/:
formbio://my-organization/my-project/logs/end/of/the/
formbio://my-organization/my-project/logs/end/of/the/:
formbio://my-organization/my-project/logs/end/of/the/rainbow/
formbio://my-organization/my-project/logs/end/of/the/rainbow/:
338361 2022-10-07T05:21:09Z formbio://my-organization/my-project/logs/end/of/the/rainbow/pot-of-gold.png
Upload / Download
‣
formbio storage cp --help
Copies and object or tree of objects from source to target, used for downloading, uploading, and copying, where source and target can be either local paths or formbio URLs in the format "formbio://<organization>/<project>/<prefix>".
Usage:
formbio storage cp [-r | --recursive] [-c | --continue] <source> <target> [flags]
Examples:
# upload single file to a directory
formbio storage cp --env=prod myfile.txt formbio://my-organization/my-project/
# download a directory into the current local directory
formbio storage cp --env=prod -r formbio://my-organization/my-project/workflow-input-files/ .
# copy a directory from one project into another
formbio storage cp --env=prod -r formbio://my-organization/project-a/my-dataset/ formbio://my-organization/project-b/my-dataset/
Flags:
-c, --continue To resume an uploaded previously partially completed, use the continue flag.
--env string The Bantha environment you want to access [local|dev|sbx|stg|prod].
-h, --help help for cp
-r, --recursive To copy a tree of objects, specify a recursive copy.
Global Flags:
--config string Select your config file (default is $HOME/.formbio.yaml).
--debug Print debug log lines in the CLI console.
Example usage:
# download a single file from the project into mylocaldir
formbio storage cp \
formbio://my-organization/my-project/logs/my_file.txt \
mylocaldir/
‣
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Starting download
Preparing Download █████████████████████████████████████████████████ 100% | 0s
SUCCESS logs/my_file.txt
DownloadingFiles █████████████████████████████████████████████████ 100% | 0s
Files Downloaded: 1/1
Total Data Downloaded: 104 KiB/104 KiB
Download complete
# upload a single file to the project root
formbio storage cp \
mylocaldir/my_file.txt \
formbio://my-organization/my-project/
‣
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Starting upload
Preparing Upload █████████████████████████████████████████████████ 100% | 1s
SUCCESS formbio://my-organization/my-project/my_file.txt
Uploading Files █████████████████████████████████████████████████ 100% | 1s
Files Uploaded: 1/1
Total Data Uploaded: 104 KiB/104 KiB
Upload complete
# upload a local directory to a project subdirectory
formbio storage cp -r \
mylocaldir \
formbio://my-organization/my-project/subdir/
‣
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Starting upload
Preparing Upload █████████████████████████████████████████████████ 100% | 2s
SUCCESS formbio://my-organization/my-project/logs/subdir/mylocaldir/my_file1.txt
SUCCESS formbio://my-organization/my-project/logs/subdir/mylocaldir/my_file2.txt
Uploading Files █████████████████████████████████████████████████ 100% | 1s
Files Uploaded: 2/2
Total Data Uploaded: 209 KiB/209 KiB
Upload complete
# resume a partially completed upload
formbio storage cp -c \
one_gig_file.txt \
formbio://my-organization/my-project/large-files-dir/
‣
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Starting upload
Preparing Upload █████████████████████████████████████████████████ 100% | 1s
Uploading Files ███████████████████████████████████████ 79% | 0s
Create Directory
# use the -p (--parents) flag to create several nested directories at once
formbio storage mkdir \
-p \
formbio://my-organization/my-project/existing_dir/new_dir/also_new
‣
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Successfully created new directory
Run Workflows on Form Bio
‣
formbio workflow run --help
$ ./bin/formbio workflow run --help
Runs a Nextflow workflow through the Form Bio platform
Usage:
formbio workflow run --org org --project project --repo repo --version version --run-name runName -- [--paramName <param-value>] [--paramName2 <param-value>] [flags]
Examples:
formbio workflow run --org=org-production --project=projectA --repo=formbio/nf-examplepipeline --version=main --run-name="testInputs" -- --input "formbio://org-slug/project-slug/sequencing" --batchFile "formbio://org-slug/project-slug/design.batch"
Flags:
--config string If --workflow-container is true, a stringified JSON object containing container config values (e.g. '{"inputs": ["filePath1"], "outputs": ["outputs/"]}')
--config-file string If --workflow-container is true, the location of a .json file containing container-config options
-h, --help help for run
--nextflow-params string Nextflow pipeline params JSON e.g. --nextflow-params='{"input": "gs://test-bucket/test.txt", "batchInput": "gs://test-bucket/test.batch"}'
--nextflow-version string The version of Nextflow that you want to use for your workflow run.
--org string The slug of the organization you are accessing.
--project string The slug of the project you are accessing.
--repo string The repository you want to source for your Nextflow script (e.g. formbio/nf-proteingetinfo).
--require-schema-validation If set to false, a workflow run will submit even if the inputs fail to validate against the workflow schema. (default true)
--run-name string The name of the pipeline run that it will be indexed under.
--set-default-inputs If set to false, empty workflow inputs will not be auto-filled with available default values. (default true)
--skip-workflow-results string Skip running workflow, copy provided workflow results to output folder. Must be in "org://project/workflow_run_id" format.
--timeout duration Timeout for running Nextflow on google life sciences. (default 720h0m0s)
--version string Github branch or tag to run (e.g. main or v1.0.2).
--workflow string The name of the workflow you are running.
--workflow-container If set to true, workflow container will run directly rather than use an uploaded workflow. Must include --config or --config-file
Global Flags:
--debug Print debug log lines in the CLI console.
Example Workflow run:
# For a given Form Bio Org / Project e.g. Org=form-bio-solutions / Project=rna-seq
# https://form.bio/organizations/form-bio-solutions/projects/rna-seq/home/
$ formbio workflow run \
--env prod \
--org form-bio-solutions \
--project rna-seq \
--repo nf-rnaseq \
--workflow expressiondetective \
--version v1.0.0 \
--run-name 'runRNASEq' \
-- \
--design=formbio://form-bio-solutions/rna-seq/SRP045500.design.txt \
--input=formbio://form-bio-solutions/rna-seq/pipeline-outputs/getncbi/2023-05-06T00_24_44Z_90a32859-6a83-4fec-882d-38df4147a4aa/output/ \
--genomeid=GRCh38 \
--genoannotver=latest \
--version=rnaseq_v1.0.0 \
--trim=true \
--cancer=false \
--aligner=star \
--repoDir=/seqprg \
--markdups=skip \
--stranded=U \
--readorigin=F \
--sourceType=cts \
--orientation=I \
--genomes_bucket=gs://bioinfo-workflow-references/genomes \
--taxid=9606
Creating the workflow...
Your run output is available at:
https://form.bio/organizations/form-bio-solutions/projects/rna-seqUsage:
formbio workflow run --org org --project project --repo repo --version version --runName runName -- [--paramName <param-value>] [--paramName2 <param-value>] [flags]
Legal Notices
The formbio CLI tool uses the following Open Source software libraries.
- github.com/gosimple/slug@1.12.0
- https://github.com/hashicorp/errwrap/tree/v1.1.0
- https://github.com/hashicorp/go-multierror/tree/v1.1.1
- github.com/hashicorp/hcl/json/parser@1.0.0
← Previous
Add link here
Next →
Add link here