Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 17 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ For most users, a small Ubuntu VM inside OpenStack is the simplest and most repr
Recommended head VM characteristics:

```
Operating System: Ubuntu LTS (20.04 or newer)
Operating System: Ubuntu LTS (24.04 or newer)
CPU/RAM: minimal (e.g. 1–2 vCPUs, 2–4 GB RAM)
Access: SSH enabled via key pair
```
Expand All @@ -55,7 +55,7 @@ The following steps describe how to create a head VM using the generic OpenStack
### Select the Source Image (Operating System)

1. In the source tab, Select Boot Source: Image
2. Choose a supported Ubuntu image (eg. Ubuntu 22.04)
2. Choose a supported Ubuntu image (eg. Ubuntu 24.04)
3. If the BioShell image has been previously built you should be able to select BioShell at this step.

### Choose an Instance Flavor
Expand Down Expand Up @@ -199,9 +199,9 @@ packer init .

### Step 2: Prepare Packer build configuration

Before running the build, review and update `openstack-bioshell.pkr.hcl` to ensure the values match your OpenStack environment. If using a prepared config skip to step 3.
Before running the build, review and update `[platform_name].pkrvars.hcl` to ensure the values match your OpenStack environment. If using a prepared config skip to step 3.

**Note: Example working configurations for [Nectar](build/examples/openstack-bioshell-nectar.pkr.hcl) and [Nirin](build/examples/openstack-bioshell-nirin.pkr.hcl) are included and were last successfully tested on 2 February 2026. The Nirin configuration requires you to add your project [network](#network-cloud-dependant).**
**Note: Example working configurations for [Nectar](build/packer-vars/nectar.pkrvars.hcl) and [Nirin](build/packer-vars/nirin.pkrvars.hcl) are included and were last successfully tested on 2 February 2026. The Nirin configuration requires you to add your project [network](#network-cloud-dependant).**

At a minimum, check the following fields in the `source "openstack"` block:
#### Source Image (Base OS)
Expand All @@ -214,7 +214,7 @@ Example output:
+--------------------------------------+-------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+-------------------------------+--------+
| <uuid> | Ubuntu 20.04 | active |
| <uuid> | Ubuntu 24.04 | active |
+--------------------------------------+-------------------------------+--------+
```
Copy the ID of the image you want to use and set it as:
Expand Down Expand Up @@ -284,29 +284,26 @@ If your cloud has a default network, this field may be omitted.

#### CVMFS Configuration

The [`build-bioshell.yml`](build/build-bioshell.yml) playbook configures CVMFS for the image.
The [`ansible cvmfs role`](build/ansible/roles/cvmfs/tasks/main.yml) configures CVMFS for the image.

- By default, the CVMFS HTTP proxy is set to **DIRECT** to make the build more portable across environments.
- If a infrastructure specific proxy is available (eg. `http://cvmfs-proxy-1.nci.org.au:3128;http://cvmfs-proxy-2.nci.org.au:3128` on Nirin), update the `CVMFS_HTTP_PROXY` line in the playbook.
- If a infrastructure specific proxy is available (eg. `http://cvmfs-proxy-1.nci.org.au:3128;http://cvmfs-proxy-2.nci.org.au:3128` on Nirin), add new file `[platform_name].yml` to [`ansible vars`](build/ansible/vars/) with `cvmfs_proxy: [proxy]`.

The relevant task in the playbook:

```yaml
- name: Write default.local
copy:
dest: /etc/cvmfs/default.local
content: |
CVMFS_REPOSITORIES=data.biocommons.aarnet.edu.au,data.galaxyproject.org,singularity.galaxyproject.org
CVMFS_HTTP_PROXY='DIRECT'
CVMFS_QUOTA_LIMIT=4096
CVMFS_USE_GEOAPI=yes
```
### Step 3: Build BioShell
Once the configuration has been updated, run the build:

For Nirin and Nectar users:
```
./scripts/nirin.sh // nirin users
./scripts/nectar.sh. // nectar users
```
packer build openstack-bioshell.pkr.hcl
For other platform users:
```
packer build \
-var-file="./packer-vars/[platform_name].pkrvars.hcl" \
./openstack-bioshell.pkr.hcl
```

### Step 4: Verify Image
After the build process is complete, verify the newly created image by running:
```
Expand Down Expand Up @@ -364,7 +361,6 @@ This image uses CernVM-FS (CVMFS) to provide access to shared bioinformatics sof

Access CVMFS repositories:
```
ls /cvmfs/data.biocommons.aarnet.edu.au
ls /cvmfs/data.galaxyproject.org
ls /cvmfs/singularity.galaxyproject.org

Expand Down
16 changes: 16 additions & 0 deletions build/ansible/build-bioshell.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
- name: Build BioShell base image
hosts: all
become: true

vars_files:
- "vars/tool-versions.yml"
- "vars/{{ platform }}.yml"

roles:
- common
- singularity
- nextflow
- rstudio
- cvmfs
- shelley-bio
201 changes: 201 additions & 0 deletions build/ansible/roles/common/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
---
- name: Wait for apt lock
shell: |
while fuser /var/lib/apt/lists/lock >/dev/null 2>&1 || \
fuser /var/lib/dpkg/lock-frontend >/dev/null 2>&1 || \
fuser /var/lib/dpkg/lock >/dev/null 2>&1; do
sleep 5
done
changed_when: false

- name: Update apt cache
apt:
update_cache: yes
register: apt_update
retries: 3
delay: 10
until: apt_update is succeeded

- name: Install base packages
apt:
name:
- lmod
- build-essential
- libssl-dev
- uuid-dev
- libgpgme11-dev
- squashfs-tools
- git
- wget
- autoconf
- automake
- cryptsetup
- fuse2fs
- fuse
- libfuse-dev
- libglib2.0-dev
- libtool
- pkg-config
- runc
- squashfs-tools-ng
- uidmap
- zlib1g-dev
- libseccomp-dev
- nano
- jupyter
- snakemake
- r-base
- python3
- python3-pip
- python3-venv
- ansible
- tree
state: present

- name: Install R build dependencies
apt:
name:
- build-essential
- cmake
- gdal-bin
- libcurl4-openssl-dev
- libxml2-dev
- libfontconfig1-dev
- libfreetype6-dev
- libharfbuzz-dev
- libfribidi-dev
- libudunits2-dev
- libglpk-dev
- libblas-dev
- liblapack-dev
- libopenblas-dev
- libtiff5-dev
- libjpeg-dev
- libpng-dev
- libcairo2-dev

- name: Install Spack
block:
- name: Clone Spack repository
shell: |
git clone -c feature.manyFiles=true --depth=2 https://github.com/spack/spack.git /usr/local/spack

- name: Ensure module directories exist
file:
path: "{{ item }}"
state: directory
loop:
- "{{ module_base_opt }}"
- "{{ module_base_apps }}"

- name: Create /apps symlinks and paths
shell: |
mkdir -p /apps/R/{{ r_version }}/bin /apps/Modules/modulefiles/R
ln -sf /usr/bin/R /apps/R/{{ r_version }}/bin/R

mkdir -p /apps/jupyter/{{ jupyter_version }}/bin /apps/Modules/modulefiles/jupyter
ln -sf /usr/bin/jupyter /apps/jupyter/{{ jupyter_version }}/bin/jupyter

mkdir -p /apps/snakemake/{{ snakemake_version }}/bin /apps/Modules/modulefiles/snakemake
ln -sf /usr/bin/snakemake /apps/snakemake/{{ snakemake_version }}/bin/snakemake

mkdir -p /apps/ansible/{{ ansible_version.string }}/bin /apps/Modules/modulefiles/ansible
ln -sf /usr/bin/ansible /apps/ansible/{{ ansible_version.string }}/bin/ansible

args:
executable: /bin/bash

- name: Update PATH in /etc/environment
lineinfile:
path: /etc/environment
regexp: '^PATH='
line: 'PATH="{{ updated_path }}"'
state: present
vars:
current_path: "{{ ansible_env.PATH }}"
updated_path: "{{ current_path | replace('/usr/local/games', '') | trim | replace('::', ':') | regex_replace('^:|:$', '') }}:/usr/local/spack/bin"

- name: Comment out existing module paths
lineinfile:
path: /etc/lmod/modulespath
regexp: '^(.*(/etc/lmod/modules|/usr/share/lmod/lmod/modulefiles).*)'
line: '# \1'
backrefs: yes
state: present

- name: Add new module paths
lineinfile:
path: /etc/lmod/modulespath
line: "{{ item }}"
state: present
loop:
- "{{ module_base_apps }}"
- "{{ module_base_opt }}"

- name: Create jupyter modulefile
copy:
dest: "/apps/Modules/modulefiles/jupyter/{{ jupyter_version }}"
content: |
#%Module1.0

prepend-path PATH "/apps/jupyter/{{ jupyter_version }}/bin"

- name: Create snakemake modulefile
copy:
dest: "/apps/Modules/modulefiles/snakemake/{{ snakemake_version }}"
content: |
#%Module1.0

prepend-path PATH "/apps/snakemake/{{ snakemake_version }}/bin"

- name: Create R modulefile
copy:
dest: "/apps/Modules/modulefiles/R/{{ r_version }}"
content: |
#%Module1.0

prepend-path PATH "/apps/R/{{ r_version }}/bin"

- name: Create ansible modulefile
copy:
dest: "/apps/Modules/modulefiles/ansible/{{ ansible_version.string }}"
content: |
#%Module1.0

prepend-path PATH "/apps/ansible/{{ ansible_version.string }}/bin"

- name: Ensure motd.d directory exists
file:
path: /etc/motd.d
state: directory
mode: '0755'

- name: Add BioShell welcome message with turtle
copy:
dest: /etc/motd.d/bioshell
owner: root
group: root
mode: '0644'
content: |
##################################################################
# #
# Welcome to BioShell! #
# #
# A range of commonly used bioinformatics software is available #
# via CVMFS. #
# #
# To make the software folders visible, run: #
# cvmfs_config probe #
# #
# If you need help finding and installing software, #
# ask Shelley-Bio #
# _ .----. #
# (_ \/ \_, #
# `uu----uu' #
# Basic commands: #
# shelley-bio find <tool> #
# shelley-bio search "<function>" #
# shelley-bio versions <tool> #
# shelley-bio build <tool> #
# shelley-bio interactive #
##################################################################
Loading