The Wellcome Sanger Institute in Cambridge applies and explores genomic technologies at scale to advance understanding of biology and improve health. My team at Sanger helped the Human Genetics faculty groups evaluate and access the best methods to process and absorb the huge amounts of sequencing data produced by modern studies. I worked at Sanger as a senior software engineer between 2019 and 2022.
Human Genetics Informatics (HGI) Cloud) was a complex codebase that allowed automate provisioning of resources (networks, machines, security groups, ssh keys, ip addresses etc.) for researchers in Sanger’s on-prem cloud. A mix of terraform, ansible, python and bash, this work was initiated by a contracter (Luca) we had hired. The most common use-case for this was hail deployment : provisioning spark clusters (running hail software).
I played a leading role in absorbing and internalising (making comprehensive documentaiton etc.) hgi-cloud in our team. I then extended it extended the provisioning codebase to automate provisioned of docker swarm clusters using Ansible, Terraform, Python, Docker. This took a little longer than I anticipated, but it was a useful exercise, as it involved working with all the tech used in Luca’s code. Troubleshooting various bugs forced me to learn “sysadmin stuff” like networking.
We then used the automated framework to deploy a Docker Swarm clusters for the automated deployment and maintance of all our web apps. These were then integrated in one common user-facing platform.
One of the web applications I developed was a a resource allocation GUI for users (OpenStack Cluster Usage Report) to report on usage of clusters on Sanger’s on-prem Cloud using Python, AIOTTP and Vue.js. Researchers would use it to monitor everyone’s resource usage in the cloud compute environment, allowing them to coordinate for optimal allocation. This evolved gradually into a full fledged web app as we add more features, incorporate more OpenStack querying. The biggest challenge in this was that Oenstack APIs are not very well documented. For example, I had to muddle my way around for hours in the source code of a python-openstack client before I was able to find a way to query CPU time.
I then moved on to developing CLI software (Crook/Vault) which enables the automation of our data governance policies and supports archival of data to iRODS as part of a cron job.
In between developing software, I also helped some bioinformaticians and postdocs with automating their workflow on FCE. Even though these would not normally be high in the priority list, it felt more rewarding than other tasks as I could see the direct impact of my work, and that external validation helps.
“the machine in openstack is way faster than the one in the farm, I am impressed! something that takes 2 to 3 hours on the farm takes 15-30 min on the openstack machine.
it is so much faster I set it to do some stuff that would have taken me days to do on the farm”
Sanger had a really good learning environment. There were regular seminars (HumGen Seminar, IGM Meetings). I also partcipated in training for Farm (1 day) and Python (3 Day) . My line manager (Vivek Iyer) was really supportive, allowed me to flexibility to work on a number of different projects in different parts of the stack (what I wanted), and gave me time to pick up the skills I needed to deliver them. By the time I left, I had picked up frameworks and languages for infrastucture automation (terraform, ansible, docker, docker swarm), frontend (vuejs, react, bootstrap css), system design (networking, storage etc.), bash scripting, and advanced my python skills.
25 July 2017