LCA2020 Sysadmin Miniconf Presentations
The abstracts for the presentations accepted for the Linux.Conf.Au 2020 Syadmin Miniconf are listed below.
See the presentation schedule for the speaking order .
Presentation Titles
- OpenZFS and Linux - Nikolai Lusan
- Dev/Ops relationships, status: It's Complicated - Anna Fiofilova
- Samba 2020: Why are we still in the 1980s for authentication? - Andrew Bartlett
- Tyranny of the Clock - Craig Miskell
- Configuration Is (riskier than?) Code - Jamie Wilkinson
- Easy Geo-Redundant Handover + Failover with MARS + systemd - Thomas Schoebel-Theuer
- Why Linux Systems Administrators Should Care About the Mainframe - Elizabeth K. Joseph
- SRE for Smaller Organisations - Allan Shone
- Bash-my-AWS: CLI commands for managing AWS resources - Mike Bailey
- Using a cloud to manage a cloud - Alex Sharp and Andrew Reimers
- Don’t eat my data - 30+ years of storage war stories - Steven Ellis
- AI/ML won't save you - Julien Goodwin
- Highly available DNS recursion with PowerDNS - Dave Kempe
Full Abstracts
- OpenZFS and Linux - Nikolai Lusan
OpenZFS has come a long way in recent times, and with the deprecation of BTRFS we are not likely to see a filesystem with comparative feature set anytime soon. The ZoL (ZFS on Linux) project is now the main source of contributions to the OpenZFS code base, and is incredibly stable and robust. With OpenZFS now supporting native encryption it can simplify management of you data storage from the mdadm/luks/lvm method commonly seen deployed. Other features can improve performance and allow more efficient usage of disk storage. Whether you're storing personal files, hosting a development environment/webserver/email/database, running a virtual host, holding large amounts of corporate data there is probably a ZFS usage case for you.
This talk aims to give an outline of installing, using and administering a Linux server with ZFS as the root filesystem and for data storage. This includes:
- Caveats of installation and use.
- Benefits of ZFS.
- Designing/Managing storage pools.
- Demonstration of pool creation using different disk topologies.
- Using features like native encryption, block level compression, and database optimisation.
- Using snapshots, including for off site backup.
- ZFS and other Linux filesystems on one machine.
About: Nikolai Lusan
Having studied IT and History at the University of Queensland Nikolai has been a member of the Home Unix Machine Brisbane User Group since 1996, and is a standing member of ITPA (formerly SAGE-AU). Starting work in the IT industry in 1997 in database administration Nikolai has worked as a Systems Administrator and Systems Architect in the SME environment since 1998 administering primarily Linux systems with some other systems including FreeBSD , OpenBSD, Mac OSX server, and Cisco IOS.
Nikolai has administered everything from small networks with a single file server, to small dialup ISP's, hosting companies, and a fleet of web and database servers on 3 continents.
- Dev/Ops relationships, status: It's Complicated - Anna Fiofilova
This is short and entertaining talk in stand up comedy style about evolution of a difficult relationships between Developers and Ops. The story will be told from a Developer perspective of one person. You will remember good old days, learn a few DevOps tips and tricks and hopefully have some empathy for your colleagues on the other side of DevOps :)
No prior knowledge of any tech is required. Bring on your sense of humour!
Names, characters, businesses, places, events and incidents are either the products of the author’s imagination or used in a fictitious manner. Any resemblance to actual persons, living or dead, or actual events is purely coincidental.
About: Anna Fiofilova
Anna Fiofilova grew up in the Ukraine and from early on developed a keen interest in mathematics and was lucky enough to learn basic programming at school. This resulted in her enrolling to study Computer Systems at Networks at her local university and completing a Masters degree. Anna moved to Melbourne in 2012 (just before the Higgs Boson was discovered) and joined ThoughtWorks, where she learned about gender diversity, what ‘Agile’ really means, and had the chance to work with a great group of smart people. Anna now works with REA Group as a Lead Software Engineer.
- Samba 2020: Why are we still in the 1980s for authentication? - Andrew Bartlett
As a developer of Samba's authentication and domain control features since 2001, Andrew Bartlett will ponder how 1980s network authentication technology is faring in Samba and consider the future for on-site authentication and authorization as well as the ever more important cloud/federated identity space.
This will in turn inform a bit of crystal ball gazing on where Samba needs to go, to stay relevant, doing more than just being a great AD DC following the protocol pattern from Microsoft.
About: Andrew Bartlett
Andrew Bartlett is a member of the Samba Team and leads the Samba development team at Catalyst.
Andrew is a lead developer of the Samba AD DC project.
- Tyranny of the Clock - Craig Miskell
A war story from the front-lines solving a problem with git-over-ssh on GitLab.com. I'll take you all the way from the first reports of errors, through diagnosis and experimentation, to resolution, along with a quick look at the some elements of the bigger picture.
Includes some concrete lessons learned, from the technical to the social. Also graphs.
About: Craig Miskell
Craig is a life-long nerd, long-time FOSS supporter, and an unrepentant Linux Systems Administrator, with a penchant for solving deep and complicated technical problems that resist the easy methods. He is currently an SRE for GitLab, part of the team responsible for the uptime and stability of gitlab.com, and lives in Mosgiel, New Zealand.
- Configuration Is (riskier than?) Code - Jamie Wilkinson
TL;DR: Configuration is code, and config changes should be treated with at least as much care, skepticism, and rigour as code changes are. Config presents special challenges though as it's usually not a fully operational Turing equivalent language, but has a high "force multiplier" per character relative to code itself. let's explore those challenges and how we can address them to reduce the risk of configuration-change-related outages.
Over ten years ago Puppet Labs and others espoused the idea of "configuration as code," setting a course that crossed DevOps, the APIfication of systems, the Cloud, and Serverless. Today, you can write a few lines of config and invoke thousands of CPUs, doing hundreds of operations, deploying entire clusters of systems, a huge force multiplier for IT operations.
This force multiplier comes at a cost, and that cost is risk and impact. Never before has it been so easy to destroy an entire CDN in a single command. While numbers vary, studies show that a significant number of incidents in IT operations are caused by configuration changes.
Configuration *is* code (and I'll prove it), but it sure lacks the same rigour that code receives. Configuration formats like YAML and JSON do not have the same quality of syntax checkers and debuggers that languages like C++, Go, and Ruby have. Often the first time you know that a configuration is semantically correct is when it is running in production.
So what can we do about it? Why does this presenter think that a comparison between configuration format and a debugger is even possible?
In this presentation we'll start by looking at this problem from a theoretical point, which will let us look to other areas that solve a similar problem, and then see how we can apply that perspective back to configuration to make future production changes safer than today.
About : Jamie Wilkinson
Jamie Wilkinson is a site reliability engineer at Google. He’s a contributing author to the SRE Book and has presented on contemporary topics at prominent conferences such as Linux.conf.au, Monitorama, PuppetConf, Velocity, and SRECon. His interests began in monitoring and the automation of small installations and have continued with human factors in automation and systems maintenance on large systems. Despite his more than 15 years in the industry, he’s still trying to automate himself out of a job.
- Easy Geo-Redundant Handover + Failover with MARS + systemd - Thomas Schoebel-Theuer
The talk describes a simple setup of long-distance replication with minimum effort. The new systemd interface of MARS will drastically reduce your effort to make your existing complex solution geo-redundant.
Geo-redundancy / mass data replication over long distances is now much easier to manage for sysadmins. Although systemd has some shortcomings and earns some criticism, it can ease your automation of handover / failover when combined with the new unit-file template generator from the long-distance data replication component MARS. It is very flexible, supporting arbitrary application stacks, e.g. virtual machines, containers, and much more. MARS is used by 1&1 IONOS for geo-redundancy of thousands of LXC containers, and on several petabytes of data, with very low cost.
About : Thomas Schoebel-Theuer
Thomas is an old-school Linux kernel hacker, contributor of the dentry cache. Currently, he is working on the long-distance asynchonous replication MARS, which is in use at 1&1 Ionos for some petabytes of data, spread over thousands of geo-redundant hypervisors.
- Why Linux Systems Administrators Should Care About the Mainframe - Elizabeth K. Joseph
Mainframes still exist? In fact, more mainframes were sold in 2018 than in 1990! And Linux has run on them since 1998. In this talk, I’ll quickly cover what a modern mainframe is, and why Linux Systems Administrators may want to pay attention to them, from fascinating hardware to job opportunities.
About : Elizabeth K. Joseph
Elizabeth is a long time open source contributor, having worked on Debian, Ubuntu, OpenStack, Apache Mesos, and various smaller projects over the years. She spent over a decade as a Linux Systems Administrator before transitioning into a Developer Advocate role, a title she now holds at IBM.
- SRE for Smaller Organisations - Allan Shone
The Google Site Reliability Engineering (SRE) book is a wonderful collection of ideas and practices that ultimately are great if you work at Google. Like most people, however, you do not, so you need a slightly different set of ideas and principles to work by. Within the bounds of a smaller business, how can the principles put forward by the SRE moniker be applied on a much smaller scale? A handful of guiding principles can be taken, for instance, to give you, your team, and your business, a much easier time for achieving your goals.
About : Allan Shone
Allan is a long-time Linux enthusiast and user across many devices and platforms, using Linux variants since some time in the 90s. Whilst his career began with Software Development, it's progressed through various disciplines including security, quality, infrastructure, automation, and systems. Allan's focus has been influenced by his time in various businesses, coming to the realisation that most challenges we all face really aren't technical in nature.
- Bash-my-AWS: CLI commands for managing AWS resources - Mike Bailey
If you use Amazon AWS and prefer the command line over ClickOps(TM), this talk might just make your week.
For decades, the command line and shell scripts have been a core part of unix culture and preferred tools for many sysadmins. The existing unix commands that come with any linux/bsd distro, along with streams/pipelining, plain text and a lack of mouseclicks are just a few of the attractions.
In recent years, ClickOps seems to be on the rise - largely due to enterprise web applications with suboptimal (or absent) CLI tools. Many of us have been spending more time in the AWS Web Console than we would like for this very reason.
Bash-my-AWS is a simple but extremely powerful set of CLI commands for managing resources on Amazon Web Services. They harness the power of Amazon's AWSCLI, while abstracting away the verbosity. The project implements some innovative patterns but (arguably) remains simple, beautiful, readable and easily extensible.
By emitting line oriented output instead of JSON, Bash-my-AWS commands work well in pipelines with other standard unix commands. Additionally, the commands take a novel approach to handing STDIN that greatly improves the user experience.
The talk will introduce the project and describe it's novel use of pipelining to provide improve the user experience.
Website: https://bash-my-aws.org
Git: Ma href="https://github.com/bash-my-aws/bash-my-aws">https://github.com/bash-my-aws/bash-my-awsAbout : Mike Bailey
Mike is a systems engineer with an eye for detail and a passion for unix and open source.
Mike created `deprec - deployment recipes for capistrano` in 2006 to simplify server provisioning and deployment of Ruby on Rails applications. While very popular at the time, the mass migration to the Cloud called for different approaches and tools.
In 2014 Mike created the bash-my-aws CLI tools to make managing resources in Amazon AWS a fluid and graceful experience. The project takes the unusual approach of using simple bash functions to provide the commands, while keeping them clean, short and readable. The past five years has seen collaboration with some key contributors that has make the project really, really cool.
- Using a cloud to manage a cloud - Alex Sharp and Andrew Reimers
QubesOS is a Xen based desktop operating system that is a user friendly frontend to a powerful backend, managing hypervisor backed containers called 'Qubes'. OrionVM is a wholesale cloud infrastructure provider, and it has been using QubesOS to help make its sysadmin easier. This talk is partly an overview of QubesOS and how it works, and partly an overview of how we used it, what we found useful and what still needs to be worked on.
About : Alex Sharp
Alex is one of the cofounders of OrionVM - A high performance cloud computing startup, using supercomputing technology to provide faster, more efficient cloud.
He's also the upstream maintainer for the potsdb OpenTSDB client library and the GELF library for Graylog, and a proud member of the Mars society.
About : Andrew Reimers
Andrew is a low level software programmer who likes to tinker with hardware design.
He is a UNSW computer engineering graduate.
Andrew also has bit a passion for security related topics.
- Don’t eat my data - 30+ years of storage war stories - Steven Ellis
We are in an era of explosive data growth, with everything from cat pictures to NASA’s incredible archive of space photography, and this all needs to be stored and catalogued in a reliable and easily accessible form. How can we safely store/backup/manage this data across a diverse range personal devices, never mind the challenges of enterprise class scale out storage, with the occasional (rain) cloud along the way?
We’ll start off with a bit of history on data storage formats to set the scene, and then dig into some stories from the last 35+ years of Steven’s own journey in IT, with some names changed to protect the “innocent”.
Themes covered will include
- Getting physical
- Bits, Bytes, Blocks and Blobs, with a file or three
- Performance sanity tests
- Hardware Raid - pain or pleasure?
- File-system selection and why checksums matter
- Going mad with MDADM (part 2)
- DRBDont
- HA NFS aren't the droids you're looking for
- Vacuum Cleaners and Dehumidifiers
- Raid is not a backup, and erasure coding isn’t about data deletion
- The cloud is just someone else’s SAN
- When physical disks trump flash storage
About : Steven Ellis
Steve's is an Open Source Technology Evangelist in the APAC Office of Technology team at Red Hat. In the last 25+ years he worked as a developer and transitioned to an infrastructure and operations architect across a broad range of Unix and Linux technologies. For most of that period he’s used Open Source technologies to solve business problems, and on many occasions recover lost data. He freely admits that there have been occasions of data loss, and recovery during that career and really hopes some of the session attendees can avoid his past mis-steps.
In his spare time he still hacks on the MythTV project and debugs Open Source on random bits of hardware that really should know better.
- AI/ML won't save you - Julien Goodwin
Lightning talk - Details to come
About : Julien Goodwin
Julien is a Site Reliability Engineer working on some of the largest networks on the planet. He has spoken many times at the LCA SysAdmin miniconf, as well as at OSDC and other conferences in Australia, New Zealand & the United States. He was part of the organising team for linux.conf.au 2008 in Melbourne.
- Highly available DNS recursion with PowerDNS - Dave Kempe
Lightning talk - A quick tour through a little HA powerdns DNS cluster I built for a major media platform. Batteries included
About : Dave Kempe
Dave founded Open Source IT services company Sol1 in 1999. Day to day along with running a thriving IT services company, he builds Open Source infrastructure solutions for customers both large and small. With plenty of war stories to share and experience to give, he is keen to offer some advice and knowledge to new users and veterans alike.