## Build pipelines, Python and You
Wes Mason
[@1stvamp](http://twitter.com/1stvamp)
## Who?
- @1stvamp (twitter/github)
- Engineer at ServerDensity
- I release lots of open source code
###"a sub-discipline in software engineering concerned with the compilation, assembly, and delivery of source code into finished products or other software components."
## But builds are for Games and kernels
## Yeah?
```
./env/bin/activate
pip install -r requirements.txt
```
## Continuous integration makes my builds reliable
## My configuration management system deploys everything
## Versions are irrelevant because I deploy continuously
## Our devs know how it's built
## Our ops know how it's shipped
## Our QA dept know how it's tested
## ...and do any of them know *why*?
## My build broke, but why?
## Document, document, document
## Okay, so what's a build pipeline?
{Code change} / {Release}
→
Build steps
→
Package
→
Asset
→
Store
→
Deploy?
### Build steps
- Build server
- Continuous integration
- Buildbot, Jenkins, Travis
### Buildbot
- Built by the Twisted guys
- Continuous builds
- Polling / events (e.g. svn-hooks, git-hooks, Github webhooks etc.)
- Slaves can be VMs, actual machines, local, LXCs etc.
- Configured via Python
### Travis CI
- Free for open source (paid version too)
- Configured via `.travis.yml`
- travis-solo
- buildbot-travis
### Jenkins
- Arse to setup (but so can buildbot)
- ShiningPanda plugin - Python builds win
### Automating your build
- `make install`
- No really..
### Buildout
- Created by the zope/plone guys
- Configured via INI files
- Extended via Python
- 1.x includes virtualenv like abilities
### Buildout cont.
- Can create self contained "packages"
- Install from pretty much anywhere with recipes like `buildouthttp`
- Non-python package builds
- Wrap pretty much anything
- `dumppickedversions`
### fabric
- Automation and deployment tool
- Used for deploying things via SSH
- Can be used for builds steps
- `fabtools` makes this easier
```
from fabric.api import *
from fabtools import require
import fabtools
@task
def setup():
# Require some Debian/Ubuntu packages
require.deb.packages([
'imagemagick',
'libxml2-dev',
])
# Require a Python package
with fabtools.python.virtualenv('/home/myuser/env'):
require.python.package('pyramid')
```
### Virtualenvs
- pip is not a build tool
- (...but it can be, via `build` hooks in setuptools, distutils et al)
- `pip --list --outdated`
- ..also see `yolk`
### ..wrapping everything up..
- `tox` - multiple builds again different envs
- `dh-virtualenv`
- `virtualenvwrapper`
- `pythonbrew`
- `pyenv`
### Rockstack
```
rock build
rock test
rock run
```
### "Golden image" vs "golden package" vs "just rerun all the things"
- Build whole VM images
- ..or LXCs (`docker`)
- Build OS packages (`dh-virtualenv`)
- Build eggs, wheels, tar's
- ..or just let your CM handle it
## sys.exit(0)
- Slides: [github.com/1stvamp/pysheff-build-talk](https://github.com/1stvamp/pysheff-build-talk)
## ^D
- Jonathan "Joff" Oliver: [about.me/joffie](http://about.me/joffie)
- Grab me and say hi `:-)`
