Why I built this site
25/04/2019
I give in to a long-held desire to build my own site, and ramble about why and how I built it.
Fixing a Jupyter kernel connection issue
14/03/2019
Fixing Jupyter kernel connection issues caused by dependencies not doing what they should.
Launching Sublime Text from the Command Line
18/02/2019
A quick how-to on getting the command-line launcher for Sublime text hooked up and working in your shell
What is Data Engineering?
10/12/2018
An overview of Data Engineering - what is it, how does it work, and an overview of some of the main frameworks used.
MVP isn't just a buzzword
01/08/2018
Despite my best intentions, I over-engineered a bunch of stuff, rather than talking to the actual users, and ended up wasting a bunch of my time.
Python Mutable Defaults
31/05/2018
I found out why you shouldn't use mutable objects as default arguments in functions, the hard way.
Connecting services behind IAP on App Engine
26/03/2018
Some of the trials and tribulations involved in getting IAP working on Google App Engine (especially when you don't have an extensive full-stack background).
Pickling Problems
29/01/2018
Debugging pickling problems in Spark, thrown up when migrating from Python 2.x to 3
Streaming job logs from Cloud DataProc
08/01/2018
Streaming logs in near real-time from Cloud DataProc jobs as they run, straight to the command line.
Bermann
17/11/2017
After discovering how hard it was to unit-test Spark at any sort of scale, I made a unit-testing framework for Spark RDDs called Bermann.
Cancelling Frozen Shells
28/10/2017
After getting frustrated with SSH sessions freezing if the underlying connection dropped, I dug in further, and discovered a whole extra set of escape commands for SSH sessions.
<3 Python Imports
26/10/2017
Investigating some shenanigans involving relative imports and import ordering in Python modules.
py-dataproc
21/10/2017
I used DataProc extensively, and whilst I loved the service, I hated the auto-generated APIs for Python. So I wrote my own (with blackjack etc).
Overenthusiastic CHOWNership
17/10/2017
I ran sudo chown -R /usr/bin and you wouldn't believe what happened next!