On Versions

Versions are dead, long live versions

What version of Chrome are you using? Beyond the major version number, what version of your operating system are you on? If you deploy using Linux code, what version is your Linux Kernel?

My answer to those questions: I don’t know. Or didn’t. I just checked and I’m on version 42.0.2311.39 beta for Chrome, 10.10.2 for OS X, and 3.16.7-tinycore64 for my Docker VM I use for testing images. My life isn’t better for knowing that information, though.

The same is true for most of the software you create. The version number doesn’t matter, but to this day software developers don’t want to mark their software as version 1.0. 1.0 carries a lot of weight. To a lot of developer’s it means you’re done. It means you’re confident in it. It means things aren’t going to drastically change.

The Python community is afraid of 1.0. The only reason I can understand why is because it’s the largest case of collective imposter’s syndrome I’ve ever seen.

Don’t believe me? There are 61,564 Python packages that have been released according to this page. Of those, 40,489 have a version number that begins with 0. That’s two-thirds of the packages that I can’t tell anything from those version numbers.

For example, is virtual-touchpad more stable than Werkzeug? The former is at version 0.11 while the latter is only at 0.10.1. Of course, Werkzeug is almost certainly more stable. The download numbers seem to tell me that with it’s more than 20,000 downloads in the last day. Werkzeug runs a huge chunk of the web that’s powered by Python. Flask doesn’t exist without it.

Statements like the one in the previous paragraph that begin with “of course”, however, are only obvious with the correct reference. If you’re coming from world outside of the Python community, you don’t have that reference.

Sane Versions

Enter Sematic Versions. It can be described in a tweet, but here’s the slightly more expanded version.

  • Versions begin at 0.x. Anything in 0.x hasn’t been deployed anywhere and you’re still turning it into something useful. You make no guarantees about it.
  • The first code that’s used in production is 1.0.0. Production means it’s being used and not just written.
  • Versions follow Major.Minor.Bugfix.
  • Major version numbers are for backward compatibility. If this number changes, the API has changed and it means that code written against the old version won’t work with the new version in at least one case.
  • Minor versions are for new features. Nothing should break between versions 1.0.0 and 1.1.0 or 1.101.0.
  • Bugfix versions are for bugfixes. No new features are added here, just corrections to code to make sure it does what it’s supposed to.

It’s really that simple. When I install your software package at version 1.2.0 I know that I can run anything before version 2.0.0 and it should all continue to work.

There are some devils hiding in the details. For example, how many back versions do you support? If you find a bug in version 1.3.0 that was present all the way back to 1.0.0, do you patch versions 1.0.x, 1.1.x, and 1.2.x as well? Does each new feature mean a minor version bump?

That’s up to you as a maintainer. There are no right answers to those questions: the main point is to make sure that code that works in one release doesn’t break in the next. If it does, and sometimes it needs to, bump the major version number.

Also, it’s ok to break. SemVer gives you the opportunity to convey to the users of your code that something needed change in ways that weren’t compatible with the previous code.

To the Python Community

Please consider adopting SemVer. What’s stopping you? Is it because you don’t think your code is ready to be called 1.0? I promise you, it is. It’s actually awesome!

All I want is for you to quit worrying about getting it perfect. Get it close to right, make it so people can use it. Then release it. If you get something wrong or need to fundamentally change the API, do it, but bump the major version number so everyone knows at what point their code might not just work™.

Software is just that: soft. It can, and should change. Don’t be afraid of v1.0 or v2.0 or v20.0.

Looking Toward the Hub

This past fall a (new) good friend offered to marry Brandi and I as we traveled to Terlingua to share our vows with each other, our families, and close friends. As Sharron prepared, she asked for a favorite author or two of each of ours so she could find a quote to use at the ceremony.

There are few things that will make you question your reading than to be marrying a professional writer and being asked who your favorite author is. I read a ton, but have had very few authors who are my go-to when looking for inspiration. I’m also horrible with specifics. I remember general themes, but things like names don’t stick with me. Since I drew a blank on inspiring writers, I went with my gut: Terry Pratchett.

Regardless of the where I’ve been in life the past handful of years after discovering him, I’ve reached for Terry Pratchett’s books as my release of the previous day’s activities. It’s been the thing that lets the energy expended or pent up during the day relax into a soothing sleep. His humor and view on the world is calming.

I told Sharron that Pratchett was my favorite author, not expecting her to find much of anything. His humor is great, I knew that. But something that would fit in a wedding? That’s a different story.

The day before the wedding, we arrived and she told us what she had found:

Why do you go away? So that you can come back. So that you can see the place you came from with new eyes and extra colors. And the people there see you differently, too. Coming back to where you started is not the same as never leaving.

Emphasis mine.

Having just left Austin, having just left my family and friends, having grown up as a rolling stone, and having returned to a place dear to my heart for this special occassion, this quote carried special meaning for me.

It’s been with me ever since, and even more so this last 24 hours. #RIPTerryPratchett

Python Patterns: Kwargs Helper Method

Writing usable, functioning code can be hard enough. Now imagine writing code that you need to make extensible enough that other developers can extend without simply copy-n-pasting your source code and making their own modifications. That can be rough. There are some patterns that you occasionally find in frameworks like Django, however, that I haven’t seen documented. This morning, I contributed a bugfix to werkzeug based on a pattern I’ve seen before. I’m calling it the kwargs helper method.


You have a method that returns an object or the result of a function, both of which are variable. Through other parts of your code, other developers can change what your function will return. Examples of this include Django’s ListView.get_queryset, and Werkzeug’s Rule.empty (as of v0.10).

You need to allow other developers to control what gets passed into the objects and functions as they’re called. Without such a mechanism, developers are forced to override the entire method and in the worse case re-implement part of your code. I want you to stop that.

Example of Problem Code

Here’s a contrived example of the code in question.

class Sheep(object):
    def __init__(self, name=Dolly):
        self.name = name

    def clone(self):
        return type(self)(name=self.name)

Note the type(self) call here. That returns Sheep in this example, but returns whatever type the subclass is. So when we create a BionicSheep like the one below, we have a problem:

class BionicSheep(Sheep):
    def __init__(self, turbo_legs=None, **kwargs):
        self.turbo_legs = turbo_legs
        super(BionicSheep, self).__init__(**kwargs)

    # what do do about cloning?

At this point, BionicSheep is broken if you try to clone it. The clone method won’t pass in the turbo_legs value. You now have two options: copy-n-paste the whole clone method to remove Sheep.clone from the equation entirely or call super to get the result of Sheep.clone, then add your own values and duplicate the assignment in __init__. The latter option isn’t horrible in this case, but if __init__ provided different functionality based on that kwarg you would be forced to copy-n-paste clone and provide your own duplicate implementation.


The solution is to provide a helper method that provides the kwargs outside of the actual function. I’m calling this pattern the kwargs helper method. This provides a granular hook for other developers to change the arguments that are provided without having to override the main method and possibly duplicate code.

You need to modify the Sheep.clone method to work like this to use this pattern:

class Sheep(object):
    def __init__(self, name=Dolly):
        self.name = name

    def clone(self):
        return type(self)(**self.get_clone_kwargs())

    def get_clone_kwargs(self):
        return {name: self.name}

Now you have a nice hook for providing your own custom kwargs in subclasses and nobody has to touch clone. Here’s an implementation of BionicSheep that works with the new code:

class BionicSheep(Sheep):
    def __init__(self, turbo_legs=None, **kwargs):
        self.turbo_legs = turbo_legs
        super(BionicSheep, self).__init__(**kwargs)

    def get_clone_kwargs(self):
        kwargs = super(BionicSheep, self).get_clone_kwargs()
        kwargs[turbo_legs] = self.turbo_legs
        return kwargs


I started by saying that writing functioning code is hard. Making sure your code is extensible for every other developer to use is ten times harder. Think not only about what each line of code in your codebase does, but also how it’s used and extended. I promise, some developer somewhere is going to want to change just about every line of your code. Be nice and make it easy on them.

Design Thinking vs Development Thinking

This morning I read an article on what the ideal operating system should look like. I devoured all all three parts and it got me thinking about my thought process and how I approach development. This post is a loose collection of those thoughts.

What Problem?

One thing that I’ve discovered about my thought process is how I approach problems. Too many times, it’s easiest to start from where I am right now and how I can modify the existing tool / code / product to do what I need. This provides a good starting point for context of what’s immediately possible, but not for solving the problem.

For example, let’s consider the text editor. The main purpose of a text editor is writing things down. You want to be extremely good at that if you’re going to be an editor that people want to use. Based on this description you can build an editor that’s a joy to use and makes the process of getting information into the editor easy and intuitive. There’s a problem with it: what happens when a user is done with new document that they’ve created? My original description did not include anything about saving or exporting the documents that are created.

Realizing that you’ve left saving out as a feature, you might write up a job story that looks something like this:

When writing a story I want to ensure that it’s been saved so that I can share the saved document with other people.

If you start from where you are, you might think to add a Save feature and tie that to a menu item, a keyboard shortcut, and maybe even a toolbar to provide multiple options to your user. This is a valid concern, but it overlooks one key thing. The user doesn’t care about saving, they just want it saved.

The user’s job is to write, not to save something. Explicitly saving something is a task. User’s aren’t interested in performing a task unless they have to. Auto-save is what the user needs. At this point in the process the only thing they need to know is that their work is saved. Instead of focusing on the job at hand and how this feature supports that job, adding a Save feature focuses on the task.

I’ve fallen victim to thinking that focuses on the task instead of focusing on the overall job, but I guard against it now. This causes me to think differently than a lot of developers: rather than focus on fixing one particular thing, I focus on what the underlying (or overarching?) problem or job is. This means I talk past people sometimes because I forget that we’re talking about different things.

How to fix a problem

On a recent open source project that I work on I opened a pull request that introduces a new higher level concept to the project in the service of fixing one discrete bug. To me, the discrete bug was a manifestation of the lack of that higher level structure. Without that common vocabulary, different parts of the code were touched by different developers at different times and there was a discrepancy between how the concept was represented.

To me, that larger problem was what needed fixing. To other developers, the bug needed fixing. Thinking about that larger problem, I tackled that and fixed the bug. Another developer on the project focused on the explicit problem and added the one-line fix to that code path that solved that one bug that manifested itself. On the surface, the one-line fix seems simpler because less code was involved (my fix was a little more than 30 lines). The one-line solution was only simpler when viewed as the task “fix this bug” not “fix the problem that gave rise to this bug.”

To be fair, both are legitimate ways to approach the problem. The one-line fix that focuses on the task at hand fixes the bug and avoids possible over-engineering that might happen by thinking about the bigger picture. It also runs the risk of having the same problem solved in different ways throughout the code base as each “just one-line” fix adds another branch into the complexity of the program.

Thinking like a developer vs like a designer

This all ties back to the story that started this post because of the way the problem was approached. Most developers I know would balk at the idea of creating an operating system, then starting by removing the file system and applications. “But where will I store my files and how will access them?!” I hear them all exclaim at once. Most designers I know would hear that idea, think for a second, then say “ok, so what replaces it?” followed closely by “and what was the user trying to do when they accessed those files?”

Designers tend to think in terms of solutions to general problems. Developers tend to think in terms of solutions to explicit problems. This is still a nascent revelation to me, but starts to explain to me while I’ve always felt slightly out of place in the development world.

It’s also making me question my description: am I still a developer with a bit of design knowledge or a designer that happens to program?

Rethinking Web Frameworks in Python

Listening to @pragdave talk about Exlir’s pipes he was talking about how these two styles, while fundamentally the same, have vastly different readability:


Try to explain that line of code to someone who doesn’t program. You start by telling them to just skip over everything until they hit the center, that’s the starting point. Then, you work you way back out, with each new function adding one more layer of functionality.

As programmers, we’ve taught ourselves how to read that way, but it isn’t natural. Consider this pseudo code:

"cat" | list | sorted | join

This code requires that you simply explain what | does, then it goes naturally from one step to the next to the next and the final result should be the joined sorted string.

Seeing that code example got me thinking about some of the discussions I’ve had with new programmers as I explain how Django works. I start explaining the view, to which I’m almost always asked “ok, how does the request know what view to execute?” I follow this up by moving over to URL route configuration. After that’s explained, I’m asked “ok, so how do requests come in and get passed through that?” And this goes on, until we’re standing on top of 20 turtles looking down at the simple Hello World we wrote.

In that vein, what would a web framework look like that started with the premise that a regular, non-programmer should be able to read it. Here’s an idea:

def application(request):
    request > get("/") > do_response()
    request > get("/msg") > say_hello()

So, you define an application function that takes a request, that request is then run through a get function with a route, and if that matches it would finally pass off to a final function that does something that would generate the response.

To that end, I’ve hacked up this simple script that uses werkzeug to do a simple dispatch. The implementation is a little odd and would need to be cleaned up to actually be useful, but I think I could be on to something. Just imagine this syntax:

request > get("/admin") > require_login > display_admin()

At this point, require_login can return early if you’re not logged, and display_admin could repeat the entire application style and be “mounted” on top of the /admin route and respond to request.path that is slightly different.

request > get("/users/") > display_user_list()
request > get("/user/<id>/") > display_user()
request > post("/user/<id>/") > edit_user()
# or...
request > route("/user/<id>/", methods=["GET", "POST"]) > handle_user()

Any thoughts?

My First Docker

I’ve been told I should check out Docker for over a year. Chris Chang and Noah Seger at the Tribune were both big proponents. They got excited enough I always felt like I was missing something since I didn’t get it, but I haven’t had the time to really dig into it until the last few weeks.

After my initial glance at it, I couldn’t see how it was better/different than using Vagrant and a virtual machine. Over the last few weeks I’ve started dipping my toes in the Docker waters and now I’m starting to understand what the big deal is about.

Docker versus VM

I’ve been a longtime fan of Vagrant as a way to quickly orchestrate virtual machines. That fits my brain. It’s a server that’s run like any other box, just not on existing hardware. Docker goes a different route by being more about applications, regardless of the underlying OS. For example, let’s talk about my npm-cache.

Using this blog post as a base, I wanted to create an easily deployable nginx instance that would serve as a cache for npmjs.org. The normal route for this is to get nginx installed on a server and set it up with the right configuration. You could also add it to an existing nginx server if you have one running.

Docker views something like this npm-cache less as the pieces of that infrastructure (nginx and the server its on) and more as an application unto itself with an endpoint that you need to hit. Its a subtle shift, but important in a service-oriented world.

Getting Started

Docker has been described as Git for deployment, and there’s a reason. Each step of a deployment is a commit unto itself that can be shared and re-orchestrated into something bigger. For example, to start my npm-cache, I started by using the official nginx container.

The nginx container can be configured by extending it and providing your own configuration. I used in the configuration from yammer, created a few empty directories that are needed for the cache to work, then I was almost ready to go. The configuration needed to know how to handle rewriting the responses to point to the caching server.

Parameterizing a Container

This is where things got a little tricky for me as a Docker newbie. nginx rewrites the responses from npm and replaces registry.npmjs.org with your own host information. Starting the container I would know that information, but inside the running container, where the information was needed, I wouldn’t know unless I had a way to pass it in.

I managed this by creating a simple script called runner that checks for two environment variables to be passed in: the required PORT and the optional HOST value. HOST is optional because I know what it is for boot2docker (what I use locally). PORT is required because you have to tell Docker to bind to a specific port so you can control what nginx uses.

My runner script outputs information about whether those values are available, exiting if PORT isn’t, modifies the /etc/nginx.conf file, then starts nginx. The whole thing is less than 20 lines of code and could probably be made shorter.

Deploying with Docker

I got all of this running locally, but then the thought occurred to me that this shouldn’t be that hard to get running in the cloud. We use Digital Ocean a lot at Continuum, so I decided to see what support they have for Docker out-of-the-box. Turns out, you can launch a server with Docker already configured and ready to run.

With that, deploying is ridiculously easy. I started a small box with Docker installed, then used ssh to connect to the box, and ran the following commands:

docker pull tswicegood/npm-cache
export PORT=8080
docker run -d -e HOST=<my server's IP> -e PORT=$PORT -p $PORT:80 tswicegood/npm-cache

That’s it! Including network IO downloading the npm-cache, I spent less than five minutes from start to finish to get this deployed on a remote server. The best part, I can now use that server to deploy other infrastructure too!


Making deployment of a piece of infrastructure this easy is not a simple problem. I’m sure there are all sorts of edge cases that I haven’t hit yet, but kudos to the Docker team for making this so easy.

Check out Docker if you haven’t. The Getting Started tutorial is really great.

Timeless Way of Coding

… we must begin by understanding that every place is given its character by certain patterns of events that keep on happening there.

The above quote is in the opening chapter of one of my favorite books of all time, The Timeless Way of Building by Christopher Alexander. Alexander is famed in programming circles as the author of A Pattern Language which set the stage for programming design patterns some 40 years before the Gang of Four wrote the book.

The Timeless Way is the lesser known of his two-volume set. It sets up his pattern book by defining why patterns are important. It is a more thorough explanation of quality than Zen and the Art of Motorcycle Maintenance without the personal account of a descent into madness and a focus on quality through the lens of architecture and places. It is on my list of must read books for anyone who takes themselves seriously as a programmer.

If you’ve ever had one of my code reviews, you’ve probably seen something like this:

All functions need two \n characters between them

Or this gem:

Syntax of 'key' : 'value' in dictionaries will raise a flag on pyflakes. Best to avoid.

Both of these are from a commit message this past week with some simple cleanup, code gardening if you will, on code. My change didn’t affect what the code did at all, but it did make sure that it was more idiomatic Python. Pythonistas pride themselves on a certain style so much that there is even a coined term for this: Pythonic.

The importance of these small changes is summed up in the opening quote from this post. To paraphrase:

Things keep happening the way they happen.

By focusing on producing clean, readable, simple, uncomplicated code, you create an environment where more clean, readable, simple, uncomplicated code can flourish.

Tools I Use

You can stop here if you’re not interested in specific tools, otherwise, here are a few things I use to help keep my code clean.

The editor I use the majority of the time is Sublime Text 3 (though I will always have a soft spot in my heart for Vim). I start with these language-specific settings in Python, which you can use by opening a .py file, then going to Sublime Text 3 > Preferences > Settings - More > Syntax Specific - User and copying this JSON blob into that file.

    "detect_indention": false,
    "tab_size": 4,
    "translate_tabs_to_space": true,
    "use_tab_stops": true

Beyond some basic settings that cause spaces instead of tabs to be used and setting the tab size correctly, the most important part of those settings is the rulers. There are two lines that are displayed at character 72 and 80 in every Python file I open.

Docblock comments in Python are supposed to be less than 72 characters. This allows the docblock to be displayed indented in Python’s built-in help and not wrap to the next line. I try hard to ensure all docblocks I write stop before I hit that mark. The second line at 80 characters shows the point where my Python code needs to stop.

I know many developers think that the 80-character limit is too limiting. “I have a big monitor” I hear you say. The optimal character length for a line of text is around 60 characters. Going much beyond that makes it harder for the human brain to process what it’s seeing without scanning back and forth. Plus, take your code and increase it so someone at a meet-up can see your code sitting 20 feet away from the screen, then see how your 120 characters look.

There’s an even more practical consideration when thinking about line length. Forcing this constraint on yourself causes you to think really hard about what is the most effective use of those characters. Is that line really best expressed with an 80 character string in the middle, or can that be hidden behind a variable? Do all of those and conditions in your if statement make your code more readable, or would an intent-revealing function help this code? Constraints, even annoying ones, can really help hone your code design skills.

Next up, I use the Python Flake8 Lint. This tool scans your code using pyflakes and flags errors for you. Out of the box, it can be a little annoying (especially when you’re learning pep8’s rules). It displays a pop-up when you save your file and tells you all the places your code has errors. This is really useful on your own projects, as it causes you to pay attention to make sure that your code doesn’t raise these errors. But when you’re working with other developer’s code, you might want to reduce the chattiness. You can tweak the settings under Preferences > Package Settings > Python Flake8 Lint > Settings - User. Here are the settings I use for it:

    // run flake8 lint on file saving
    "lint_on_save": true,
    // run flake8 lint on file loading
    "lint_on_load": true,

    // popup a dialog of detected conditions?
    "popup": false,

    // show a mark in the gutter on all lines with errors/warnings:
    // - "dot", "circle" or "bookmark" to show marks
    // - "" (empty string) to do not show marks
    "gutter_marks": "bookmark",

This adds a mark to the gutter on each line that has an error, suppresses the popup, and makes sure that pyflakes is run when I open a file so I can see the errors immediately. To see the actual error, I move my cursor to a line that’s marked and this plugin displays the error message in the status bar.

These might seem like draconian tools that get in the way of coding quickly. Coding fast and coding sloppy are not synonymous. Spend a little time working within these constraints and your fellow developers will thank you.

Plus, you’ll be making sure that the code you write helps to create a better codebase by increasing the quality of the patterns that keep happening there.

The Case for Django

I get asked a lot where to start if you’re looking to python for web backed work. A lot of people look at Django and Flask and feel that Flask is where they should start. It’s nice and small, very simple, and after all they’re not doing anything big and complicated, so why start with a big, complicated framework?

This reminds me if something that happens in the running world. People get started running then either a) read Born to Run, or b) hear someone talking about the benefits of so-called barefoot running. (For the record, I’ve only seen a few people actually run barefoot. Most run with minimalist shoes like Vibram FiveFingers™.)

There are many benefits to running with minimal shoes. Proponents point to studies that show lower injury rates amongst bare footers. They talk about our natural instinct to run and how the modern shoe with all of its support and cushioning is actually doing more harm than good.

The next part of their pitch is ignored by many of the so-called Born-to-Runners: it takes a lot of practice to be able to get to the point where you can run 10k, much less an ultra-marathon with minimal shoes. You practically have to start over and slowly build. There is a huge payoff, but it takes time. Otherwise, you’re more likely to injure yourself.

I’m speaking from experience. I didn’t read Born to Run, but I know the claims. When I started running a few years ago, I switched on and off from a minimal pair of running shoes and a pair of FiveFingers™. I figured since I was just starting out I wouldn’t have any bad habits to break.

There was one snag in my plan: I wasn’t ready for them. I hadn’t built up the running specific muscles. My form wasn’t there yet. I quickly started having plantar fasciitis issues. They weren’t debilitating, but enough to make me take a week off to rest and work on stretching. It flared right back up as soon as I started running again. I had a half marathon a few months out so something had to give. A trip to the running store and about $100 later I had a pair of running shoes that felt like pillows on my feet and a week later the pain was completely gone.

The same thing applies to web frameworks. It might seem like a good idea to stick with frameworks that can be coded in one file, or ones that don’t do everything. Those frameworks are built on top of a lot of hard won lessons.

When you’re starting out, you don’t know what a properly factored web application looks like (yet). You don’t know where to draw the line between your model and controller layers (yet). You don’t really know the trade-offs involved in going with a relational database and a NoSQL database. And that’s ok. Micro frameworks assume you do, though. They give you a lot (or a little, depending on how you look at it) of rope and it’s really easy to end up with your app looking an awful lot like a noose.

So skip the minimalist when starting out, whether that’s shoes or web frameworks. Build on the experience of others, then start stripping away those layers once you’ve got a solid base.

Moving On

I took over as the Director of Technology at the Texas Tribune a year ago this month, but if you’re reading this you probably already knew that. What you probably don’t know is that year ago June I had one foot out the door. So what kept me at the Tribune? A beer with Emily Ramshaw.

She reminded me why I left the start-up world in the first place: to make a difference in the world with my work. I do that at the Texas Tribune, but my role has always been a supporting one. It needed to be given my straddling the business side, the editorial side, and everything in between. I used to describe my job as not being on the front lines of changing the world, but I was the one supplying the ammo.

I’ve realized over the last few months that I wanted that to dynamic to change. It became crystal clear in the early morning hours of June 26 as I watched tools I had help put in place shed a light on the inner workings of Texas politics. Without our work, everyone would have read about the filibuster and questionable voting times the morning after instead of watching it happen live.

I’m happy to announce that as of today, I’m moving on from Director of Technology at the Texas Tribune to the editorial staff as the Tribune’s first News Apps and Data Editor. I’ll be continuing the amazing work that’s already been done and working full time with Ryan Murphy on our newly formed News Apps team.

I gotta say, I’m super stoked.

Past, Present, and Future of Armstrong

Most of you who know me have heard me talk about Armstrong, the open-source news platform that I helped create when I first joined the Texas Tribune. I have and continue to talk at length about Armstrong and its future, but I’ve never collected those thoughts into one cohesive document outlining how we got to where we are now, what the current state of the project is, and where I hope to see it go.

This post is my attempt to do that.


Armstrong is peculiar if you look at it from the outside. It might be hard to understand exactly what it is and how it got to the point it is. This section should help you understand that a bit more.

Not a CMS

Using common names can be unfortunate. Most people hear Armstrong CMS and think it’s a content management system akin to Eidos on the closed-source side or Wordpress and Drupal on the open-source side. I’ve always envisioned Armstrong as a platform to build on top of, not simply a CMS. The distinction is small, but important.

You work within a CMS

You build on top of a platform

A CMS is something you use. It provides the tools you need to manage content. A platform is something that provides a base to build upon. It’s my belief that more 95% (maybe more) of the pieces that make a news focused website a news website are all the same. Everyone needs a way to collect similar content into sections, a way to schedule content, or a way to control publication status on content. That last 5% of content, however, is unique and what makes a site interesting. Being able to reuse data about bills throughout your site without having to confine it to a big blob of text is what allows us to do interesting things with a site. This is where Armstrong comes in.

Armstrong isn’t meant to be used out-of-the-box any more than a Lego™ train set is meant to be a toy train set as soon as you unpack it. Both require assembly, but both allow you to exercise some creativity in how you assemble them. To do this, we’ve taken an unconventional approach to packaging everything in the project.

Everything is a package

Python is famous for it’s horrible packaging solutions. It’s gotten a lot better over the last few years, but people still package most software in one big bundle. I created the packaging schema for Armstrong based on the way I wished Python packages were handled, as if they followed an adopted version of the Unix Philosophy:

Packages should do one thing, and do them well. Packages should work together.

This means that there’s some 25 packages on PyPI for Armstrong. Many of these work in concert with other Armstrong packages to form a bigger whole. I broke it down along two main lines with a few others thrown in.


This section of Armstrong contains all of the pieces essential to nearly all websites. These packages either have no models, or are meant to be used almost exactly as-is with little or no modification.

All core packages contain an arm_ prefix in the last part of their name: armstrong.core.arm_content, armstrong.core.arm_wells, and so on. This is to avoid potential naming conflicts since Django still assumes that its apps are all flat without full module names.

Try as I might, armstrong.core.arm_content did end up pretty big. It contains most of the mixins used to build the larger models throughout the system. Anything that can loosely be considered content in an Armstrong project probably has some connection to this package.


These apps are meant to be usable out-of-the-box, but are most useful as example implementations. This is one area that I didn’t document as well as I should have. Almost no one would (or should, for that matter) use armstrong.apps.articles as their out-of-the-box article implementation. For very simple sites it will probably work, but my assumption has always been that most sites will take that as a guide and build something similar to it.

A great example of this is the armstrong.apps.donations project. We use that pretty extensively at the Texas Tribune since we’re a member-driven organization, but we don’t use it in its out-of-the-box configuration. We have a custom tt_donations app that extends the views to add extra functionality and we have custom backends for all of our payment processing and CRM syncing.

Hatband and Pops

Any news tool is only as good as it’s admin interface. Unfortunately, most of our time while funded by the Knight Foundation was spent on backend code, but we do have a solid start of a custom admin interface that’s broken down into two pieces.

Hatband is the collection of Armstrong-specific interfaces to the Django admin. It’s meant to be used as a drop-in replacement for django.contrib.admin that extends the behavior. It provides several custom inline interfaces and will hopefully contain even more. It exposes a JSON API for searching any type of model that’s registered with the admin and has search_fields turned on. The plan has always been to use this to create a rich API on top of the admin. Search is simply the first foray into that.

Where Hatband is behind-the-scenes, Pops is the user-interface side. It’s currently built using Twitter’s Bootstrap framework and was built on top of a fork of django-admintools-bootstrap. Pops is meant to be entirely standalone and have no ties to Armstrong at all since it’s simply the skin on the interface.

Current State

The original Knight Foundation grant to The Bay Citizen and The Texas Tribune ended in early summer of 2012. Since that time there hasn’t been any full-time development dedicated to Armstrong, but that’s not to say that development has stopped. Both the BC and TT, along with a handful of other organizations, use Armstrong internally and are continuing development on the project.

There are a few things I want to call out.

Timed Releases of Armstrong

The original idea, and one I’ve deviated from since last year, was to do timed releases, every three month. You might have noticed that the version numbers are pretty high. That’s because they follow the format vYY.MM. That way you know when the last major release of Armstrong was. Each one of those releases is just the latest stable code from all of the components of Armstrong that are considered production ready.

One key point, however, is that the main armstrong package (note the lowercase, that means it is code in its packaged state, not the project as a whole), is just a collection of other packages. You don’t have to install armstrong to be able to use various components. For example, you can pull armstrong.utils.backends into any project without using anything else from Armstrong if that’s all you need.

Release Components

All of the components of Armstrong are released independent of the major armstrong releases. There’s been a handful of component releases in the last year and more are being worked on right now. Each of these follows Semantic Versioning, or SemVer. That means that you can always upgrade within a major version, say from v1.2 to v1.6, and not worry about your code breaking. If anything breaks, a new major number is introduced. So far, we’ve only had to do that once: armstrong.apps.related_content.

All of the components in Armstrong that reach a v1.0.0 release, other than those grandfathered in for armstrong’s first stable release (v11.09), are being used in production. Following SemVer, production code goes to v1.0.0 as soon as it’s production. Part of the criteria for code making it to stable is that it’s being used on at least one site as production code. There’s one v0.x release that has running code so you can install it, then once that’s ready the v1.0.0 should be a simple version number bump with no code change.

The production requirement goes for point releases of code once it goes stable as well. Someone has to be using the code that’s in a pull release in order for it to be considered stable (and tested) enough for a release. Unit tests are a requirement, but code that is running on production is the final requirement for any component that’s released as stable.

This points the burden mainly on the Bay Citizen and Texas Tribune to make sure we’re running code that we’re trying to get released. Right now we’re the main ones that are effected by this, but it allows us to ensure the quality of the code. As more organizations start to contribute, they’ll have to play by the same rules.

Future Plans

There are a handful of areas where I would like to see Armstrong grow toward in the future. These coincide with the technical direction here at the Texas Tribune, so we’ll be driving many of these changes based on restructuring of our internal code.

Streams of Content

One of the biggest regrets in Armstrong was that I relented when arguing that we should structure content to exist independently with streams that any content could opt-in to. The argument against this route was that we had a tried solution—monolithic concrete-model dependencies—so why try something new until we’d replicated what we knew works. The old method does work, but it’s not scalable, whereas independent streams that you push content in to means you can decouple that relationship and scale it to many different types of content.

For example, say you have an Education Stream that contains content related to education. Stories can put themselves in that stream by providing the information the stream expects, but so can an update to data about a campus. All the data has to do is be able to render itself the way the stream expects and it can opt-in.

My initial plan is that an object would provide at least one rendered version of itself and its canonical URL. For a section stream, that rendered version would probably be the title, plus artwork, byline, and description.

This decentralization means that the stream display can be moved around to different services. You could also make it streams all the way down. Content notifies data streams that expect certain JSON documents and those data streams notify content destination streams (think: HTML, iOS, computers on the dashboard, TVs, and so on and so forth).

Those with a background in enterprise software might recognize this type of decentralization by another name: Service Oriented Architecture, or SOA. This type of architecture is not simply nice, it’s a requirement in a multi-device world. Building services that can only return HTML is shortsighted and going to cause problems as the number of devices our content is displayed on explodes in the coming years. Decoupling content from the various streams they’re consumed in is the first step in future proofing Armstrong.

Testbed for a new Django Admin

One area where I think a SOA allows greater freedom is the admin interface. The Django admin is great for what it gets you out-of-the-box, but you outgrow it very quickly especially when it’s laid next to modern web tools. You have to remember, the Django admin was designed in 2004/2005 when your main option for dealing with any type of data was phpMyAdmin and editing the database directly!

One thing I hope to do with the Hatband/Pops combo is create a testbed for experimenting on top of Django’s admin. These roles aren’t set in stone, but my thought is that Hatband will serve as the place for Armstrong-specific experimentation and Pops will be the place for generic Django experimentation.

Since starting Hatband and Pops, a few other tools have popped up in this space. Nobody has gotten significant traction, but I’m not opposed to joining forces with one of them if somebody does start to head down a solid path, but there are a few things that I see as a requirement.

  1. It needs to build on top of the existing django.contrib.admin code. The admin’s bones (with a few exceptions) are really solid. Rewriting it from the ground up isn’t a good use of time. It needs a lot of refactoring and many of the changes wouldn’t be backwards compatible, but it’s possible to make it happen by simply building on top of what already exists.

  2. It needs to focus on decoupling the actual interface from the discovery and registration of apps along with exposing an API to them. Right now, the biggest wart on the Admin is how tightly coupled display and discovery are. Any new admin needs to focus on providing a solid API (both Python and REST) for working with the apps that are registered. On top of that you can build a solid, default client interface. Having a dogfooded API ensures that others can build alternative interfaces on top of it. Think: native iOS apps for the admin!

  3. It needs to exist outside of core Django. I think the Admin is one of the reasons Django has gotten the traction it has. That’s great, but right now it moves too slowly for that to be useful for such a potentially rich web application. Having the admin exist as a separate project means that it can move more nimbly, release more often if it needs to, and gather support from those who might have no interest in working on a traditional web framework, but would love to work on a web application.

Separating Editing and Publishing

Currently, CMS tools assume that you’re editing and publishing content in the same tool. Those are two different workflows that need to have different tools: a collaborative authoring/editing tool and a publishing tool. They can exist on the same domain and even within the same major tool, but each workflow needs a different presentation and to be separated from the other.

The editing tool needs to have real-time feedback for the user when edits are happening. It should update in real-time showing who is editing what, the changes they’ve made, and so forth. It could include the ability to lock a field by focusing on it, but it should allow you to return control back to other users by removing focus, but ideally it’s smart enough to work with multiple users editing the same content.

This interface needs to focus the user on writing and editing. Things like sections, tags, locations, and so forth are all secondary content that should be tucked away, accessibly only when called on, to allow the user to focus on the task at hand. It would take a lot of user testing to design this system in a way that it could replace the existing tools (the number of emailed Word documents at the Tribune is still a source of embarrassment for me), but you’d end up with a solid workflow to take something from idea to finished product with the right focus.

Once the content has been written and edited, it needs to be published. Focusing explicitly on content takes all layout decisions away from the authoring experience and moves them to a place where you can make device-specific choices.

Responsive design should be the first choice for all content so it reaches the broadest audience, but there is also room for device-specific display where it make sense. Layout tools should enable this.

There is a start for this in armstrong.core.arm_layout with some code to help reuse model-specific rendering throughout the site, but those are baby steps toward a GUI-based layout tool.

Being able to control any aspect of the display of the site across multiple devices is the Holy Grail of a news platform, and one I hope we’re able to tackle as part of the Armstrong community.

Up Next

There are some very immediate plans, however. First, we need to roll another release of armstrong. I plan on creating armstrong v13.06 at the end of the month with the latest stable versions of all of the stable components.

v13.06 is going to be a maintenance release to get everything updated to the latest and remove the component-level requirement on Django and ensure full testing of Django from Django 1.3.x through Django 1.5.x. From here forward, the only dependency on Django is going to be specified at the armstrong level, leaving it up to developers to work through whether they can upgrade. We’ll continue to test components against all supported versions of Django.

This release will put us back on the timed release. The next release after this will be v13.09. I know the folks over at the old Bay Citizen have some new code they would like to see released and I’m hoping to have a solid admin interface for armstrong.apps.related_content by then as well.