Skip to main content

A Creative Trajectory

Introduction

Humans are storytellers. As anyone who knows me can confirm, I definitely enjoy the activity of telling stories. We process and communicate by telling stories. So let me tell you all a story about my life as a web developer the last 7 years. It may just be self-reflection, but hopefully it's also useful for some others. Perhaps we can see my story as a highly personal history of the web.

I always say that what pulls me to software development most is creativity. I am creative; I can't help myself. I enjoy thinking about creativity. So this is going to be about my creative trajectory over the last 7 years.

Why 7 years? Because in early 2010 I decided to withdraw myself from a software development community I had been involved in for about 12 years previously, and I took new paths after that. It is now early 2017; time to take stock.

Letting go of an intense involvement with a software development project, up to the point where it became part of my identity, was difficult. Perhaps that's a geeky thing to say, but so it is. I needed to process it.

Life after Zope

Zope was a web framework before that concept existed. The modern web framework only materialized sometime in the year 2005, but Zope had been ahead of its time. I was involved with the Zope community from when it was first released, in 1998. I learned a lot from Zope and its community. I made many valued connections with people that last until this day.

Zope helped shape who I am and how I think. In 2013 I wrote a retrospective that went into the history of Zope and my involved with it.

But I did not just process it by writing blog posts. I also processed it creatively.

Frameworks

So I am a web developer. Back in 2010 I saw some people argue that the time of the framework had passed. Instead, developers should just gather together a collection of building blocks and hook it up to WSGI. This would provide more flexibility than a framework ever can.

In a "X considered Y" style post I argued that web frameworks should be considered useful. Not that many people needed convincing.

I wrote:

The burden of assembling and integrating best of breed components can be shared: that's what the developers of a framework do. And if you base your work on a pre-assembled framework, it's likely to be less work for you to upgrade, because the framework developers will have taken care that the components in the framework work together in newer versions. There is also a larger chance that people will write extensions and documentation for this same assembly, and that is very likely something you may benefit from in the future.

I follow the ReactJS community. The React community definitely is a community that lets you self-assemble a framework out of many parts. This gives that ecosystem flexibility and encourages creativity -- new approaches can be tried and adopted quickly. I like it a lot.

But figuring out how to actually start a React-based project had become a major effort. To get a best-of-breed development platform, you needed to learn not only about React but also about a host of packaging and build tools: CommonJS and Webpack and npm and Babel and so on. That's quite intimidating and plain work.

Dan Abramov realized this and created create-react-app which makes it easy to start a working example, with minimal boilerplate code, and details how to move on from there too. It's a build framework for React that gathers best-of-breed components in one place. It's demonstrates how frameworks can make life easier for developers. It even goes a step further and allows you to opt out of the framework once you need more control. That's an interesting idea!

Ideas and the Client-side Web

So frameworks are useful. In late 2010 I had an idea.

This it how almost all JavaScript development used to be done (and still is done in many cases): the server framework generated the HTML that would be the UI, and handled all the details of UI interaction in request/response cycles. But sometimes this would not be enough. More dynamic behavior on the client-side would be needed. You'd then write a little bit of JavaScript to do it, but only when absolutely needed.

This made JavaScript the ugly stepsister of whatever the dominant server programming language was; a minor servant of Python, or Ruby or Java or PHP whatever the server language was. The framework was on the server. JavaScript was this annoying thing you'd have to use; a limited, broken language. As a web developer you'd spend as little as time possible writing it.

But JavaScript had been gathering strength. The notion of HTTP-based APIs had attracted wide attention through REST. AJAX had been coined by 2005. Browsers had grown in capabilities. To exploit this more and more JavaScript needed to be written.

jQuery was first released in 2006. jQuery had better APIs over sucky browser APIs, and hid incompatibilities between them. Its core concept is the selector: you select things in the web page you want to enhance. Its core concept exactly supports this paradigm of using client-side JavaScript: as a servant to the server, which does the HTML generation.

The notion of a single-page web application had also emerged. It promised more powerful and responsive UIs than server-side development could bring. The backend would be a HTTP API. Separation between UI on the client and business logic on the server could become stronger.

By 2010, I had played with client-side template languages already. I was about to build a few large web dynamic single-page applications. I wanted something that would help organize client-side code better.

So my idea was to bring concepts from server frameworks over to the client and to see what happened. Cool things happened! We started with templates and then quickly moved to MVC patterns. We created a notion of components you could compose together. We created a client-side form library based on that. In 2011 we released this as Obviel.

For a little while I thought I was the only one with this cool idea of a client-side web framework. It turns out that I was not: it was a good idea, and it was an idea of the time. Even before I'd released Obviel, I started to hear about Backbone. Ember and Angular soon followed.

I continued working on Obviel for some. I created a template language with built-in i18n system for it, and a client-side router.

In 2011 and 2012 we built a lot of stuff with Obviel. In the beginning of 2013 those projects were done. Obviel didn't get any traction in the wider community. It was a project I learned a lot from, so I don't regret it. I can claim deep experience in the area of client-side development.

I went to my first JS conference in September of 2013. Pete Hunt gave a keynote that was the second time React had been introduced to a wider audience. Apparently it went over a lot better than the first time. It certainly made an impression on me. I eventually let go of Obviel and started to use React.

The Server in Modern Client-side times

In early 2013 I read an interesting blog post which prompted me to write Modern Client-Side Times, in which I considered the changed role of the server web framework if it was to be the servant to JavaScript instead of its master.

I wrote a list of what remains on the server:

What remains is still significant, however:

  • serving up JSON on URLs with hyperlinks to other URLs
  • processing POSTs of JSON content (this may include parts of form validation)
  • traversal or routing to content
  • integrating with a database (SQLAlchemy in this case)
  • authentication - who are you?
  • authorization - who can access this URL?
  • serve up an initial web page pulling in a whole bunch of static resources (JS, CSS)

I also wrote:

Much of what was useful to a server side web framework is still useful. The main thing that changes is that what goes over the wire from server to client isn't rendered HTML anymore. This is a major change that affects everything, but much does stay the same nonetheless.

I didn't know at the time of writing that I would be working on just such a server web framework very soon.

On the Morepath

In 2013 I threw some smaller pieces I had been playing with for a while together and created Morepath. I gave an over-long keynote at PyCON DE that year to describe the creative processes that had gone behind it. I gave a more focused talk at EuroPython 2014 that I think works better as an introduction.

I announced Morepath on my blog:

For a while now I've been working on Morepath. I thought I'd say a bit about it here.

Morepath is a Python web micro-framework with super powers. It looks much like your average Python micro-framework, but it packs some seriously power beneath the hood.

One of the surprises of Morepath was the discovery that a web framework that tries to be good at being a REST web server actually works very well as a server web framework as well. That does make sense in retrospect: Morepath is good at letting you build REST services, therefore it needs to be good at doing HTTP, and any HTTP application benefits from that, no matter whether they render their UI on the client or the server.

2014 was full with Morepath development. I announced it at EuroPython. It slowed down a little in 2015, then picked up speed again in 2016. Over the last few years a few new core contributors came by, so Morepath is not just me anymore.

I'm proud that Morepath is micro in implementation, small in its learning surface, but macro in power. Morepath itself is currently a little over 2000 lines of Python code, but it does a lot, helped by the powerful Reg (<400 lines) and Dectate (<750 lines) libraries.

The Future?

In late 2016 Morepath started to gain a bit more attention in the wider community. I hope that continues. A few more people using it that may turn into contributors is invaluable for an open source project.

There was a mention of Morepath in an Infoworld article, I was interviewed about for Podcast__init__, and I also interviewed about it for an upcoming episode of Talk Python to Me.

Ironically I've been writing some Django code lately. I'm new to Django (sort of), but I feel a lot like I was programming 7 years ago, where the server rules and JavaScript is the thing you use if you have to. The paradigm works, and it's the way to go for some projects, but it's definitely not my preferred way to work anymore. But I get to help with architecture and teach a bit so I'll happily take a bit of Django.

The Django management UI is cool. It makes me want to implement the equivalent for Morepath and PonyORM and React and Mobx. Or something.

I've been itching to do something significant on the client-side again. It's been a little while since I got to do React. I enjoyed attending React Europe 2015 and React Europe 2016 and I want to work with that stuff again.

The space where client and server interacts is fertile with creative potential. That's what I've found with Obviel and React on the client, and with Morepath on the server side. While GraphQL replaces the REST that Morepath is based around (oops!), I'd enjoy working with it too.

Where might the web be going? I like to think that by being creative I sometimes get to briefly peek into its near future. I hope I can continue to be creative for the next 7 years, as I really enjoy it.

I'm a freelancer, so the clients I work for in part also shape my creative future. Hint. Let me know if you have something interesting for me to do.

Comments

Comments powered by Disqus