Skip to main content

New Year's Python Meme

From Tarek through Lennart:

  1. What’s the coolest Python application, framework or library you have discovered in 2009?

Not quote Python, but I've been busy exploring various Javascript libraries and frameworks. I already knew YUI in 2008, and this year I've explored JQuery and affiliated extensions quite a bit.

I think I've had the most exploratory fun with JSON Template (Python and Javascript implementations both).

I started to use zest.releaser and this has made it a lot easier for me to release Python packages.

I only recently started using pyflakes, and it's been quite useful. A neat tool to supplement my toolbox.

There are other things I ran into that do seem cool but I can't really feel I've fully discovered them as I haven't really had a chance to use them.

  1. What new programming technique did you learn in 2009?

If I can count techniques I've been trying to pioneer myself: Template-driven development where the web browser renders the templates. This along with the notion of client-side views can lead to surprisingly clean rich client-side apps.

I also learned quite a bit about dependency management in a large collection of related libraries.

  1. What’s the name of the open source project you contributed the most in 2009? What did you do?

Grok and the Zope Toolkit. I'm not sure which one comes first, but luckily they're related. The Zope Toolkt is a set of libraries based on a refocused Zope 3. It's used by the Zope community in a multitude of projects, including Grok.

I also contributed a bit to JSON Template, making it possible to distribute its Python implementation on PyPI.

I've created a whole set of libraries:

hurry.resource should get a honorary mention. I actually created it in 2008, but it has seen quite a bit of uptake by others in 2009. People have been wrapping a number of Javascript and CSS frameworks with it.

hurry.resource is a general way to distribute and reuse javascript and css libraries in a fine-grained manner. You should be able to integrate it with any web framework.

In terms of bang for the buck, I think there were two projects I was involved in that had the most effect:

  • refocusing Zope 3 as the Zope Toolkit and improving the way in which it is being developed and managed.

  • cleaning up the dependencies of the Zope Toolkit. Cleaner dependencies help us cut out unused code, making the code base easier to understand, reuse and improve.

    In 2007 we had split up Zope into a collection of libraries. The dependency relationships between these libraries was rather convoluted, however, meaning that pulling in a single library would frequently pull in all the others.

    We had a long-standing wish to clean up these dependencies but unfortunately were making up slow progress. The project was however daunting.

    In early 2009, I organized a sprint that tackled this project head-on, cleaning up dependencies and also developing techniques, insights and tools.

    This project has been picked up by a large group of Zope developers throughout the year. Different people at different time cleaned up this and that, resulting in a vastly improved dependency structure compared to last year.

    The gains made by the Zope Toolkit are now making their way into Grok and Zope 2.

  1. What was the Python blog or website you read the most in 2009?
planet.python.org, though that's of course lots of blogs really. I've also read the Python programming reddit a lot.
  1. What are the three top things you want to learn in 2010?

I find it easier to say what I want to hack on, undoubtedly learning much as a side effect:

  • a new, lightweight publisher for the Zope Toolkit
  • making Grok smaller and more lightweight
  • rich client-side frameworks talking to RESTful backends

There's a lot more I'd like to hack on, so we'll see what I get around to.

Titus Brown on the importance of communication

I just read an insightful article by Titus Brown about the importance of communication in a team project, in the context of open source projects. It is a very good articulation of a topic I've thought about a lot myself as well.

When helping to manage Zope's participation in the Google Summer of Code I recognized the problems that Titus is talking about. One thing I told students in advance is simply to make noise

I've indeed seen some students, but not all, pop up on various Zope related mailing lists. Others have been more quiet. I'd prefer to hear some noise from you guys. Noise (if at least somewhat on-topic) is good for open source communities.

Noise is also good for you, as if you're noisy, people will get to know you and might actually start listening to some of your ideas. Don't be afraid to make noise. I've made an idiot and a nuisance of myself (on occasion, not all the time I hope!) in the permanent record for years and people still seem to listen to me.

That did have a positive effect. Still, the next year I started to explicitly require all participating students to give regular updates, just like what Titus is planning.

Without regular updates, it's far too easy for time (days, weeks, months) to pass by without obvious progress. Sometimes that is because the progress is hidden. Sometimes that is because the progress itself is in fact lacking: nobody knows you're blocked if you don't tell people about it.

Unfortunately even with those guidelines in place, it turns out to be difficult to make people communicate... Requiring regular reports helped me catch some potential problems early, but even raising my problem didn't always help to make the problem actually go away...

In part that's probably due to shyness or lack of confidence on the part of these people. When you communicate a lot, you may in fact end up saying things that aren't too smart once every while, and people probably worry overly much about that. Feedback can help you learn, and can only get feedback if people know you're there in the first place.

In part it's just due to differences in personality: not everybody communicates in the same way. That's fine - there are many ways to make a positive contribution, as long as you show what you're up to.

Sometimes too there is too much chatter and you just want to hunker down and get work done without distractions. It's just important to remember to resurface once every while.

On the other hand, communication can also be a great way to keep yourself motivated and productive. Sometimes it is quite comfortable to let time go by and slack off, and then it is nice nobody worries about your progress...

I'll just quoting myself again about noise (through Reinout):

I’m a natural noise-maker, and I discovered that while as a result of this I embarrass myself in public on a regular basis, it also means a lot of people know who I am. That’s a good thing.

And this, which I cannot even remember saying (through Michael Hudson):

faassen: anyway, if nothing else flapping your arms running around
will at least give an impression of activity. :)

Since this is a somewhat self-absorbed post anyway, quoting myself and all that, let me finish with a story about my early involvement in the Python community that I think is relevant to this discussion.

The comp.lang.python newsgroup when I joined it sometime in late 1998 was a small and friendly place. Thinking back it seems I spent most of my time posting very silly posts about the Python Secret Underground (PSU), an organization about which, naturally, I cannot say anything more. It was a lot of fun but perhaps not the most constructive contribution to the Python community.

I first went to an international Python conference in in the US in the year 2000. It didn't seem odd to me then, but afterwards I realized that many people there who I had never met before knew who I was. I also knew who many of them were. I think the silly interactions about the PSU that I was involved in on comp.lang.python explain a lot of that. It was completely unintentional, but today I still benefit from the connections I made then.

So, contribute to an open source community! Communicate! Communicate by creating code, writing documentation, or perhaps just by making people chuckle. Communication makes you more engaged in the project, and more motivated to contribute, and more creative and productive as a result. You may also be pleasantly surprised by the feedback.

Balancing stability and innovation in open source

A good open source community should have a good developer and user support infrastructure. It should have things such as:

  • releases that are recent enough to be useful
  • up to date and helpful documentation
  • a good web site
  • a helpful and active mailing list
  • automated tests
  • well maintained change notes
  • well managed issue tracker
  • well managed version control system

and so on. These are very important. Most open source communities probably feel they can do a lot better in some of the areas above.

All that is necessary and important, but it isn't enough.

A living, functional open source community should also have vision. It should support an atmosphere of ideas. Not just small ideas, but also some bigger ideas, that affect the very core of the project and everyone that uses it.

A community shouldn't only be in maintenance mode with improvements taking place in the periphery, or only in matters where consensus can easily be reached. A community should also be able to innovate in areas where consensus is difficult, because sometimes that is necessary in order to move forward.

Innovation is much easier in a smaller, younger community. In a larger, more mature community it is easy to forget about innovation because there is so much more focus on maintenance and stability. In such a community, innovations will need to be more incremental in order to safeguard stability. Incremental improvements can add up, however - evolution can be very innovative. A living, functional community can aim improvements in directions where they do add up.

A innovative community should have the following qualities:

  • creative energy. There should be people who come up with ideas, preferably based on experience. There should be people who are not satisfied with the status quo and are willing to do something about it. These ideas shouldn't be just be in the periphery but also about core aspects of the project.
  • supportive atmosphere: Creative energy should be channeled positively, not dispersed. A community should have an atmosphere where new ideas can be discussed and not just always be rejected or worse, ignored. Discussions should aim to refine ideas or offer alternative approaches.
  • process: The community should have clear mechanisms in place that can help lead a discussion to a conclusion that can be accepted by the community.
  • leadership: A process is important but not sufficient. In order to make the process work, the community should have people who are able to lead the process to conclusions.
  • implementers: people who actually go and implement the idea. Without them, it all stays with a discussion and the community is ultimately inert.

A living open source community shouldn't discourage fundamental innovations because the support infrastructure isn't right yet. It will never be perfect - this is another way in which perfection can be the enemy of the good. The community should support improvements to the support infrastructure and allow innovation at the same time. A great quality of open source communities is that people can work in parallel.

It is often possible to do both at the same time: innovate and as a side effect also improve the support infrastructure of the project, such as documentation. The community should have a culture where innovations go hand in hand with improvements to the support infrastructure that this innovation touches, such as documentation.

To conclude: in open source communities, a healthy balance between stability and innovation needs to be maintained. We should not lose sight of one by focusing too much on the other.

P.S. I just realized that it is possible to interpret this post as an argument against Python's PEP 3003, the "Python Language Moratorium". It wasn't intended as such; I'm very much in favor of this moratorium. The Python development community is a mature community where innovation on the language level has certainly not been lacking. The moratorium on language changes helps in the important area of stability, and also helps refocus innovation on areas that do need improvement more, such as the interpreter, libraries and the packaging infrastructure.

A history of Python packaging

Introduction

Earlier this year I was at PyCon in the US. I had an interesting experience there: people were talking about the problem of packaging and distributing Python libraries. People had the impression that this was an urgent problem that hadn't been solved yet. I detected a vibe asking for the Python core developers to please come and solve our packaging problems for us.

I felt like I had stepped into a parallel universe. I've been using powerful tools to assemble applications from Python packages automatically for years now. Last summer at EuroPython, when this discussion came up again, I maintained that packaging and distributing Python libraries is a solved problem. I put the point strongly, to make people think. I fully agree that the current solutions are imperfect and that they can be improved in many ways. But I also maintain that the current solutions are indeed solutions.

There is now a lot of packaging infrastructure in the Python community, a lot of technology, and a lot of experience. I think that for a lot of Python developers the historical background behind all this is missing. I will try to provide one here. It's important to realize that progress has been made, step by step, for more than a decade now, and we have a fine infrastructure today.

I've named some important contributors to the Python packaging story, but undoubtedly I've also did not mention a lot of other important names. My apologies in advance to those I missed.

The dawn of Python packaging

The Python world has been talking about solutions for packaging and distributing Python libraries for a very long time. I remember when I was new in the Python world about a decade ago in the late 90s, it was considered important and urgent that the Python community implement something like Perl's CPAN. I'm sure too that this debate had started long before I started paying attention.

I've never used CPAN, but over the years I've seen it held up by many as something that seriously contributes to the power of the Perl language. With CPAN, I understand, you can search and browse Perl packages and you can install them from the net.

So, lots of people were talking about a Python equivalent to CPAN with some urgency. At the same time, the Python world didn't seem to move very quickly on this front...

Distutils

The Distutils SIG (special interest group) was started in late 1998. Greg Ward in the context of this discussion group started to create Distutils about this time. Distutils allows you to structure your Python project so that it has a setup.py. Through this setup.py you can issue a variety of commands, such as creating a tarball out of your project, or installing your project. Distutils importantly also has infrastructure to help compiling C extensions for your Python package. Distutils was added to the Python standard library in Python 1.6, released in 2000.

Metadata

We now had a way to distribute and install Python packages, if we did the distribution ourselves. We didn't have a centralized index (or catalog) of packages yet, however. To work on this, the Catalog SIG was started in the year 2000.

The first step was to standardize the metadata that could be cataloged by any index of Python packages. Andrew Kuchling drove the effort on this, culminating in PEP 241 in 2001, later updated by PEP 314:

Distutils was modified so it could work with this standardized metadata.

PyPI

In late 2002, Richard Jones started work on the Python Package Index, PyPI. PyPI is also known as the Cheeseshop, a name I prefer but apparently has been deprecated. The first work on an implementation started, and PEP 301 that describes PyPI was also created then. Distutils was extended so the metadata and packages themselves could be uploaded to this package index. By 2003, the Python package index was up and running.

The Python world now had a way to upload packages and metadata to a central index. If we then manually downloaded a package we could install it using setup.py thanks to Distutils.

Setuptools

Phillip Eby started work on Setuptools in 2004. Setuptools is a whole range of extensions to Distutils such as from a binary installation format (eggs), an automatic package installation tool, and the definition and declaration of scripts for installation. Work continued throughout 2005 and 2006, and feature after feature was added to support a whole range of advanced usage scenarios.

By 2005, you could install packages automatically into your Python interpreter using easy_install. Dependencies would be automatically pulled in. If packages contained C code it would pull in the binary egg, or if not available, it would compile one automatically.

The sheer amount of features that Setuptools brings to the table must be stressed: namespace packages, optional dependencies, automatic manifest building by inspecting version control systems, web scraping to find packages in unusual places, recognition of complex version numbering schemes, and so on, and so on. Some of these features perhaps seem esoteric to many, but complex projects use many of them.

The problems of shared packages

The problem remained that all these packages were installed into your Python interpreter. This is icky. People's site-packages directories became a mess of packages. You also need root access to easy_install a package into your system Python. Sharing all packages in a direcory in general, even locally, is not always a good idea: one version of a library needed by one application might break another one.

Solutions for this emerged in 2006.

Virtualenv

Ian Bicking drove one line of solutions: virtual-python, which evolved into workingenv, which evolved into virtualenv in 2007. The concept behind this approach is to allow the developer to create as many fully working Python environments as they like from a central system installation of Python. When the developer activates the virtualenv, easy_install will install all packages into its the virtualenv's site-packages. This allows you to create a virtualenv per project and thus isolate each project from each other.

Buildout

In 2006 as well, Jim Fulton created Buildout, building on Setuptools and easy_install. Buildout can create an isolated project environment like virtualenv does, but is more ambitious: the goal is to create a system for repeatable installations of potentially very complex projects. Instead of writing an INSTALL.txt that tells others who to install the prerequites for a package (Python or not), with Buildout these prerequisites can be installed automatically.

The brilliance of Buildout is that it is easily extensible with new installation recipes. These recipes themselves are also installed automatically from PyPI. This has spawned a whole ecosystem of Buildout recipes that can do a whole range of things, from generating documentation to installing MySQL.

Since Buildout came out of the Zope world, Buildout for a long time was seen as something only Zope developers would use, but the technology is not Zope-specific at all, and more and more developers are picking up on it.

In 2008, Ian Bicking created an alternative for easy_install called pip, also building on Setuptools. Less ambitious than buildout, it aimed to fix some of the shortcomings of easy_install. I haven't used it myself yet, so I will leave it to others to go into details.

Setuptools and the standard library

The many improvements that Setuptools brought to the Python packaging story hadn't made it into the Python Standard Library, where Distutils was stagnating. Attempts had been made to bring Setuptools into the standard library at some point during its development, but for one reason or another these efforts had foundered.

Setuptools probably got where it is so quickly because it worked around often very slow process of adopting something into the standard library, but that approach also helped confuse the situation for Python developers.

Last year Tarek Ziade started looking into the topic of bringing improvements into Distutils. There was a discussion just before PyCon 2009 about this topic between various Python developers as well, which probably explains why the topic was in the air. I understood that some decisions were made:

  • let the people with extensive packaging experience (such as Tarek) drive this process.
  • free the metadata from Distutils and Setuptools so that other packaging tools can make use of it more easily.

Distribute

By 2008, Setuptools had become a vital part of the Python development infrastructure. Unfortunately the Setuptools development process has some flaws. It is very centered around Phillip Eby. While he had been extremely active before, by that time he was spending a lot less energy on it. Because of the importance of the technology to the wider community, various developers had started contributing improvements and fixes, but these were piling up.

This year, after some period of trying to open up the Setuptools project itself, some of these developers led by Tarek Ziade decided to fork Setuptools. The fork is named Distribute. The aim is to develop the technology with a larger community of developers. One of the first big improvements of the Distribute project is Python 3 support.

Quite understandably this fork led to some friction between Tarek, Phillip and others. I trust that this friction will resolve itself and that the developers involved will continue to work with each other, as all have something valuable contribute.

Operating system packaging

One point that always comes up in discussions about Python packaging tools is operating system packaging. In particular Linux distributions have developed extremely powerful ways to distribute and install complex libraries and application, manage versions and dependencies and so on.

Naturally when the topic of Python packaging comes up, people think about operating system packaging solutions like this. Let me start off that I fully agree that Python packaging solutions can learn a lot from operating system packaging solutions.

Why don't we just use a solution like that directly, though? Why is a Python specific packaging solution necessary at all?

There are a number of answers to this. One is that operating packaging solutions aren't universal: if we decided to use Debian's system, what would we do on Windows?

The most important answer however is that there are two related but also very different use cases for packaging:

  • system administration: deploying and administrating existing software.
  • development: combining software to develop new software.

The Python packaging systems described above primarily try to solve the development use case: I'm a Python developer, and I'm developing multiple projects at the same time, perhaps in multiple versions, that have different dependencies. I need to reuse packages created by other developers, so I need an easy way to depend on such packages. These packages are sometimes in a rather early state of development, or perhaps I'm even creating a new one. If I want to improve such a package I depend on, I need an easy way to start hacking on it.

Operating system packaging solutions as I've seen them used are ill suited for the development use case. They are aimed at creating a single consistent installation that is easy to upgrade with an eye on security. Backwards compatibility is important. Packages tend to be relatively mature.

For all I know it might indeed be possible to use an operating system packaging tool as a good development packaging tool. But I've heard very little about such practices. Please enlighten me if you have.

It's also important to note that the Python world isn't as good as it should be at supporting operating system packaging solutions. The freeing up of package metadata from the confines of the setup.py file into a more independently reusable format as was decided at PyCon should help here.

Conclusions

We are now in a time of consolidation and opening up. Many of the solutions pioneered by Setuptools are going to be polished to go into the Python Standard Library. At the same time, the community surrounding these technologies is opening up. By making metadata used by Distutils and Setuptools more easily available to other systems, new tools can also more easily be created.

The Python packaging story had many contributors over the years. We now have a powerful infrastructure. Do we have an equivalent to CPAN? I don't know enough about CPAN to be sure. But what we have is certainly useful and valuable. In my parallel universe, I use advanced Python packaging tools every day, and I recommend all Python programmers to look into this technology if they haven't already. Join me in my parallel universe!

Update: I just found out there was a huge thread on python-dev about this in the last few days which focused around the question whether we have the equivalent of CPAN now. One of them funny coincidences... http://thread.gmane.org/gmane.comp.python.distutils.devel/11359

The Success of the Zope Component Architecture

Prompted by recent brief negative pronunciations by Malthe on the Zope Component Architecture (ZCA), I thought I'd talk a bit about what I think about it. I'm not going to go into hermeneutics here of what Malthe might mean -- others attempts at exegesis exist in the comments to that blog entry already. Instead, I'll just talk about what I think makes the ZCA useful, and why it is successful. Finally I'll go into some reasons why people are frustrated by the ZCA.

What is the ZCA used for?

What is the ZCA used for? It's used to glue things to each other: glue event handlers to events, glue views to models, glue plugins into applications and libraries, and more abstractly, glue adapters to adaptees. In ZCA terms, providing such glue is termed providing configuration.

Why was the ZCA created? The Zope community had been building pluggable web applications for a long time and we noticed our components became overly complex and were hard to glue together and override. The ZCA is one answer to this problem.

The ZCA is implemented by zope.component. It's a library for gluing. It's built on top of zope.interface, a library that helps one define the bits that are being glued.

What advantage does such gluing bring?

  • many, perhaps all, larger applications contain glue. The ZCA makes the glue explicit and uniform.
  • ZCA glue can be overridden explicitly.
  • you can extend existing systems by gluing in new things.

One place where the ZCA is helpful is when you want to write a library that offers a few plugin points to configure it for a particular environment. For instance when I wrote hurry.resource, a library for handling javascript and other resources, I included a few plugin points in it that allow it to be plugged into a particular web framework. Then to allow it to be used with Zope Toolkit-based frameworks such as Grok, I wrote hurry.zoperesource to provide the knowledge about that.

This way, hurry.resource doesn't need to know anything in particular about URL generation or requests; its plugins can take care of this. This allowed me to write and test hurry.resource without worrying too much about the larger Zope Toolkit framework, knowing I could plug it in later, and now the library becomes more useful for a broader group of people.

The ZCA doesn't just allow one to glue one thing to another, but also to override the glue in specific cases. A common example of overriding glue occurs with views. Zope Toolkit applications follow a view/model approach, where the view is looked up on the model dependent on its class (or interface). It happens frequently enough in an application that I want many models to share a particular view, but override one model with a more specialized view. This is much like the way inheritance works in plain Python: I implement a method on a base class shared completely by some subclasses, but for one subclass I'd like to override it.

What I described just now is overriding for particular subclasses. The ZCA also allows other ways of overriding based on the zope.configuration library, overriding one glue registration with another one. I myself find myself using this kind of override less frequently, but it's still very useful in the 1 percent of cases where other options would be very ugly.

The question is sometimes asked why not just modify code dynamically for overrides? Why not monkey patch it? (or "open the class", or whatever other terminology one would like to use).

Brandon Rhodes at PyCon 2008 gave a great presentation explaining why the ZCA approach can be superior to monkey-patching and some other approaches. The point he makes is that the ZCA is the most composable and maintainable approach of the alternatives (subclassing, mixins, monkey-patching, adaptation). I recommend everyone interested in this topic to read his slides.

Of course as with everything in programming, everything depends on the trade-offs. In many circumstances the ZCA is overkill. The ZCA can be misused. But I also maintain that in many circumstances the ZCA is very useful.

Component approaches have become quite popular with web frameworks. Many popular ones adopt elements of it, but often in a somewhat limited way. For instance, one very popular interface is the WSGI interface, and one popular form of adaptation of this WSGI interface is to use WSGI middleware and framework components. With WSGI we see that just one well-defined, consensus interface has become an amazing source of creativity and pluggability within web development. With the ZCA we are able to define more than one interface in our applications, and potentially create ecosystems of creativity around those. Not that this is easy, but at least we have a mechanism and method to do so.

The ZCA is successful

Why do I say the Zope Component Architecture (ZCA) is successful? It's successful as it's being used, by many people, for many years now. Of course you can say it's only used by that weird Zope community, and a bit by weird Twisted people as well perhaps. That's fine, but realize that the wider Zope community is big and is made of many parts: Plone, Silva, Zope 2, Zope Toolkit, Grok. I'll also count BFG as part of the wider Zope community.

The ZCA is successful for me. Without it, I'd have to invent something very much like it. It comes back in much code that I've written. I'm able to do all kinds of small, cool things in my applications and libraries on a daily basis, and I'm creating more reusable code as a result.

Frustration with the ZCA

The ZCA isn't perfect. Humans aren't perfect, either. It's not easy to create good reusable interfaces, or to build pluggable frameworks and applications. It's easy to overdo the pluggability. It's easy to create the wrong pluggability points. It can get overly complex and overwhelming.

Writing XML configuration files to glue things together can be cumbersome and lead to repeating yourself, though his last issue has been overcome for some time now by the Grok project and its reusable solution (grokcore.component) is available to everybody.

It's easy to let the pain of the mistakes in the use of the ZCA, and the pain of complex applications in which it is used in general, overshadow the many successes of the ZCA: in a large part thanks to to the ZCA's ability to glue things together people are able to use Zope Toolkit libraries within a Zope 2 or Plone context. Grok was able to remix the Zope Toolkit in part because we could easily modify and extend the framework. I think the web frameworks that use the ZCA offer pluggability and flexibility unrivaled by the competition.

We generally don't say that the problems and complexities in our large libraries and applications are due to the Python programming language. While Python isn't perfect, we tend to think overall it helps. I also tend to think that the ZCA helps.

Are there alternatives to the ZCA? Certainly: any library or application specific pluggability system is an alternative. Sometimes a specific approach is better than a generic one. It can be easier to understand it's smaller.

Usually I'd say it's the other way around. Each custom pluggability system is another one to learn, and has its own limitations. I've found multiple times that I can make a far more powerful pluggability system for a library in a shorter time if I build it on top of the ZCA.

There are also more general approaches where there's at least some overlap with the ZCA: setuptools entry points, WSGI middleware, and PEAK-Rules. To use any of them for pluggability and overriding one needs to have a notion of an interface one can implement and plug into, a notion that the ZCA makes explicit.

Conclusions

The ZCA is useful. The ZCA is powerful. The ZCA is successful. The ZCA is imperfect. When I've run into its imperfections I've helped build solutions on top of it, such as Grok's in-python gluing approach. Sometimes I've considered its use overkill and gone for some other approach. I think it's useful to step back and consider alternative approaches. But let's consider them in the light of the success of the ZCA as much as in the light of its flaws.

Grok 1.0 released!

Grok 1.0 was released today! Almost exactly three years after the first lines of code were written, and many Grok-based applications later, we've finally released Grok 1.0.

Grok is a web framework based on the Zope Toolkit. It's based around a powerful component architecture (zope.component) and a powerful system for connecting these components (zope.configuration and Grok's own Martian).

Here is the official release announcement:

http://grok.zope.org/project/releases/1.0

And for much more about grok, see http://grok.zope.org

We're not stopping here: the Grok development team is already hard at work at creating many new extensions for Grok, and of course we're now we're excited about all the new features and improvements we can add to Grok!

Europython slides are online

Long time no blog. I've been at Pycon and EuroPython since then (which both were great experiences!). At EuroPython I gave a talk titled "Things I helped create". I've just uploaded the slides for this in the upload area of the EuroPython wiki:

http://wiki.europython.eu/TalkMaterials?action=AttachFile&do=get&target=creation.odp

I was overly ambitious in the amount of material I should be able to present in an hour, so the talk ends somewhat abruptly and doesn't cover quite a few other things I worked on. I hope to rework some of that material for another talk in the future.

To PyCon I Go

I decided to attend PyCon this year; there are a lot of people going there that I'd like to meet, and a lot of people I'd like to run into again. Oh, and there are cool talks too!

I'll be there for the duration of the conference itself, and the first 2.5 days of the sprint.

I'll be wearing many hats: my Startifact hat, my Grok hat, my Zope Foundation chairman hat (we have a new board, but haven't elected a new chairman yet), my Zope Framework Steering Group hat and my Zea partners hat. And my Martijn-the-python-programmer hat, if that's a hat. If you want to talk to me in any of these capacities or any other, please do!

A constructive general meeting of the Zope Foundation

We just had a constructive general meeting of the Zope Foundation on IRC, the first one under the new bylaws.

Even though we had some issues with cleaning up our membership roster, we luckily got it all cleared up during the meeting. The election results were ratified by the membership and we are starting off with a larger more active board. As the old board, we tried to clear the way for such a board, and I hope we'll be able to continue to increase the relevance and visibility of the Foundation.

The new board consists of:

  • Christian Theune
  • Stephan Richter
  • Tres Seaver
  • Jens Vagelpohl
  • Rob Page
  • Andreas Jung
  • Martijn Faassen (myself)

I think this is a very good range of people active in our community, with some fresh people who weren't active on previous boards, and hopefully a renewed energy. Cheers to the new board!

As a special note here I want to thank Lennart Regebro in helping clearing up the issues caused by the bylaws transition that came up earlier this week. While the initial interaction we had earlier this week was very discouraging to me, I understand we were both upset for understandable reasons. Lennart has been very constructive in helping us resolve the issue considering our "limbo members". As a result of this and thanks to the support by the membership during our general meeting, we can now put that particular episode behind us.

I know Lennart quite well, like him and respect him, and I know he cares about Zope and the Foundation. My previous blog posting wasn't intended to cause community members to give him undeserved grief. I can do that just fine by myself when I want to. :). Lennart is good people.

My blog entry was just my public response to what had become a public issue, and an expression of my feelings at the time. I feel a lot better now, and wish to thank everybody who gave me support. Your appreciation is appreciated!

The Perils of Volunteering

I'm on the board of the Zope Foundation. In fact, until thursday this week, when hopefully the new board gets seated, I'm its chairman.

The old bylaws of the Zope Foundation sucked. We were burdened with them from the start; we inherited them as the board from the startup phase. We tried to work with them but they didn't work, and then we decided to go with new bylaws.

So we painstakingly work on them. We finally had a big meeting in december trying to get all our members there so we can vote for the new bylaws and have it pass. It did pass, and we were happy.

So the Zope Foundation has new bylaws. But we have membership under the old bylaws. Our membership roster is old and stuffy, and it won't work as our bylaws changed and membership categories are different. The bylaws old or new don't provide for a membership transition. We're in a limbo. How do we get from old to new?

A few hard-working volunteers, Zope Foundation board members, try to figure this out. We've been working on this frustrating bylaws stuff for about the entire existence of the Foundation by now.

Now we all want to get this over with and make way for a new board. We had already discovered that to do such a transition we need to make reasonably bold steps (like replacing the bylaws!), otherwise nothing much will happen.

We decide to mail everybody that is an existing member and ask them what membership status they have, and if they don't answer, we'll assume they don't want to be a member.

Something's weird. We noticed that some supposed members didn't receive the email. It turns out we missed out on members who were member only through their employer's membership under the previous structure. So we contact those.

That, we thought, was it, doublechecking we did everything all right. But we hadn't. Mea Culpa as the chairman of the board. I should've made sure we had made a second try to contact people. I think I got distracted by the other issue, and thought it was done.

We announce the foundation elections to our membership as it is then, after the cleanup. We neglect to announce it in public. Of course we don't have this obligation, but we should have anyway to just keep interested bystanders informed.

People point out they don't hear much from the Zope Foundation. Alexander Limi did today just before we had a board meeting. We do have a website and such that people can read, but it's not working in getting to people. I, slightly overburdened volunteer, grumble, but point taken.

The new board should definitely improve this, so I put it on the list of todo items for the new board. Even the old board will. So when we announce the election result, we make sure we did announce that to zope-announce, today.

Lennart Regebro doesn't always read the email the Foundation sends to him directly about his membership status, but he does read zope-announce.

So Lennart gets upset as he hadn't been aware of any election! He contacts us. Me in particular. He tells me he wants us to redo the election. Oh joy... Deep sigh.

So today I have a headache from all this. I was spending quite a few hours trying to help the Zope community organize itself, not getting paid for it, and I got an angry Lennart as a result.

I want to think about something else. I try to relax. Then accidentally read the blogs as feeds are so very nicely integrated in my web browser these days. So what do I see? Lennart analyzing all our mistakes, my mistakes, in public in great detail in a blog entry.

It's an excellent way to discourage volunteers who have been genuinely trying to do their best. I know, it wasn't Lennart's intent to discourage us. Not sure what the intent of that blog posting was. Maybe it was to get us to say sorry. Sorry, Lennart.

I feel discouraged. I want to chuck the whole Foundation, and what the heck, chuck the whole Zope thing to boot. That's just me throwing a hissy fit, though, and I should be better after some sleep. I will probably be able to joke and laugh about this soon. But that's not how I feel right now.