# SQLAlchemy 0.6 has been released.
SQLAlchemy 0.6

# stdeb produces Debian packages from Python packages
stdeb on GitHub

# Mike Malone, an engineer at SimpleGeo put together a Cassandra mock in Python.
Fake Cassandra Gist

# Mike Dirolf posted a recipe on using date range queries on a Mongo collection via the Python driver
Mongo date range

Armin Ronacher has released a new micro-framework called “Flask” inspired by Ruby’s Sinatra framework
Flask 0.1
Flask on GitHub
Flask Project Page

Paul Bohm released “Tragedy” a high-level Cassandra Object Abstraction for Python.
Tragedy on GitHub

Johann Rocholl gave a presentation to the Seattle Google Technology Users Group (GTUG) on profiling and performance tuning App Engine applications.
Making app engine fast

Check the redis fork here: Redis include directive fork

list_len = len(mylist)
if list_len > 0:
for item in list_len:
…

This just feels wrong and I wanted to make two styleistic points here.

1) IF you have a method that returns a list, always return a list. If it is empty don’t return None and make me check if the value is None or not. Just return an empty list [] or tuple ()

2) If code follows step #1, don’t check – just iterate. If the list is empty obviously the iteration will never happen which is what is desired in the snippet above.

Simon Wilson presented dmigrations . Installing dmigrations is as simple as installing it in INSTALLED_APPS, and it will registers a few custom admin commands:

./manage.py dmigrate app APP_NAME

./manage.py dmigrate list

./manage.py dmigrate addcolum

My take: dmigrations is great and will work for migration problems right now, but in its current form is unlikely to end up in django as the annointed migration solution. Why? Because it basically wraps SQL directly, loosing some of the cross-database portability of the Django DB.

Andrew Godwin: South — described as the next step of dmigrations. Philosophy: Migrations are essential, branched development / missing migrations, inter-app dependencies, database abstraction needed too. Can handle model dependency (foreign keys.)

My Take: worth looking into, more database indapenent.

Russel Keith-Magee: Django-evolution, his response to a mailing list thread that would not end. Google Summer of code 2006. Russel complaining of Magic moniker attached to django-evolution, everything done via introspection of models. Goals: Hint and Tweak, Simple changes without user intervention, easy entre for customization, raw sql, self documenting, self auditings, Validation (where possible).

Signature: pickled summary of django model. Stores state of django models at syncdb, can be diffed against current models, diffs used to generate hints.

Mutation: Atomic unit of change, common operations built in. Can be user-defined, can be raw sql, know the effect they will have.

Evolution: Ordered collection of mutations. Two flavors of mutations: hinted and stored. Executed evolutions stored in database.

Hinted Evolution: Best guess by looking at diff, if acceptable can be used to execute evolution right away. Can also be used as prototype to stored evolution. Can’t resolve ambiguous updates (rename) can’t fill in the blanks (initial data)

Stored Evolution:
Named sequence of mutations. Defined per application, stored in evolutions directory of app (can be put into version control).

django-evolution extends syncdb — as in ./mange.py syncdb — schema change detected you need an evolutions.

Custom commands:

./manage.py evolve –hint

./manage.py evolve

My Take: Highly interesting, most likely to end up in django. Grok this.

After reading around 6000 bug tickets, certain patterns have emerged in the submitted patches…

Code Quality Matters — funny quote from Leah Culver: “Have you every written a library? It’s like people seeing you in your underwear. You gotta make sure it’s clean.”

Do the basics properly. The word “print” is in your patch. Similarly for “import pdb”. If you think PEP 8 is the name of a new energy drink. Run the django testsuite. Submit patches, and not entire files. Start from top of tree when using svn diff.

Read the contributing document (contributing.txt), use sensibly names, comments should last, comments should be correct.

Fix problems, not symptoms. The crowd is smarter than you think — the code mostly works. If you find yourself ripping out lots of code, stop and think.

The code for the ORM is located in django/db with juicy bits in the following locations:

django/db/models/query.py (public queryset API)

django/db/models/sql/* (Public API->SQL conversion. Deep dark internals. Does’t know DB, knows SQL)

django/db/backends/* (Individual DB wrappers, third-party wrappers possible. This is where you actually talk to the DB). See (dummy or mysql dir for examples)

The different pieces in depth

Blog.objects.filter(owner=user) = [model, manage, QuerySet method]

Model Manages:

inherit from django.db.models.Manager, ideal for extra methods that act on the whole table at once, not just one record, usually wraps/proxies public querySet methos, has method called get_query_set() which returns QuerySet (QuerySet.all)

QuerySets:

django.db.models.query.QuerySet

Every time you call a method on a QuerySet it returns a copy (clone) of that QuerySet (side effect free?) For example every time you call filter on a query set you get a different queryset.

Query

django.db.models.sql.query.Query, an attribute of QuerySet, holds the internal state of the current query, knows how to produce SQL. This is where you implement something that knows how to speak to MySQL, or CouchDB or Hadoop for example. (Aka class from Hell, almost every data-structure known to man used here). AsSQL is when state is rewritten into SQL. This is a change from before QuerySet Refactor landed in trunk where the internal representation for everything was strings.

Every method of QuerySet updates the Query (calls method on Query).

Query.results_iter() is how the results end up from the DB back into Python objects, reconstructing Python objects happens in various other Query methods: Query.select_related, Query.extra_select…etc

Looking inside the Query. Query.add_filter() is the guts — the place to start when trying to grok the Query class. Query.setup_joins() converts filter/exclude into table joins. Query.join() responsible for joining a pair of tables

Recap on flow/organization: Manager->QuerySet->Query

Customization:

Custom Managers (Easy)

Custom QuerySet (Not so Easy)

–> (Some example QuerySet) ValuesQuerySet, DateQuerySet, EmptyQuerySet

Custom Query (Not so Easy)

GeoDjango for example has to use very different types of query.

Note to self: I’m wondering how(or even IF) one would  use Django’s ORM for a Solr-backend for intergrated search via a DJango ORM model.

Review-Board is a python/django based code-review application. In 2004 VMware was approx. 600 employee’s and already had a rigorous code-review process in place. At first simple HTML snippets were emailed around. Fast forward to 2007 and the company had grown to 5000 employee’s, and the previous system of email around snippets was unmanageable.

And thus Review-Board was born.

Earlier in the day Guido mentioned that he wrote a similar, python based code review tool that is used internally at google. I’ll try to dig that up and make a follow up post.

Stats: 2 core developers, 74 contributors, 295 mailing list members.

I’ll be installing this on my own dev box — In the future expect a post or two about how it intergrates with other tools I like (git/svn, trac…etc)

Earlier in the week I received “Practical Django Projects” in the mail — chomping at the bit for enough time to get into it.

Exciting times.

I wonder if there isn’t some room in the Django ORM back-end to support stuff like S3 and Google App Engine — with some warnings of course that the store isn’t relational… etc

At first I thought it was a perl specific choice, but a few quick tests with C and Python confirmed this was not the case.

It seemed like an odd design choice given that truthiness in this (programming) context is quasi-inspired after the presence of current in a given transistor. So why wouldn’t you want negative integers to be false?

Then I started thinking that it was either an optimization issue — so I started looking at the x86 assembly generated by GCC for my C example (see below). I used the -S flag to gcc to save the assembly so gcc -S test.c created the test.s (assembly). A quick glance revleaed that this was indeed the case, particularly the following three lines:

  movl  $-1, -4(%rbp)
  cmpl  $0, -4(%rbp)
  je  .L2

movl moves the value of X (-1) to the appropriate place (this is int x = -1) then the actual comparison happens with the cmpl operator which compares that value to zero. If those to values are equal (0 and X) the je executes (jump-if-equal) and executes that branch.

Now the next question, for another day, why is gcc assembly ifs to JE and JLE — which would return false for anything less than 1 for example.

My guess is because it’s more expensive. But more to come on that subject.

perl:

#!/usr/bin/env perl
if (-1) {
  print STDERR "I am true \n";
}

python:

#!/usr/bin/env python
num = -1
if (num):
  print "I am true"

C:

#include  

int main (void) {
  int x = -1;
  if (x) {
    printf("I am true");
  } else {
    printf("not true");
  }
}

It will be hosted the weekend of September 6th and 7th at the GooglePlex (Mountain View, CA). Since this is the first conference (and hosted at Google) space will be limited to 200 attendees so I suggested putting the Django blog in your feed reader and keeping a keen eye out for ticket announcements. Tickets will be free, but a dontation to Django is always appreciated.

http://code.djangoproject.com/wiki/SprintSausalitoJuly2008

In the mean time check out GNU Common Lisp (CLISP) which just pushed out version 2.46 today:

http://freshmeat.net/projects/clisp/?branch_id=1341&release_id=280569

Check it out: http://www.hivedb.org/

We’ve just published a roadmap and schedule that brings us to a 1.0 release on September 2nd, 2008.

The big feature will be merging the newforms-admin work into trunk… awesome, can’t wait.

Here is the HBase Jira issue: https://issues.apache.org/jira/browse/HBASE-636

Simon Wilson of OpenID and django-openid fame wrote a post on logging in Django and got me thinking a long these lines.

App.yaml sucks. Why? Because it’s not really “pythonic” and I can’t write little bits of python code to make configuration any easier.

For example if wanted to stick and API key in settings.py and wanted a different one for development I could easily do something like

if DEBUG:
  MY_KEY = '123'
else:
  MY_KEY = 'ABC'

Unfortunately Google decided to go with a stricter separation of concerns — at the cost of flexibility.

This is also partly what is wrong with Java. When is the last time you configured a serious Java applications without editing *atleast* one (if you are lucky) xml file (or more if you are not).

http://groups.google.com/group/django-developers/msg/c2be209a9f10ff30

The goal of the queryset-refactor branch was to add a layer of special sauce to the Django ORM and make it easier (possible?) to (among other things) have model inheritance.

Here is a link to the Queryset Refactor Wiki

What has me really pumped is the idea of Django 1.0 being that much closer.