The two real competitive advantages I see are the pricing points (lack of contract, quick setup) and more technically that it eliminates the need of an origin server.

That last point is key — you’ll hear things like: “Cloud front doesn’t have the ability to pull from an origin server, there for it is a big joke that can’t compete with Akamai, LimeLight…etc”.

This is bogus.

In most typical CDN setups that I know there is what is called an ‘origin’ server, the server where you continually host the content you want pushed out to the delivery network. As requests come in for specific assets most CDN providers will ‘pull’ the content off of your server by convention (specific URI/path) and publish to their cloud.

The problem with this setup?

It’s so not cloud.

You are forced to maintain a perpetually running server, with enough storage for all the assets, which sits there slurping up space, electric and maintenance (admin) fees

Since CloudFront is built on the S3 storage service — S3 is in essence the origin server. In fact CloudFront is merely an S3 bucket that has been blessed into a ‘distribution’ via a simple RESTful API call…

That’s the origin of cloudfronts competitive advantage.

Having worked with… less enlightened solutions… I’m thinking the ultimate CloudFront deployment scenario would be via an SVN or GIT hook. If you commit an asset to /static/css or /static/css a post-commit hook would automatically publish to CloudFront.

If I have time I’ll take a look at implementing something quick and dirty tomorrow.

Simon Wilson presented dmigrations . Installing dmigrations is as simple as installing it in INSTALLED_APPS, and it will registers a few custom admin commands:

./manage.py dmigrate app APP_NAME

./manage.py dmigrate list

./manage.py dmigrate addcolum

My take: dmigrations is great and will work for migration problems right now, but in its current form is unlikely to end up in django as the annointed migration solution. Why? Because it basically wraps SQL directly, loosing some of the cross-database portability of the Django DB.

Andrew Godwin: South — described as the next step of dmigrations. Philosophy: Migrations are essential, branched development / missing migrations, inter-app dependencies, database abstraction needed too. Can handle model dependency (foreign keys.)

My Take: worth looking into, more database indapenent.

Russel Keith-Magee: Django-evolution, his response to a mailing list thread that would not end. Google Summer of code 2006. Russel complaining of Magic moniker attached to django-evolution, everything done via introspection of models. Goals: Hint and Tweak, Simple changes without user intervention, easy entre for customization, raw sql, self documenting, self auditings, Validation (where possible).

Signature: pickled summary of django model. Stores state of django models at syncdb, can be diffed against current models, diffs used to generate hints.

Mutation: Atomic unit of change, common operations built in. Can be user-defined, can be raw sql, know the effect they will have.

Evolution: Ordered collection of mutations. Two flavors of mutations: hinted and stored. Executed evolutions stored in database.

Hinted Evolution: Best guess by looking at diff, if acceptable can be used to execute evolution right away. Can also be used as prototype to stored evolution. Can’t resolve ambiguous updates (rename) can’t fill in the blanks (initial data)

Stored Evolution:
Named sequence of mutations. Defined per application, stored in evolutions directory of app (can be put into version control).

django-evolution extends syncdb — as in ./mange.py syncdb — schema change detected you need an evolutions.

Custom commands:

./manage.py evolve –hint

./manage.py evolve

My Take: Highly interesting, most likely to end up in django. Grok this.

After reading around 6000 bug tickets, certain patterns have emerged in the submitted patches…

Code Quality Matters — funny quote from Leah Culver: “Have you every written a library? It’s like people seeing you in your underwear. You gotta make sure it’s clean.”

Do the basics properly. The word “print” is in your patch. Similarly for “import pdb”. If you think PEP 8 is the name of a new energy drink. Run the django testsuite. Submit patches, and not entire files. Start from top of tree when using svn diff.

Read the contributing document (contributing.txt), use sensibly names, comments should last, comments should be correct.

Fix problems, not symptoms. The crowd is smarter than you think — the code mostly works. If you find yourself ripping out lots of code, stop and think.

The code for the ORM is located in django/db with juicy bits in the following locations:

django/db/models/query.py (public queryset API)

django/db/models/sql/* (Public API->SQL conversion. Deep dark internals. Does’t know DB, knows SQL)

django/db/backends/* (Individual DB wrappers, third-party wrappers possible. This is where you actually talk to the DB). See (dummy or mysql dir for examples)

The different pieces in depth

Blog.objects.filter(owner=user) = [model, manage, QuerySet method]

Model Manages:

inherit from django.db.models.Manager, ideal for extra methods that act on the whole table at once, not just one record, usually wraps/proxies public querySet methos, has method called get_query_set() which returns QuerySet (QuerySet.all)

QuerySets:

django.db.models.query.QuerySet

Every time you call a method on a QuerySet it returns a copy (clone) of that QuerySet (side effect free?) For example every time you call filter on a query set you get a different queryset.

Query

django.db.models.sql.query.Query, an attribute of QuerySet, holds the internal state of the current query, knows how to produce SQL. This is where you implement something that knows how to speak to MySQL, or CouchDB or Hadoop for example. (Aka class from Hell, almost every data-structure known to man used here). AsSQL is when state is rewritten into SQL. This is a change from before QuerySet Refactor landed in trunk where the internal representation for everything was strings.

Every method of QuerySet updates the Query (calls method on Query).

Query.results_iter() is how the results end up from the DB back into Python objects, reconstructing Python objects happens in various other Query methods: Query.select_related, Query.extra_select…etc

Looking inside the Query. Query.add_filter() is the guts — the place to start when trying to grok the Query class. Query.setup_joins() converts filter/exclude into table joins. Query.join() responsible for joining a pair of tables

Recap on flow/organization: Manager->QuerySet->Query

Customization:

Custom Managers (Easy)

Custom QuerySet (Not so Easy)

–> (Some example QuerySet) ValuesQuerySet, DateQuerySet, EmptyQuerySet

Custom Query (Not so Easy)

GeoDjango for example has to use very different types of query.

Note to self: I’m wondering how(or even IF) one would  use Django’s ORM for a Solr-backend for intergrated search via a DJango ORM model.

Review-Board is a python/django based code-review application. In 2004 VMware was approx. 600 employee’s and already had a rigorous code-review process in place. At first simple HTML snippets were emailed around. Fast forward to 2007 and the company had grown to 5000 employee’s, and the previous system of email around snippets was unmanageable.

And thus Review-Board was born.

Earlier in the day Guido mentioned that he wrote a similar, python based code review tool that is used internally at google. I’ll try to dig that up and make a follow up post.

Stats: 2 core developers, 74 contributors, 295 mailing list members.

I’ll be installing this on my own dev box — In the future expect a post or two about how it intergrates with other tools I like (git/svn, trac…etc)

The SAN like storage mounts are (only) available in a given “Availability Zone”. Nothing specific is mentioned about distributed file systems like GFS, so that is something I’ll be looking into ASAP. Like a regular volume you can create several EBS volumes and do your own software raid on top of that.

There are also some interesting scenarios around the snapshot facility such as having one “master” EBS volume in the designated “write” availability and replicating out to several “slave” availability zones in geographically disparate regions.

Performance seems inline with high end RAID arrays:

To recap:

Price: $0.1 per GB
Performance:
Features: Reliability, Snapshots, Traditional Posix filesystem

Earlier in the week I received “Practical Django Projects” in the mail — chomping at the bit for enough time to get into it.

Exciting times.

I wonder if there isn’t some room in the Django ORM back-end to support stuff like S3 and Google App Engine — with some warnings of course that the store isn’t relational… etc

The page “Intense Debate” seems to be about some sort of video blogging system. See screenshot:
What is going on?

Update: redirect is no longer happening as of 7:06 pm PST.

At first I thought it was a perl specific choice, but a few quick tests with C and Python confirmed this was not the case.

It seemed like an odd design choice given that truthiness in this (programming) context is quasi-inspired after the presence of current in a given transistor. So why wouldn’t you want negative integers to be false?

Then I started thinking that it was either an optimization issue — so I started looking at the x86 assembly generated by GCC for my C example (see below). I used the -S flag to gcc to save the assembly so gcc -S test.c created the test.s (assembly). A quick glance revleaed that this was indeed the case, particularly the following three lines:

  movl  $-1, -4(%rbp)
  cmpl  $0, -4(%rbp)
  je  .L2

movl moves the value of X (-1) to the appropriate place (this is int x = -1) then the actual comparison happens with the cmpl operator which compares that value to zero. If those to values are equal (0 and X) the je executes (jump-if-equal) and executes that branch.

Now the next question, for another day, why is gcc assembly ifs to JE and JLE — which would return false for anything less than 1 for example.

My guess is because it’s more expensive. But more to come on that subject.

perl:

#!/usr/bin/env perl
if (-1) {
  print STDERR "I am true \n";
}

python:

#!/usr/bin/env python
num = -1
if (num):
  print "I am true"

C:

#include  

int main (void) {
  int x = -1;
  if (x) {
    printf("I am true");
  } else {
    printf("not true");
  }
}

It will be hosted the weekend of September 6th and 7th at the GooglePlex (Mountain View, CA). Since this is the first conference (and hosted at Google) space will be limited to 200 attendees so I suggested putting the Django blog in your feed reader and keeping a keen eye out for ticket announcements. Tickets will be free, but a dontation to Django is always appreciated.

http://code.djangoproject.com/wiki/SprintSausalitoJuly2008

The S3 manage allows you to create US or EU buckets and drag and drop stuff from your desktop. The EC2 component comes with an AMI browser and built in SSH client.

Immediately I found a few bugs. The S3 component was’t displaying any buckets (that I knew I had) and instead displayed and error “Invalid Argument”. So I guessed that perhaps my credentials had a space in them so I deleted the account and created a new one. However after entered the Access and Secret key, it now says “License invalid”.

I hope I can figure out the “License invalid” issue so I can actually try the product.

Jollat is a free download, but only for 30 days.

In the mean time check out GNU Common Lisp (CLISP) which just pushed out version 2.46 today:

http://freshmeat.net/projects/clisp/?branch_id=1341&release_id=280569

Congrats!

Check it out: http://www.hivedb.org/