Courtesy of news.com
Sun has announced it is partnering with Amazon’s EC2 cloud computing service to offer (Open)Solaris on EC2 instances… I wonder if this will pave the way for BSD (Free, Open) derivatives to be officially supported by Amazon…
Filed under: Virtualization | No Comments »
Django Developer (and the man behind djangosnippets.com) posts his view of the “must have” list to work towards for Django 1.0
http://groups.google.com/group/django-developers/msg/c2be209a9f10ff30
Filed under: Programming | No Comments »
The Django team has knocked off one of the milestones for the 1.0 release — Changeset 7477 merges the queryset refactoring effort into django-trunk.
The goal of the queryset-refactor branch was to add a layer of special sauce to the Django ORM and make it easier (possible?) to (among other things) have model inheritance.
Here is a link to the Queryset Refactor Wiki
What has me really pumped is the idea of Django 1.0 being that much closer.
Filed under: Programming | No Comments »
As I previously mentioned here: (”Debian packages for solr in testing“)– apache-solr is now in the main Ubuntu / Debian repositories. Once you’ve upgraded to Hardy Heron (good how-to here) you can install solr with the following command:
apt-get install solr-tomcat5.5
Note, I’ve already noticed one bug with the get-properties link in the solr admin and created a bug report on Launchpad: https://bugs.launchpad.net/ubuntu/+source/solr/+bug/218419
Filed under: Linux | No Comments »
Jan Rain, the company behind MyOpenID has released an ajaxy/html widget for sites that takes care of the front end integration called the “ID Selector”
To get an overview of the bigger openid brokers out there check out: provider comparison
Simon Wilson one of the OpenID developers and Evangelists put together a django project called django-openid that provides some middle way to enable the openid field on request objects (request.openid)… I believe it’s only OpenID 1.0, and not 2.0 — but for reference check it out here anyway: django-openid
Filed under: Programming | No Comments »
Here is the link to the main annoncement page, and below you’ll find links to the projects I care about the most
Filed under: Programming | No Comments »
HA! The google team have answered a lingering question about data portability. I’ve been vocal about my reservations with the GAE platform. The fact that people have ported the GAE python framework didn’t really mitigate my concerns. All code is throw away code anyway, the data is what is where the value is.
These two announcements from the GAE team have allayed some of my fears.
Getting your data on and off google address the issue of data portability, specifically importing and exporting large amounts of data.
The GAE Team has also released a Helper for Django that should smooth over some of the rough spots in the GAE/Django bridge.
Exciting times. My biggest remaining gripe with GAE is that one single Apps account can not host GAE instances on domains other than the main domain — but hopefully that will change sometime soon.
Filed under: Virtualization | 1 Comment »
Spock Proxy , developed at Spock.com seems like a mysql-proxy work-alike except that it makes sharding or horizontal partitioning of the database easy and transparent (or at least that is the idea).
The Spock Proxy project page mentions that Spoke is built on Rails, and ActiveRecord, but from the description it seems like this could be transparently applied to other frameworks that make it hard to split up database requests (Django, I’m looking at you).
Filed under: Programming, Startups | No Comments »
I’ve started a new project, an apache module, called mod_otter that will hopefully alleviate some problems involving with some popular and widely deployed software.
I’ll be hacking away over the weekend, a mini and initial development sprint.
More details soon, check back soon. I think I’ll make http://yousefourabi.com/mod_otter (or something similar) the official blog of the project
Filed under: mod_otter | 1 Comment »
Werner Vogels, CTO of amazon has just written about Amazon’s latest feature addition: persistent storage for amazon ec2 — This recent announcement come a few weeks after Amazon announced static IP’s (Elastic IP’s) and Availability Zones — or the ability to specify the location of an instance on creation.
Persistent storage for Amazon EC2 will be offered in the form of storage volumes which you can mount into your EC2 instance as a raw block storage device. It basically looks like an unformatted hard disk. Once you have the volume mounted for the first time you can format it with any file system you want or if you have advanced applications such as high-end database engines, you could use it directly.
Reading the post, what the technology sounds like is some sort of home grown SAN — there are however some limitations — the storage device can only be mounted by one instance at a time, and more annoying is only available from one availability zone. However, one nice and unexpected feature is the ability to store snapshots of the volume to S3 and then create volumes in other Availability Zones from that snap shot.
I think it is an important step forward, and the pace of development at Amazon is impressive… but I’m really getting annoyed by features that are missing something, or have some constraints. What gets to me the most is that only one instance and mount one volume at any given time — a truly distributed file system that allowed multiple running instances to use it concurrently would really blow me away.
I guess the context of this is really a database server, and you would only have one storage device per zone to correspond to each mysql-slave per zone… Another scenario would be an Apache SOLR master instance would use the volume as the persistent storage for the Lucene Index and replicate out to the Slaves that would just store on the transient EC2 drive.
Filed under: Virtualization | 2 Comments »
The April 11th business week article titled: “Apple’s OS Edge is a threat to Microsoft” is just full of fail. I mean really, they should have done a little peer-review to make sure what they wrote was in the general ball-park of technically accurate According to them OS X pioneered SMP systems…
it all started with Mac OS X, the multi-core, multi-processor platform officially released in 2001. Based on “Mach,” a university UNIX research prototype, Mac OS X represented a clean break with the computer industry’s uniprocessor past. The modular new OS allowed Apple to condense its core task management function into a tiny computing kernel.
Filed under: Uncategorized | No Comments »
I’ve just started to start playing around with Google App Engines, and almost immediately I’ve ran into an annoyance.
If you want to host your app engine on it’s own vanity domain (myapp.com) you must have your google apps account registered exactly on that domain, you can host engines w/different domains than your main google apps account…
Lame…
Filed under: Virtualization | No Comments »
Update: It’s very clear to me that the Google App Engine is modeled after Django… See below
Huge win for Django! It will be interesting to see how this plays out on one hand you have heroku the all in one IDE/Enviroment for Ruby On Rails that is backed by Amazon AWS and auto-deploys for you (to the Amazon cloud) and on the other hand you have Python, Django and Google App engine with a similar but different model.
I’m firmly on the Python side, and thrilled that my favorite framework is going to be run on Google’s cloud offering.
I’ve already gotten my account activation (see below) but there seems to be a glitch and it just forwards me to the waiting list. I’ve downloaded the App Engine SDK… but won’t play with it until the weekend.
Update One: After watching the introductory video, and reading through some documentation, I’m noticing some very clear patterns modeled after Django. Particularly the Database models, and the “config” yaml file which is equivalent to settings.py (Django). See the intro video here: http://www.youtube.com/watch?v=bfgO-LXGpTM
Hello,
Thanks for signing up to try Google App Engine! Your account has been activated, so you can begin building applications!
To start creating applications with Google App Engine, simply follow this link (you may need to sign in with your {{ removed }} Google Account):
Thanks!
The Google App Engine Team
Filed under: Programming, Virtualization | No Comments »
Marketcetera, the “open source” Automated Stock Trading has tickled the funny bone of the blog hype machine…
All the other cool kids(Ostatic, VentureBeat, The Open Road) are blogging about “Marketcetera” so I’ll yield and follow along…
The platform which is currently at the 0.4.2 release — 1.0 scheduled for Q3 2008 uses the “FIX” protocol to execute automated trades with brokerages and banks.
Marketcetera has received a 4 million dollar A round from Shasta Ventures and Jack Selby (Clarium Capital (hedge fund)) — the general plan is to start charging for data that will be helpful to the analysis the big boys (hedge funds) do.
Oh, right, that open source thing. I started poking around on the site, and on the left navigation there is a link to “Contribute”… sounds promising… I was redirected to a page with a license agreement… and things get interesting… I was presented with the following license agreement (see below, link here for full version: http://www.marketcetera.com/contributors/display)
I’m wondering… this doesn’t sound like the licenses I’m used to, this clearly isn’t OSI ratified yet… is it fair (and legal) to call this open source (Disclaimer: not a lawyer)
This is the current version 0.1 of the Marketcetera Contributor License Agreement, as of March 23, 2007.
Summary (see below for full license):
The Marketcetera Contributor License Agreement (CLA) means, in simple terms, that:
Filed under: Business, San Francisco, Startups | 2 Comments »
Continuing my HTTP bender, I’d like to discuss some fun / necessary things you can do to manipulate HTTP headers using the Apache mod_headers module.
If you were to make a request to Slashdot and examine the Http headers using either HttpLiveHeaders or FireBug you would notice one of two unusual headers: X-Bender, and X-Fry with quirky messages. These are pop-culture references to the show “Futurama“.
There are a few ways you could accomplish this — if you were already working with mod_perl, mod_python, or PHP it would be trivial to manipulate the headers programatically. However, it’s also possible to do so with a simple one liner in your Apache conf.
First you’ll need to enable mod_headers — On my Debian server this was done with the following command: a2enmod headers, after which I had to restart Apache.
Then simply add something like the following line in your conf, of course replacing “HeaderNameHere” and “Header Value” with whatever you want.
Header add HeaderNameHere "Header Value here"
The “Header” directive has a number of possible actions including: set, unset, append, add and echo. The documentation is fairly well written and can be found here: mod_headers documentation
One useful application would be in a load-balanced apache setup, you could set the actual hostname as a header for debugging purposes. You can echo environment variables like so: %{FOOBAR}e which would display the “FOOBAR” variable.
Have fun tweaking Apache headers, hope this helps.
Filed under: Semantic Web | No Comments »
Search.com has just launched a redesign — very cool. Check it out: http://www.search.com
Filed under: Semantic Web | No Comments »
I try not to re-hash content from other sites, but here’s a link that is too tasty to pass up on
15 handy wordpress plugins (for power users)
Filed under: Semantic Web | No Comments »
Dear Digg,
I’m a huge fan. I check our site a few times a day. However, there is one thing that has really been bugging me. The maintenance page. Whenever you throw up that oh-so-useful page full of links to the favorite sites of your employees… you return an 200 status code.
Think of me not as a big fan, but rather an eager web-crawler from Google or some new startup. When I find my way to your page, and your error page is return a 200 Status (OK) — I take it to mean that everything is hunky dory, and this content is really what you are all about.
Let me school you on this radical bit of information: You can return other status codes — such as 500 — and return a pretty html document containing all the links to your favorite sites.
Speaking of documents…
There is one document in particular I would like to refer(get it…) you to: rfc 2616. Section 10 is particularly relevant titled “Status Code definitions.” The most relevant among this list are the following status code: 200, 304, and 503. I mention these because there seems to be a certain level of confusion among your engineers.
200: OK — This is what we meant to serve out, and feel free to associate this context with our URI
304: Not Modified: A cache setting, meaning “No Update” use cached version
503: Service Unavailable: Bingo! Doesn’t that sound like something we would want to return as our status when we have our maintenance page up? For extra browny points, consider sending Retry-After header with some amount of time…
Here is the output from Live HTTP headers on one of these requests.
Thanks again!
Yousef
GET / HTTP/1.1
Host: digg.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13)
Gecko/20080311 Firefox/2.0.0.13
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plai
n;q=0.8,image/png,*/*;q=0.5
Accept-Language: en
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
HTTP/1.x 200 OK
Date: Mon, 31 Mar 2008 23:38:53 GMT
Server: Apache
Last-Modified: Mon, 31 Mar 2008 20:52:50 GMT
Etag: “714c87-15e9-449c1d5b88c80″
Accept-Ranges: bytes
Keep-Alive: timeout=5, max=10000
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
Cache-Control: private
Content-Encoding: gzip
Content-Length: 2069
———————————————————-
http://digg.com/img/down.png
GET /img/down.png HTTP/1.1
Host: digg.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13)
Gecko/20080311 Firefox/2.0.0.13
Accept: image/png,*/*;q=0.5
Accept-Language: en
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://digg.com/
HTTP/1.x 200 OK
Age: 2027
Date: Mon, 31 Mar 2008 23:04:19 GMT
Connection: Keep-Alive
Via: NS-CACHE: 100
Etag: “720139-89b-41414d62cbc00″
Server: Apache
Last-Modified: Thu, 18 May 2006 19:13:52 GMT
Accept-Ranges: bytes
Content-Length: 2203
Keep-Alive: timeout=5, max=9998
Content-Type: image/png
———————————————————-
http://digg.com/favicon.ico
GET /favicon.ico HTTP/1.1
Host: digg.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13)
Gecko/20080311 Firefox/2.0.0.13
Accept: image/png,*/*;q=0.5
Accept-Language: en
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
HTTP/1.x 200 OK
Age: 2932
Date: Mon, 31 Mar 2008 22:49:13 GMT
Connection: Keep-Alive
Via: NS-CACHE: 100
Etag: “2884541-47e-418dfb6884480″
Server: Apache
Last-Modified: Tue, 18 Jul 2006 18:57:06 GMT
Accept-Ranges: bytes
Content-Length: 1150
Keep-Alive: timeout=5, max=9997
Content-Type: image/x-ico
Filed under: Semantic Web | No Comments »
I’ve used VMWare server, VMWare Workstation, and VMWare fusion (Mac OS X desktop client). Recently I’ve been thinking about creating my own VMWare appliance as a development environment for a project.
After some initial research I realized there was nothing special about so called “Appliances” they are just like the VMWare instances I was already used to creating — except that they are meant for other people.
The easiest way to get started is with the help of a website called “EasyVMX” is a simple web-application that creates the .vmx configuration file for your VMWare appliance. It’s fairly straight forward, and you can specify if you want the appliance to use an ISO image or physical CDROM (two of the most typical install sources).
Once you’ve filled out the form, you can then launch it with the free VMWare player. Make sure your installation source (ISO or CD) is in place (right path to file, media in drive…etc) — and there you have it, you’ve started creating your vmware appliance for free.
You could always sign up for a free 30 day trial of VMWare workstation, but unless you are ready to fork over $180 you would end up moving to VMWare player anyway — might as well start there.
Filed under: Virtualization | No Comments »
Amazon has finally launched two features that have prevented its EC2 service from really taking on conventional web hosts: Static IPs (aka Elastic IPs) and “Availability Zones.”
The “Elastic IPs” come with a little “Amazon special sauce” and can be re-assigned to your EC2 instances programatically on the fly. There are some initial limits, currently you can only request up to 5 “Elastic IPs” per account.
“Availability Zones” allow you to specificy the (abstract) location of the physical machine that will host your virtual instance. You can for example specify that your two ec2 instances should be created in two separate datacenters, geographically separate enough to satisfy disaster recovery plans…etc
RightScale, a company that provides a management interface (written in Ruby on Rails) to Amazons infastructure has a good blog post on the new changes here: “Setting up a fault tolerant site using amazons availability zones“.
Filed under: Startups, Virtualization | No Comments »