Fun with decorators on the Google AppEngine

July 22nd, 2008

Well I thought I’d give writing for the Google AppEngine a try and have found it a delightful platform to work with. Easy deployment, easy scalability (the DataStore non-RDBMS is a bit harder to get your mind around), and a squeaky clean WebOb-based WSGI minimalist framework called WebApp. Anyways, in developing a simple REST service, I ran across a great place to use Python Decorators. I needed to restrict access to a specific REST method based on IP Address so I wrote the following decorator:

1
2
3
4
5
6
7
8
9
def bind_ip(ips):
    def _dec(handler):
        def _check_ip(self, *args, **kwargs):            
            if self.request.remote_addr in ips:
                return handler(self, *args, **kwargs)
            else:
                  self.error(403)
        return _check_ip
    return _dec

The decorator returns a closure which only calls the method it encapsulates if the IP address of the client is specifically allowed. Notice the @bind_ip on line 11

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import wsgiref.handlers
from google.appengine.ext import webapp
from util import bind_ip
 
allowed_ips = set([
    '127.0.0.1',
    'ip.of.rest.client'
    ])
 
class SecretHandler(webapp.RequestHandler):
  @bind_ip(allowed_ips)
  def get(self, secret):
     """ Do something Secret """
     self.response.out.write (secret)
 
def main():
  application = webapp.WSGIApplication([
          (r'/secret/(.*)', SecretHandler)
      ])
  wsgiref.handlers.CGIHandler().run(application)
 
if __name__ == '__main__':
  main()

Decorators are a simple way to easily annotate how functions should be called, and can be used for Authentication, Data Validation or however else you’d like to use them.

An illegal function

May 1st, 2007

Can a function be illegal? Thanks to the blessings of interpolation and of Mathematica (Wolfram just happened to release version 6 today), I present the HD-DVD AACS Processing key interpolated into a polynomial equation! :)

the Illegal Function

AjaxTrans rewritten in Mod-Python

March 4th, 2007

After many months of dissatisfaction regarding PHP (especially its Unicode Support), I decided to rewrite AjaxTrans in mod_python. A language which has allowed a much greater flexibility in the area of character sets. This transition has allowed me to incorporate 4 new languages into AjaxTrans:

  1. Chinese
  2. Japanese
  3. Korean
  4. Russian

Enjoy!

PS: Comment on any bugs/errors/interface mistakes (The spanish interface translation is likely full of errors as it’s written by a student who’s only currently taking Spanish… in High-School—me) feedback much appreciated, also translations of the interface into your language would be greatly welcome.

From a self-confessed bandwidth-o-holic…

March 4th, 2007

This is my first blog, and coincidentally my first blog post, so please forgive me if I am completely irrelevant, make horrendous errors in nature of fact, grammar or spelling or even abuse commas by creating very long winded introductions.

Anyways, just about a week ago I noticed my server had slowed to a crawl. Not just any crawl mind you, but like that slow-as-molasses kind of crawl which makes you want to scream. I quickly opened my trusty copy of PuTTY and ssh’d into the server. PuTTY was configured to automatically send my username to the server and wait for “keyboard-interactive authentication” to enter my password. I waited for about five seconds before the password prompt appeared, I dutifully entered my password and hit the enter key. Nothing… Waiting… still nothing and then finally

Last login: Mon Aug 21 13:43:39 2006 from 68.68.*.*
www ~ $

Curious as to the cause of such slowness in my precious 3GHz P4 Gentoo -O3 -funroll-loops -fomit-frame-pointer -fgo-uber-fast server, I prodded around a little bit. After checking top, CPU usage wasn’t the culprit, memory was fine, EUREKA the apachetop was showing that the connection was bottlenecking at only 6 KB/s. Had someone switched the colo’s OC-3 for a Dialup connection? The next day I took a visit to the colo and visited my cute-little-rack-mounted-box did a diagnostic from there, only to find the same results: 6KB/s. This was driving me crazy, I couldn’t even transfer a day’s log file, let alone serve more than 2 clients with my bandwidth intensive translation application. Turned out some backhoe operator hadn’t “Dialed Before he Dug” and had cut right through a fiber-optic line which my server depended on the traffic was being routed through an alternate connection which was did not have nearly enough capacity for the event.

So this got me thinking, what is life like without broadband. I hadn’t used a dialup connection in years, much less used a dialup connection to connect to the bandwidth-heavy Web 2.0 goodness which dominates the current incarnation of the web. I wondered what my AjaxTrans application looked like from the eyes of its dialup users. Whipping out my trusty web-development swiss army knife, the Firefox Web Developer Toolbar, I opened the tools menu to see what was there. Ahh, “View speed report” that looks nifty, I click and this is what I find:

Connection Rate Download Time
14.4K 86.79 seconds
28.8K 43.50 seconds
33.6K 37.31 seconds
56K 22.47 seconds
ISDN 128K 7.02 seconds
T1 1.44Mbps 0.79 seconds

23 Seconds! That must be unbearable. I quickly read the report to see what I could do to change this enormous amount of loading time, and I saw that it was loading 7 Scripts externally in addition to the images and CSS.

After asking some friends what they would do to alleviate the problem, one suggested that I use two tools:

  1. The Dojo Toolkit Compressor
  2. JSMin

I flattened the seven scripts into one file, streamlined.js and with one command I was off:

java -jar custom_rhino.jar -c streamlined.js | jsmin > compressed.js

I had reduced the file size by 50%, but I wasn’t quite done yet. I used an old trick from when I did code for dialup connections… gzip. After editing my .htaccess

<Files ~"\.js$">
ForceType application/php
</Files>

I added

<?php ob_start("ob_gzhandler"); ?>

to the beginning of compressed.js and squeezed another 16% size out of it.

The final results:

Connection Rate Download Time
14.4K 26.61 seconds
28.8K 13.40 seconds
33.6K 11.52 seconds
56K 6.99 seconds
ISDN 128K 2.28 seconds
T1 1.44Mbps 0.38 seconds

The comparasion:

Just under 7 seconds! Yay, I reduced the load time by a factor of pi! How nerdy… well a factor of 3.147 anyways, a 66% size reduction.

You have no idea how much I sympathize with you dialup users.