10 avril 2011

A very simple HTTP server with basic MVC in Python

This is a simple proof of concept of a very simple HTTP server implementing a very basic MVC model (actually only routing and controller) in Python.

We will use the BaseHTTPServer as the web server.

The controller base class


All the controller needs is a reference to the BaseHTTPServer so that it can access the web request and write out the response.

class Controller(object):

    def __init__(self, server):
        self.__server = server

    @property
    def server(self):
        return self.__server

The router


The router needs to manipulate the request and response but also instanciate the controllers, that is why it receives an instance of BaseHTTPServer.

The routes are defined by 3 components:

- A regular expression that must be matched to trigger the route
- A controller class to be instanciated
- A method name to be called on the controller

The tricky part here is to instanciate the controller and call a method on it.

class Router(object):

    def __init__(self, server):
        self.__routes = []
        self.__server = server

    def addRoute(self, regexp, controller, action):
        self.__routes.append({'regexp': regexp, 'controller': controller, 'action': action})
        
    def route(self, path):
        for route in self.__routes:
            if re.search(route['regexp'], path):
                cls = globals()[route['controller']]
                func = cls.__dict__[route['action']]
                obj = cls(self.__server)
                apply(func,(obj, ))
                return

        # Not found
        self.__server.send_response(404)
        self.__server.end_headers()


The request handler


All the request handler needs to do is to instanciate the Router and add to it the routes it needs to manage.

This simple request handler is only able to reply to GET requests.

class MyRequestHandler(BaseHTTPRequestHandler):

    def __init__(self, request, client_address, server):
        
        routes = [
            {'regexp': r'^/$', 'controller': 'HomeController', 'action': 'indexAction'},
            {'regexp': r'^/content/', 'controller': 'ContentController', 'action': 'showAction'}
        ]
        
        self.__router = Router(self)
        for route in routes:
            self.__router.addRoute(route['regexp'], route['controller'], route['action'])

        BaseHTTPRequestHandler.__init__(self, request, client_address, server)
    
    def do_GET(self):
        self.__router.route(self.path)

A simple Hello World controller


class HomeController(Controller):

    def __init__(self, server):
        Controller.__init__(self, server)

    def indexAction(self):
        self.server.send_response(200)
        self.server.send_header('Content-type', 'text/html')
        self.server.end_headers()
        self.server.wfile.write('Hello world')


By the way, in MVC a controller should not do View stuff, see "Going further with a templating engine" below to make the separation better using a view layer.

A controller rendering a file


This controller will serve a file in the "public/" directory. Be carefull there is not enough error checking in this code...

class ContentController(Controller):
    
    CONTENT_BASE_PATH = 'public/'

    def __init__(self, server):
        Controller.__init__(self, server)
        
    def showAction(self):
        filename = ContentController.CONTENT_BASE_PATH + self.server.path[9:]
        if os.access(filename, os.R_OK) and not os.path.isdir(filename):
            #TODO: is there any possibility to access files outside the root with ..?
            file = open(filename, "r")
            content = file.read()
            file.close()
            
            #TODO: set correct content type
            self.server.send_response(200)
            self.server.send_header('Content-type', 'text/html')
            self.server.end_headers()
            self.server.wfile.write(content)
        else:
            self.server.send_response(404)
            self.server.end_headers()

Running the server


What we have defined here is a simple http server that will reply "Hello world" when the URL "/" is accessed, and return the content of a file named myFile (if it exists) when the URL "/content/myFile" is reached.

def main():
    try:
        httpd = HTTPServer(('', 8000), MyRequestHandler)
        print 'Server started...'
        httpd.serve_forever()
    except:
 print 'Server shutting down'
 httpd.socket.close()

if __name__ == '__main__':
    main()

Going further with a templating engine


It is quite simple to integrate a templating engine like Cheetah in this code.

Here is a example of controller rendering a template:

class TemplateController(Controller):

    def __init__(self, server):
        Controller.__init__(self, server)

    def listAction(self):
        self.server.send_response(200)
        self.server.send_header('Content-type', 'text/html')
        self.server.end_headers()
        self.server.wfile.write(Template ( file = 'templates/hello_world.tmpl', searchList = [{ }] ))
        return

Conclusion


That's the reason why I love Python so much, in very few lines we could implement basic MVC over a simple web server.

3 mars 2011

Implement your own phpdoc annotations

The goal is to be able to implement our own phpdoc annotations for custom use.

Access the doc block of a method


...
  $method = new ReflectionMethod('MyClass', 'myMethod');
  $doc = $method->getDocComment();
  ...

A note from the php.net documentation about getDocComment():

In order to retrieve a comment with this method, the comment must be imediately before the class, function or method, start with /** and end with */

Does the method have the annotation?


...
  if (preg_match('/@my_annotation/, $doc) {

    // Do something interesting with $method

  }
  ...

Putting it all together


A function that dumps the public methods of a class unless they are marked with the @ignore annotation:

function listMethods($class_name) {

    foreach(get_class_methods($class_name) as $method_name) {

      $method = new \ReflectionMethod($class_name, $method_name);
      $doc = $method->getDocComment();

      if (! preg_match('/@ignore/', $doc)) {
        echo "$class_name.$method_name";
      }      
    }

  }

PHPUnit - detect failing, skipped and incomplete tests

Sometimes when extending PHPUnit it might be useful to test assertions that are supposed to fail or mark the test as incomplete or skipped. However one wouldn't want the whole test case to fail when testing such assertions. Here is a simple way to achieve this.

Please be warned that this technique should be used carefully in order not to "mask" real failing tests.

To illustrate the method let's write a simple test case extension.

class CustomTestCase extends \PHPUnit_Framework_TestCase {

  // Failing assertion
  public function assertMeantToFail() {
    $this->assertTrue(false);
  }

  // Test incomplete assertion
  public function assertMeantToBeIncomplete() {
    $this->markTestSkipped();
  }

  // Test skipped assertion
  public function assertMeantToBeSkipped() {
    $this->markTestIncomplete();
  }

}

Now we would like to write a test case to check if the above assertions actually do what they are expected to do.

The point is that PHPUnit will use Exceptions to notify the test runner of a failing, skipped or incomplete test. Thus it is possible to intercept the correct Exception to check if the test does what we expect.

class CustomTestCaseTestCase extends CustomTestCase {

  public function testFailingTest() {
    try {
      $this->assertMeantToFail();
    } catch (\PHPUnit_Framework_ExpectationFailedException $ex) {
      // As expected the assertion failed, silently return
      return;
    }
    // The assertion did not fail, make the test fail
    $this->fail('This test did not fail as expected');
  }

  public function testSkippedTest() {
    try {
      $this->assertMeantToBeSkipped();
    } catch (\PHPUnit_Framework_SkippedTestError $ex) {
      // As expected the assertion was skipped, silently return
      return;
    }
    // The assertion did not mark the test skipped, make the test fail
    $this->fail('This test was not skipped as expected');
  }

  public function testIncompleteTest() {
    try {
      $this->assertMeantToBeIncomplete();
    } catch (\PHPUnit_Framework_IncompleteTestError $ex) {
      // As expected the assertion was marked as incomplete, silently return
      return;
    }
    // The assertion did not mark the test incomplete, make the test fail
    $this->fail('This test was not incomplete as expected');
  }

}

The above code is meant to work with PHP 5.3 and namespaces. To make it work on other PHP versions remove the \ at start of the class names.

I should not tell you, but...


Alternately you can use the @expectedException annotation to make a failing test succeed. Although this seems to be a very bad practice, because it can hide problems in your code, I present it here for sake of completeness.

/**
     * This test will succeed !!!
     * @expectedException PHPUnit_Framework_ExpectationFailedException
     */
    public function testSucceed()
    {
        $this->assertTrue(false);
    }

This last method does not work for incomplete or skipped tests.

5 février 2011

Enable CTRL-C / CTRL-V in gnome-terminal

gconftool-2 -t str -s /apps/gnome-terminal/keybindings/copy "c"
gconftool-2 -t str -s /apps/gnome-terminal/keybindings/paste "v"

From http://geekybits.blogspot.com/2007/10/gnome-tip-changing-keyboard-shortcuts.html

3 novembre 2010

Playing with HSV colors and HTML5

In a previous post I explained how to draw points on an HTML5 canvas. This time I will use this technique to draw colored bars.

The idea behind this post is to implement a color picker with HTML5 and Javascript.


The HSV color model

HSL and HSV are the two most common cylindrical-coordinate representations of points in an RGB color model, which rearrange the geometry of RGB in an attempt to be more perceptually relevant than the cartesian representation.

Citation from wikipedia
HSV stands for hue, saturation, and value.The hue can variate from 0 to 360°, the saturation and the value must be between 0 and 1. When the full range of these values is walked through, it generates a color cylinder as in the following image:


If a hue is selected (i.e. the main color is fixed), going through the range of S and L will produce a 2D slice of the cylinder.


Converting from HSV to RGB

Using the algorithm exposed in the above wikipedia page we can implement a function that will convert a color given in HSV to a RGB color.
function colors.hsv_to_rgb(h, s, v) {
    var c = v * s;
    var h1 = h / 60;
    var x = c * (1 - Math.abs((h1 % 2) - 1));
    var m = v - c;
    var rgb;
    
    if (typeof h == 'undefined') rgb = [0, 0, 0];
    else if (h1 < 1) rgb = [c, x, 0];
    else if (h1 < 2) rgb = [x, c, 0];
    else if (h1 < 3) rgb = [0, c, x];
    else if (h1 < 4) rgb = [0, x, c];
    else if (h1 < 5) rgb = [x, 0, c];
    else if (h1 <= 6) rgb = [c, 0, x];
    
    return [255 * (rgb[0] + m), 255 * (rgb[1] + m), 255 * (rgb[2] + m)];
  } 

Selecting a color

The idea of the color picker is to show a bar with all the colors (hue).



Once the color is choosed, we have to choose a saturation and value to refine the selection.




Drawing a hue bar

Because we use the HSV color model it is very simple to draw the hue bar. We simply go through the possible values of the hue (0 to 360) using a saturation and a value of 1.
// imgd represents the image data of the canvas object 
  function draw_horizontal_hue_bar(imgd, x, y, width, height) {
    var inc = 360 / width;
    for (var h = 0; h < 360; h += inc) {
      for (var i = 0; i <= height; i++) {
        var rgb = hsv_to_rgb(h, 1, 1);
        point(imgd, x + Math.round(h/inc), y + i, rgb[0], rgb[1], rgb[2]);  
      }
    }
  }

 


Drawing a slice of the color cylinder

Now that we have fixed hue let's draw a slice of the color cylinder in order to choose the saturation and the value. This is also very simple since we can go through the values of the saturation and value (0 to 1) keeping the hue fixed.
function draw_sv_grid(imgd, hue, x, y, size) {
    var inc = 1 / size;
    for (var v = 0; v <= 1; v += inc) {
      for (var s = 0; s <= 1; s += inc) {
        var rgb = this.colors.hsv_to_rgb(hue, v, s);
        this.point(imgd, x + Math.round(v * size), y + Math.round(s * size), rgb[0], rgb[1], rgb[2]);  
      }
    }
  }

 
What's next
With the above algorithm we should have everything we need to implement an HSV color picker.

Some interesting problems remain though, as how to get the color of a pixel clicked in the canvas.

This will be for another post. Meanwhile, happy coding... ;-)

14 septembre 2010

Drawing a point on an HTML5 canvas

Drawing a single point seems to be the easiest task to do on any graphic device. However it's not as easy as expected on a simple HTML5 canvas element. Even if the HTML5 canvas provides advances graphic primitives to draw lines or images, it does not have any function to draw a single point.

In order to draw a single point you'll have to:

  • obtain a portion of the image data
  • calculate the index of the pixel in the image data
  • set the RGB and alpha component of the pixel
  • write back the data to the image

Getting a portion of the image data that can be processed and written back is a powerful procedure that allows for example to write image filters. However it's quite too complicated when you just want to draw one or more pixels.

Here is a real example:

// Get a reference to the canvas element 
  var elem = document.getElementById('canvas');
  if (!elem || !elem.getContext) return;

  // Get the 2D graphic context   
  var context = elem.getContext('2d');    
  if (!context) return;     

  // Obtain a portion of the graphic data
  function get_image_data(context, w, h) {     

    // Not all browsers implement createImageData. On such browsers
    // we obtain the ImageData object using the getImageData method. 
    // The worst-case scenario is to create an object *similar* to
    // the ImageData object and hope for the best luck.     
    if (context.createImageData) {      
      return context.createImageData(w, h);     
    } else if (context.getImageData) {       
      return context.getImageData(0, 0, w, h);     
    } else {       
      return {'width' : w, 'height' : h, 'data' : new Array(w*h*4)};     
    } 
  }

  // Draw a pixel on the canvas   
  function point(imgd, x, y) {          

    // Calculate the pixel offset from the coordinates     
    var idx = (x + (y * imgd.width)) * 4;      

    // Modify the graphic data     
    imgd.data[idx] = 0;     // Red     
    imgd.data[idx+1] = 0;   // Green     
    imgd.data[idx+2] = 0;   // Blue     
    imgd.data[idx+3] = 255; // Alpha channel   
  }      


  var imgd = get_image_data(context, 400, 300);   
  point(imgd, i, j);  // Replace i and j with real coordinates
  context.putImageData(imgd, 0, 0);  

 

Wordle-like PHP script

Introduction

Tag clouds (or word clouds) are very trendy in the fashionable web 2.0.

But very often those clouds are rendered via HTML / CSS and are quite ugly.

I discovered recently the website http://www.wordle.net and was fascinated by the beautiful images it can generate. However they are rendered through a Java applet and the only way to use them is to take a screenshot and extract the image. I wanted clickable tag clouds, and, unless you manually create an HTML image map, it's not possible with Wordle. So I decided to create my own tag cloud generator using PHP.

I started crawling the web to get some information, and I found out an interesting post on StackOverflow from a guy asking how to implement "something like Wordle". Suprisingly, Jonathan Feinberg, the creator of Wordle, replied to the post explaining the basic idea:

Each word "wants" to be somewhere, such as "at some random x position in the vertical center". In decreasing order of frequency, do this for each word:

place the word where it wants to be
while it intersects any of the previously placed words
move it one step along an ever-increasing spiral
That was all I needed to start coding a proof of concept, but a lot of problems still needed to be solved...


Bounding Boxes

In his reply Jonathan Feinberg says: "The hard part is in doing the intersection-testing efficiently, for which I use last-hit caching, hierarchical bounding boxes, and a quadtree spatial index".

Well that was quite too much for me, I needed to find a less efficient but simplier way to test for intersection.

So I came up with this idea:
  1. Each time a word is drawn, store its bounding box in an array
  2. To test if a new box intersects with the already drawn boxes do this:
    • For each bounding box in the array:
      • If the new box intersect the bounding box there is an intersection
This leads to another problem to solve: how to test if two rectangles intersect.


Rectangle Collision Detection

The scipt I wrote only allows to draw words either horizontaly or verticaly. Thus we need to test the collision of axis-aligned boxes. This is quite simple to do.

Two axis-aligned boxes do not intersect when their projection on one of the axis are disjoint. This is not the case for rotated boxes!

if ($box1->bottom > $box2->top) return false;
if ($box1->top < $box2->bottom) return false;
if ($box1->right < $box2->left) return false;
if ($box1->left > $box2->right) return false;

return true; 

 

For arbitrarily rotated boxes you will need some more 2D geometry to test the collision, but for now let's keep it simple.


Searching a place for the new word

We have now all the pieces to write down the routine searching for a free space to draw the new words. We start in the center of the image and move the word along a spiral until it does not intersect with the words already drawn.

$i = 0;  
$x = <image_center_x>; 
$y = <image_center_y>;
while (! $place_found) {
  $x = $x + ($i / 2 * cos($i));
  $y = $y + ($i / 2 * sin($i));
  $new_box = <place the word at x,y >;
  $place_found = <the new word does not overlap with existing words>;
  $i += 1;
}

return array($x, $y);

 

Changing the center of the spiral or its equation will lead to another distribution of the words in the image.

Since the PHP functions to draw text and to get its drawn dimension work with the top left corner as reference point, the above algorithm will tend to place all the vertical words on the left of the image. To prevent this I added a little bit of noize (random numbers) when selecting the center of the spiral for the vertical words.


Clickable images?

As stated at the top of this post, I wanted the generated tag clouds to be clickable. In other words I needed a mechanism to detect which word was clicked.

Since we store the bounding boxes of all the words we draw to detect the collisions, we can use this data to generate an HTML image map.

The problem is that each generation of a tag cloud will generate a different image. This is caused by the noize added when searching for the position of the words, but also to some randomness I added in the calculation of the font sizes.

That means that the tag cloud image and the image map must be rendered and sent to the client in a single call. Unfortunately it is not possible to send back to the client browser an encoded image at the same time as some HTML.

To solve this issue I used an advanced feature of the HTML img tag that allow to embed a base-64 encoded image in the URL.

First render the image in a temporary file and encode its content in base 64:

$file = tempnam(getcwd(), 'img');
imagepng($cloud->get_image(), $file);
$img64 = base64_encode(file_get_contents($file));
unlink($file);
 

Then set the data as the image URL

<img usemap="#mymap" src="data:image/png;base64,<?php echo $img64 ?>"
  border="0" alt="" />
 
Unfortunately this does not work in ... well, as usual ... Internet Explorer... This is out of the scope of this article but you can find more information on how to fix this problem here: Embedding Base64 Image Data into a Webpage

Since we now return HTML instead of a PNG image, we can as well send back the HTML image map.


See it in action

Sorry IE users this wil not work... ;-(



Get the code

The source code of the complete script can be found on GITHub.