Monday, October 26, 2009

Rdbhdb v 0.9

A new version of Rdbhdb is available from Pypi. Rdbhdb is the DB API 2.0 module for accessing Rdbhost databases remotely from Python applications.

Rdbhdb v 0.9 download page on PyPI
Rdbhdb v 0.9 download page on Rdbhost

New in this version:

  • Uses gzip decompression, optionally, for data downloads from server.
  • Supports the sending of binary data (as buffer datatype) in query parameters.
  • .execute_deferred() method provided to execute queries in deferred mode. Deferred mode does not return results, but allows a more generous time limit for query completion. Useful for updates or inserts that might need extra time.
  • .nextset() method implemented on cursors. Multiple queries can be aggregated into a single .execute() call, delimited by ';'. One result set is returned for each query, and .nextset() advances to the next result set in the returned list.
  • .https attribute added to connection, to force use of SSL. Defaults to False, no SSL.

This version is the one currently in use on rdbhost.appspot.com, for its monitoring function.

Edited 28 Oct 09: Added rdbhost hosted download link.
Edited 9 Nov 09: Spelled PyPI properly

SSL Downtime

I was told yesterday by a site user that www.rdbhost.com's SSL certificate was expired.

For the moment, the secure site of www.rdbhost.com is down, and will be back up later today when I have time to get the certificate updated and installed.

David Keeney

As of 8PM on Tuesday, SSL is back up and working.

Friday, October 23, 2009

User side testing and Twill

History

RdbHost has two faces: one that the human user sees, and one that their software sees. The software side, the 'web service' has always had good testing. That it was designed to be computer parse-able without screen-scraping meant it was easy to write automated tests for it. There are in fact two sets of web service tests, one that uses urllib2 to query directly, and another that tests the DB API module Rdbhdb by using it to query the server.


Website Testing

Until recently, the human-readable (aka the web site) side of the business was not tested in any automated way. I would test features I thought might be affected by a change, manually, after an update. I am not sharing any details, but that strategy has proven to be deficient. Now we have a test suite for the user side as well.


Twill

The software used for the new tests is Twill. Twill can be run as its own scriptable shell, or included as a python module and the functions called from python. The Twill python module has an instantiated object syntax, where you create a browser object and call methods on it. Unfortunately, the browser object does not support the 'show' method. I ended up giving up on the explicit browser object, and just used the implicit browser. It works well, but doesn't satisfy my 'explicit code is more comfortable' preference.



Tests and Email

Many features of the website use email for various authentication functions. To reset your password, the site emails a password reset-link. To change your email address of record, you have the site email a change link to the new address. Creating a new account involves emailing the initial login password to the submitted address. How does one test such a system with Python and Twill?

My solution was to use one of the spam-dodging sites that allow one to receive email anonymously. Just email to 'anyname@spamspot.com' and spamspot.com will post the email content to a blog style web page, intermingled chronologically with other user's emails. The test code can request an email, for whatever functionality being tested, give the site a spamspot.com email address to mail to, wait a few seconds for the email to arrive and be posted, and then use Python and Twill to request the spamspot page and parse out the rdbhost email and its embedded password or links using 're' module. It is appropriate (if also a little paranoid) to change the password shortly thereafter to minimize tampering by other readers of spamspot.

There are many anonymous email receiver websites. Spamspot.com (mentioned above) and Makemeaking.com are more useful for our purposes than some, because they include the email inline to the page, without requiring another link-following step, and the prerequisite link parsing step.



Edited 24 Oct 09: added reference to Makemetheking.com
Edited 26 Oct 09: minor grammatical corrections