More changes to Rdbadmin and Rdbhost

A few recent improvements to Rdbadmin:

• Main page shows tables and fields.
• New section of left-hand menu shows schemas.
• Menu options create and rename schemas.

Improvements to Rdbhost:

• Login page (and login link from front page) is https.
• Login cookie is 'secure' so will not be available to non-https pages.
• Attempting to access another account while logged into yours will result in an error. You must log out of your account to access any other.

Improvements to Rdbadmin program

Rdbhost sports its own admin utility, Rdbadmin, implemented in Javascript. It is useful to create and remove tables and views, adding and deleting data from tables, and performing queries.

Rdbadmin has been upgraded, with both new features and bug fixes.

New features include:

• Table and view lists are sorted.
• Tables and views can belong to schemas other than public. This will become more important with upcoming features.
• New feature to set field defaults on tables.
• HTML content is now escaped, so that it reads as the tags themselves, and the data inline formats cannot interfere with the display table structure.
• In the select page, query data values are parameterized, so the data values cannot be misinterpreted by the SQL engine as statements.
• The login and logout functionality has been removed. Logging in is done via the www.rdbhost.com front page (or the www.rdbhost.com login page), and logging out from the profile page. The admin script gets role and authcode from the rdhost member cookie, and just bounces the user to the login page if member cookie is missing. The 'guest_account' functionality is still present, where the 'r'-role can be provided on the url.

As always, the admin program is linked from the profile page. As always, your questions and criticisms are welcome.

Guest usage of Rdbhost accounts

There is a handy way to provide other people read-only access to your Rdbhost database.

If you have enabled the 'r'-role, and given it reasonable (ie: only SELECT) permissions on the tables, you can allow other people to access your database using that role.

Just use the url for the rdbadmin page, and append the 'r'-role name to the url, with a '?'.

For example, our stackoverflow database is account 'db0000000767', and the 'r' role has been enabled and GRANTed appropriate privileges. The guest usage url is thus:
http://www.rdbhost.com/rdbadmin/main.html?r0000000767 .

Try it here.

There is no way to provide an authcode with the rolename, so it will not work for 's'-roles.

YASOP: Yet another Stackoverflow post

The latest data dump from Stackoverflow has been placed in our stackoverflow account

See: http://www.rdbhost.com/rdbadmin/main.html?r0000000767

I was just reading some old stuff on meta.stackoverflow.com about hosting a publicly query-able copy of the dump, with reporting or graphing options.

This account (at the above url) provides the data, with the flexible querying of SQL, Google's Appengine provides some charting tools, and our Rdbhdb module provides a DB API type interface for accessing the data from GAE.

Have fun.

Rdbhost.com

Google Appengine

Rdbhdb DB API module

Rdbhdb v 0.9

A new version of Rdbhdb is available from Pypi. Rdbhdb is the DB API 2.0 module for accessing Rdbhost databases remotely from Python applications.

Rdbhdb v 0.9 download page on PyPI

Rdbhdb v 0.9 download page on Rdbhost

New in this version:

• Uses gzip decompression, optionally, for data downloads from server.
• Supports the sending of binary data (as buffer datatype) in query parameters.
• .execute_deferred() method provided to execute queries in deferred mode. Deferred mode does not return results, but allows a more generous time limit for query completion. Useful for updates or inserts that might need extra time.
• .nextset() method implemented on cursors. Multiple queries can be aggregated into a single .execute() call, delimited by ';'. One result set is returned for each query, and .nextset() advances to the next result set in the returned list.
• .https attribute added to connection, to force use of SSL. Defaults to False, no SSL.

This version is the one currently in use on rdbhost.appspot.com, for its monitoring function.

Edited 28 Oct 09: Added rdbhost hosted download link.

Edited 9 Nov 09: Spelled PyPI properly

SSL Downtime

I was told yesterday by a site user that www.rdbhost.com's SSL certificate was expired.

For the moment, the secure site of www.rdbhost.com is down, and will be back up later today when I have time to get the certificate updated and installed.

David Keeney

As of 8PM on Tuesday, SSL is back up and working.

User side testing and Twill

History

RdbHost has two faces: one that the human user sees, and one that their software sees. The software side, the 'web service' has always had good testing. That it was designed to be computer parse-able without screen-scraping meant it was easy to write automated tests for it. There are in fact two sets of web service tests, one that uses urllib2 to query directly, and another that tests the DB API module Rdbhdb by using it to query the server.

Website Testing

Until recently, the human-readable (aka the web site) side of the business was not tested in any automated way. I would test features I thought might be affected by a change, manually, after an update. I am not sharing any details, but that strategy has proven to be deficient. Now we have a test suite for the user side as well.

Twill

The software used for the new tests is Twill. Twill can be run as its own scriptable shell, or included as a python module and the functions called from python. The Twill python module has an instantiated object syntax, where you create a browser object and call methods on it. Unfortunately, the browser object does not support the 'show' method. I ended up giving up on the explicit browser object, and just used the implicit browser. It works well, but doesn't satisfy my 'explicit code is more comfortable' preference.

Twill

Tests and Email

Many features of the website use email for various authentication functions. To reset your password, the site emails a password reset-link. To change your email address of record, you have the site email a change link to the new address. Creating a new account involves emailing the initial login password to the submitted address. How does one test such a system with Python and Twill?

My solution was to use one of the spam-dodging sites that allow one to receive email anonymously. Just email to 'anyname@spamspot.com' and spamspot.com will post the email content to a blog style web page, intermingled chronologically with other user's emails. The test code can request an email, for whatever functionality being tested, give the site a spamspot.com email address to mail to, wait a few seconds for the email to arrive and be posted, and then use Python and Twill to request the spamspot page and parse out the rdbhost email and its embedded password or links using 're' module. It is appropriate (if also a little paranoid) to change the password shortly thereafter to minimize tampering by other readers of spamspot.

There are many anonymous email receiver websites. Spamspot.com (mentioned above) and Makemeaking.com are more useful for our purposes than some, because they include the email inline to the page, without requiring another link-following step, and the prerequisite link parsing step.

Spamspot.com

Makemetheking.com

Edited 24 Oct 09: added reference to Makemetheking.com

Edited 26 Oct 09: minor grammatical corrections