Articles tagged recipe

Cancelling PostgreSQL statements from Python

Posted by Daniele Varrazzo on July 20, 2014
Tagged as recipe

Cancelling a long running query from Python is not something that happens automatically: the libpq doesn't react to Python signals so the only way to stop a query is to run a pg_cancel_backend from another process. Killing the Python process won't cancel the query: it will run until completion and then rolled back. This makes working wth long-running query from the Python interpreter somewhat frustrating.

Using psycopg in green mode moves the waiting from the libpq C code to Python: this gives Python some chance of interaction: it is possible for instance to catch a ctrl-c and send a cancel request:

from select import select
from psycopg2.extensions import POLL_OK, POLL_READ, POLL_WRITE

def wait_select_inter(conn):
    while 1:
        try:
            state = conn.poll()
            if state == POLL_OK:
                break
            elif state == POLL_READ:
                select([conn.fileno()], [], [])
            elif state == POLL_WRITE:
                select([], [conn.fileno()], [])
            else:
                raise conn.OperationalError(
                    "bad state from poll: %s" % state)
        except KeyboardInterrupt:
            conn.cancel()
            # the loop will be broken by a server error
            continue

psycopg2.extensions.set_wait_callback(wait_select_inter)

An interactive session would look like:

>>> cnn = psycopg2.connect('')
>>> cur = cnn.cursor()
>>> cur.execute("select pg_sleep(10)")
^C
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
QueryCanceledError: canceling statement due to user request

The connection is now in error state, but a cnn.rollback() would make it working again.

PostgreSQL notifications with Psycopg2 and Eventlet

Posted by Daniele Varrazzo on December 1, 2010
Tagged as async, eventlet, notify, recipe

PostgreSQL supports asynchronous notifications, a simple messaging system allowing clients to be notified about events occurred in the database. Notifications can be sent by any session using a "NOTIFY channel" command and will be received by any session which has subscribed with a LISTEN channel to receive updates. The system has been greatly improved in PostgreSQL 9.0 with the addition of a message payload, making the feature even more useful. Previously a typical use case would have been to notify interested sessions that a certain table was updated: now it is possible to signal for instance which record was changed. You can put the NOTIFY command in a database trigger for automatic notifications on insert or update... the possibilities are actually quite interesting.

Read more...

Links about building Psycopg on Mac OS X

Posted by Daniele Varrazzo on November 11, 2010
Tagged as os-x, recipe

Looks like building Psycopg on OS X is tricky: the code needs no tweak, but linking against the right library seems problematic.

Read more...

Passing connections to functions using a decorator

Posted by Daniele Varrazzo on October 22, 2010
Tagged as recipe

In many script I write, there are functions requiring database operations. Every time I need them, I try to write such code in functions like:

@with_connection
def do_some_job(cnn, arg1, arg2=None):
    cur = cnn.cursor()
    cur.execute(SQL, (arg1, arg2)) # or something else

do_some_job(42, arg2='hi')

Read more...