Psycopg 2.3.0 released

Posted by Federico Di Gregorio on December 1, 2010
Tagged as news, release

Hi *,

lately my involvment on psycopg has been quite small (work and other projects are taking their toll) but thanks to Daniele (with some help from Jan) here is a new psycopg2 release. The number of changes is big (an excerpt from the NEWS file is at the end, as always) so we decided to bump up the version to 2.3.0 and to test, and test... and test. We tested this new release on Linux and Windows, with Python 2.4, 2.5, 2.6, 2.7 and on PostgreSQL from 7.4 to 9.0.

As always, signed download from:

Have fun, federico

And here are the NEWS:

psycopg 2.3 aims to expose some new features introduced in PostgreSQL 9.0.

Main new features

  • dict to hstore adapter and hstore to dict typecaster, using both 9.0 and pre-9.0 syntax.
  • Two-phase commit protocol support as per DBAPI specification.
  • Support for payload in notifications received from the backed.
  • namedtuple-returning cursor.
  • Query execution cancel.

Other features and changes

  • Dropped support for protocol 2: Psycopg 2.3 can only connect to PostgreSQL servers with version at least 7.4.
  • Don't issue a query at every connection to detect the client encoding and to set the datestyle to ISO if it is already compatible with what expected.
  • mogrify() now supports unicode queries.
  • Subclasses of a type that can be adapted are adapted as the superclass.
  • errorcodes knows a couple of new codes introduced in PostgreSQL 9.0.
  • Dropped deprecated Psycopg "own quoting".
  • Never issue a ROLLBACK on close/GC. This behaviour was introduced as a bug in release 2.2, but trying to send a command while being destroyed has been considered not safe.

Bug fixes

  • Fixed use of PQfreemem instead of free in binary typecaster.
  • Fixed access to freed memory in conn_get_isolation_level().
  • Fixed crash during Decimal adaptation with a few 2.5.x Python versions (ticket #7).
  • Fixed notices order (ticket #9).

Comments (15 posted).

Comment #1
Shawn Koppenhoefer
Dec. 2, 2010, 12:07 p.m.

This is great news indeed! Out of the box, it doesn't work for me,.. but I'm hopeful.

/shawn (@drkdev on twitter)

p.s. I have a fresh virgin Enthought installation. From the downloaded src of Psycopg 2.3.0 I DO get successful installation via 'python setup.py install' command. BUT... Once inside iPython,.. I get this:

>  import psycopg2
ImportError
/Library/Frameworks/Python.framework/Versions/6.3/lib/python2.6/site-packages/psycopg2/__init__.py in <module>()
68
---> 69 from _psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/6.3/lib/python2.6/site-packages/psycopg2/_psycopg.so, 2): Symbol not found: _PQbackendPID
  Referenced from: /Library/Frameworks/Python.framework/Versions/6.3/lib/python2.6/site-packages/psycopg2/_psycopg.so
  Expected in: flat namespace
 in /Library/Frameworks/Python.framework/Versions/6.3/lib/python2.6/site-packages/psycopg2/_psycopg.so

so no joy :-(

I am on a MacOS X 10.6.5 btw.

Comment #2
Daniele Varrazzo
Dec. 2, 2010, 1:24 p.m.

Never a joy for OS X users :D

Check one of these articles: there should be a solution for your problem.

Comment #3
Adrian Klaver
Dec. 2, 2010, 3:59 p.m.

Would it possible at some time to revive the mailing list?

Comment #4
Charlie Clark
Dec. 2, 2010, 10:38 p.m.

@Shawn - what an earth is Python 6.3 doing in there? That looks seriously borked.

A possible solution would be to install the MacPorts version, already updated to 2.3, and copy the files over to your Python.

Comment #5
takoma
Dec. 3, 2010, 10:47 a.m.

I can not install psycopg2-2.3.0 (although I could install psycopg2-2.3.0-beta1 on same environment).

Install error log

$(env0)[xx@yy env0]$ pip -E . install psycopg2
Running setup.py egg_info for package psycopg2
.........(ommit)

/usr/bin/ld: build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o: relocation R_X86_64_PC32 against `psyco_typecast_datetime_init' can not be used when making a shared object; recompile with -fPIC

/usr/bin/ld: final link failed: Bad value

collect2: ld returned 1 exit status

error: command 'gcc' failed with exit status 1

----------------------------------------
Command /opt/env0/bin/python -c "import setuptools;__file__='/opt/env0/build/psycopg2/setup.py';execfile(__file__)" install --single-version-externally-managed --record /tmp/pip-bxDXoC-record/install-record.txt --install-headers /opt/env0/include/site/python2.7 failed with error code 1
My environment
CentOS 5.5 64bit, Python 2.7.1, Virtualenv, PostgresSQL 9.0.1(installed by PostgreSQL Yum repository)

pg_config output

$ pg_config
BINDIR = /usr/pgsql-9.0/bin
DOCDIR = /usr/share/doc/pgsql
HTMLDIR = /usr/share/doc/pgsql
INCLUDEDIR = /usr/pgsql-9.0/include
PKGINCLUDEDIR = /usr/pgsql-9.0/include
INCLUDEDIR-SERVER = /usr/pgsql-9.0/include/server
LIBDIR = /usr/pgsql-9.0/lib
PKGLIBDIR = /usr/pgsql-9.0/lib
LOCALEDIR = /usr/pgsql-9.0/share/locale
MANDIR = /usr/pgsql-9.0/share/man
SHAREDIR = /usr/pgsql-9.0/share
SYSCONFDIR = /etc/sysconfig/pgsql
PGXS = /usr/pgsql-9.0/lib/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--disable-rpath' '--prefix=/usr/pgsql-9.0' '--includedir=/usr/pgsql-9.0/include' '--mandir=/usr/pgsql-9.0/share/man' '--datadir=/usr/pgsql-9.0/share' '--with-perl' '--with-python' '--with-tcl' '--with-tclconfig=/usr/lib64' '--with-openssl' '--with-pam' '--with-krb5' '--with-gssapi' '--with-includes=/usr/include' '--with-libraries=/usr/lib64' '--enable-nls' '--with-ossp-uuid' '--with-libxml' '--with-libxslt' '--with-ldap' '--with-system-tzdata=/usr/share/zoneinfo' '--sysconfdir=/etc/sysconfig/pgsql' '--docdir=/usr/share/doc' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -I/usr/include/et' 'CPPFLAGS= -I/usr/include/et'
CC = gcc
CPPFLAGS = -I/usr/include/et -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include
CFLAGS = -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -I/usr/include/et -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv
CFLAGS_SL = -fpic
LDFLAGS = -L/usr/lib64
LDFLAGS_EX =
LDFLAGS_SL =
LIBS = -lpgport -lxslt -lxml2 -lpam -lssl -lcrypto -lgssapi_krb5 -lz -lreadline -ltermcap -lcrypt -ldl -lm
VERSION = PostgreSQL 9.0.1

how to solve it?

Thanks.

Comment #6
Daniele Varrazzo
Dec. 3, 2010, 11:07 a.m.

The build issue on CentOS 5.5 x86_64 has been reported yesterday: see ticket #23.

In the ticket there is a temporary solution: we will release a proper fix soon.

Comment #7
takoma
Dec. 3, 2010, 11:16 a.m.

Thanks. I use 2.3.0-beta1 until the fix will be released.

Comment #8
Daniele Varrazzo
Dec. 4, 2010, 9:57 p.m.

The build problem on CentOS has been fixed in release 2.3.1.

Comment #9
Kurt Schwehr
Dec. 5, 2010, 6:15 p.m.

I've just updated fink to have psycopg2 2.3.1 for Mac OSX users. This is one of the packages in fink that has always worked right out of the box for me.

Comment #10
Divius
Dec. 16, 2010, 5:15 p.m.

Hi!

Psycopg2 is in fact incompatible with PostgreSQL 7.4 due to usage of new lo_create instead of old lo_creat.

See http://www.postgresql.org/docs/7.4/static/lo-interfaces.html

Regards, Divius.

Comment #11
Daniele Varrazzo
Dec. 16, 2010, 6:10 p.m.

This needs some clarification.

You can talk with a PG server 7.4, but psycopg requires a client libpq version newer than that.

I actually don't know what is the minimum version required: will check that and document. For other functions, the library degrades gracefully, e.g. with libpq < 8.3 there is no lo_truncate.

Having an up-to-date libpq on the client is advisable, e.g. binary data in PG 9 is sent in a format that only libpq 9 understand (unless you disable it in the server). There is no problem instead in talking with old server versions, up to 7.4, using newer libpq.

The interactions among server/lib versions are actually messy: will write some article/page/docs about that.

Comment #12
Divius
Dec. 16, 2010, 6:27 p.m.

BTW I succeeded in building psycopg2 for libpq 7.4 with a few source code changes! At least most common things work.

Comment #13
Daniele Varrazzo
Dec. 16, 2010, 6:47 p.m.

It's not exactly a priority, but if you send the patch I can integrate it. Be sure the test suite passes entirely though (make check).

You may open an issue in the tracker to have it integrated, thanks!

Comment #14
Daniele Varrazzo
Dec. 16, 2010, 6:57 p.m.

OTOH 7.4 is not supported anymore, so psycopg wouldn't benefit from security bugfixes... I really think using a libpq library less than 8.1 (soon to be phased out as well anyway) would be a very bad idea.

http://wiki.postgresql.org/wiki/PostgreSQL_Release_Support_Policy

Why do you still have 7.4 around on a client?

Comment #15
Divius
Dec. 16, 2010, 7:23 p.m.

  1. My patches seem to cause regressions, I don't think they should be integrated into the main tree.
  2. I have VERY old certified OS (seems to be based on RedHat 5 or older). I cannot upgrade server version. Maybe client upgrade is possible... I'm just not sure it's possible to compile new client on such OS.

BTW, is it possible to get sources and compile libpq without server?

Feel free to add a new comment. You can use reST syntax.