Articles from Daniele Varrazzo

Psycopg 2.4.5 released

Many thanks to everybody that contributed with bug reports and comments to this release!

What's new in psycopg 2.4.5

The close() methods on connections and cursors don't raise exceptions if called on already closed objects.
Fixed fetchmany() with no argument in cursor subclasses (ticket #84).
Use lo_creat() instead of lo_create() when possible for better interaction with pgpool-II (ticket #88).
Error and its subclasses are picklable, useful for multiprocessing interaction (ticket #90).
Better efficiency and formatting of timezone offset objects thanks to Menno Smits (tickets #94, #95).
Fixed rownumber during iteration on cursor subclasses. Regression introduced in 2.4.4 (ticket #100).
Added support for inet arrays.
Fixed commit() concurrency problem (ticket #103).
Codebase cleaned up using the GCC Python plugin's static analysis tool, which has revealed several unchecked return values, possible NULL dereferences, reference counting problems. Many thanks to David Malcolm for the useful tool and the assistance provided using it.

After a short discussion on this list we decided to change the definitions of isolation levels to make sure old code using numeric constants (both psycopg1 and psycopg2) continue to works. Other small fixes are included in the release: see below for details.

What's new in psycopg 2.4.4

register_composite() also works with the types implicitly defined after a table row, not only with the ones created by CREATE TYPE.
Values for the isolation level symbolic constants restored to what they were before release 2.4.2 to avoid breaking apps using the values instead of the constants.
Named DictCursor/RealDictCursor honour itersize (ticket #80).
Fixed rollback on error on Zope (ticket #73).
Raise DatabaseError instead of Error with empty libpq errors, consistently with other disconnection-related errors: regression introduced in release 2.4.1 (ticket #82).

Psycopg 2.4.3 released

Mostly a bugfix release, with as usual a couple of small feature added:

Here is what's new in this release:

connect() supports all the keyword arguments supported by the database
Added new_array_type() function for easy creation of array typecasters.
Added support for arrays of hstores and composite types (ticket #66).
Fixed segfault in case of transaction started with connection lost (and possibly other events).
Fixed adaptation of Decimal type in sub-interpreters, such as in certain mod_wsgi configurations (ticket #52).
Rollback connections in transaction or in error before putting them back into a pool. Also discard broken connections (ticket #62).
Lazy import of the slow uuid module, thanks to Marko Kreen.
Fixed NamedTupleCursor.executemany() (ticket #65).
Fixed --static-libpq setup option (ticket #64).
Fixed interaction between RealDictCursor and named cursors (ticket #67).
Dropped limit on the columns length in COPY operations (ticket #68).
Fixed reference leak with arguments referenced more than once in queries (ticket #81).
Fixed typecasting of arrays containing consecutive backslashes.
errorcodes map updated to PostgreSQL 9.1.

Psycopg 2.4.2 released

Psycopg 2.4.2 has been released: it brings a few small but interesting new features, and a lot of bug fixes.

Transaction control has been overhauled: a new connection method set_session() allows setting all the session properties affecting the transactions behaviour: the isolation level but it can also be used to have auto-commit, read-only, and deferrable transactions. You can use for example:

conn.set_session('read committed')
conn.set_session(readonly=True, autocommit=True)
conn.set_session('serializable',
                 readonly=True, deferrable=True)

There is also a welcome improvement for all the users thinking that

conn.set_isolation_level(
    psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)

was an excessively verbose syntax: a new read/write property allows to set

conn.autocommit = True

a much handier syntax for an often used connection property.

The improvements to the transactions control are not only at interface level: Psycopg doesn't require any more setup queries when connecting to a database. A sequence of statements:

import psycopg2
conn = psycopg2.connect('')
curs = conn.cursor()
curs.execute('SELECT 1')
curs.execute('SELECT 2')
conn.commit()

in older versions of the library would have resulted in the following commands sent to the backend:

SHOW default_transaction_isolation
SET DATESTYLE TO 'ISO'
SHOW client_encoding
BEGIN; SET TRANSACTION ISOLATION LEVEL READ COMMITTED
SELECT 1
SELECT 2
COMMIT

In Psycopg 2.4.2 the only commands sent are instead the essential:

BEGIN
SELECT 1
SELECT 2
COMMIT

with the BEGIN/COMMIT obviously omitted in autocommit mode (the datestyle and encoding statements were already dropped in 2.3).

The new release also brings a lot of bug fixes, so we encourage updating soon, particularly if you use Psycopg in multithread programs:

Fixed bug with multithread code potentially causing loss of sync with the server communication or lock of the client (ticket #55).
Don't fail import if mx.DateTime module can't be found, even if its support was built (ticket #53): a fix for the "ghost mx.DateTime" issue reported in virtualenv.
Fixed escape for negative numbers prefixed by minus operator (ticket #57).
Fixed GC race condition during copy in multithread programs, potentially resulting in refcount errors. Fixed by Dave Malcolm (ticket #58, Red Hat Bug 711095).
Trying to execute concurrent operations on the same connection through concurrent green thread results in an error instead of a deadlock.
Fixed detection of pg_config on Windows. Report and fix, plus some long needed setup.py cleanup by Steve Lacy: thanks!

Building Psycopg on Windows using MinGW

My goal was to install Psycopg on Windows using MinGW and the PostgreSQL binary package.

I have used the MinGW GCC binaries packaged by Giovanni Bajo. The package takes care of a lot of details, for instance registering MinGW as default compiler for Python, plus some magic I don't even want to know, and makes the entire process simple enough.

The first step is to ensure that setup.py can find pg_config. There is a bug causing it not to be found if it is in the path: it will be fixed in Psycopg 2.4.2. On a few older versions you will have to specify the pg_config path in the setup.cfg or by the --pg-config option, e.g.

python setup.py build_ext --pg-config=C:\path\to\pg_config.exe build

The library built depends on libpq.dll, so at runtime this library should be available, e.g. on the path or it may be copied in the psycopg2 directory. The libpq in turn depends on a few other dlls, all found in the PostgreSQL bin directory: libeay32.dll, ssleay32.dll and libintl-8.dll: they should be made available to the client as well. Unfortunately if any of these libraries is missing you will only get an "ImportError: dll load failed". The problem is very easy to debug using a dependency walker.

Another problem you may find is building for Python 2.6 and newer: some MinGW versions ship with a broken msvcr90 lib, and the result is again a rather unhelpful ImportError. the dependency walker is useful in this case too, showing the missing localtime function in the library. The bug was reported in the issue 3308: my solution was to download a newer MinGW and use the libmsvcr90.a from there.

So, overall, it can be done, but the result still depends on many dlls. I'm torn if the solution could be to have all the dependencies copied in the package directory. Of course you can still use Jason's binary package: he builds both the libpq and the openssl as static libs and creates a self-contained psycopg, which is probably the handiest result, but can't be obtained only using the PostgreSQL binaries.

psycopg2 porting to Python 3: a report

I've mostly finished the porting of psycopg2 to Python 3. Here is a report of what done and what can be improved.

New Psycopg Mailing List Online!

After a long while Psycopg has a Mailing List again!

You are welcome to subscribe, either if you are an user dealing with the first stumbling blocks (albeit a look at the documentation or the FAQ wouldn't hurt!) or if you want to contribute to the psycopg2 development, about which there are several upcoming news.

To post, send mail to <psycopg@postgresql.org>. Yes, we are proud to be hosted on the mighty shoulders of the PostgreSQL Infrastructure team. I want to thank them and the PostgreSQL Global Development Group wholeheartedly for making this happen.

See you there!

Psycopg 2.3.2 released

Hello,

just released Psycopg 2.3.2. The release fixes a bug reported in 2.3.0 and 2.3.1 preventing Psycopg to connect to pgBouncer. Thanks to Marti Raudsepp for the bug report and the patch.

Psycopg 2.3.1 released

Hello,

just released Psycopg 2.3.1. No new feature since release 2.3.0: the only fix is for a build issue on CentOS 5.5 x86_64 (ticket #23).

PostgreSQL notifications with Psycopg2 and Eventlet

PostgreSQL supports asynchronous notifications, a simple messaging system allowing clients to be notified about events occurred in the database. Notifications can be sent by any session using a "NOTIFY channel" command and will be received by any session which has subscribed with a LISTEN channel to receive updates. The system has been greatly improved in PostgreSQL 9.0 with the addition of a message payload, making the feature even more useful. Previously a typical use case would have been to notify interested sessions that a certain table was updated: now it is possible to signal for instance which record was changed. You can put the NOTIFY command in a database trigger for automatic notifications on insert or update... the possibilities are actually quite interesting.

psycopg

Articles from Daniele Varrazzo

Psycopg 2.4.5 released

What's new in psycopg 2.4.5

Psycopg 2.4.4 released

What's new in psycopg 2.4.4

Psycopg 2.4.3 released

Psycopg 2.4.2 released

Building Psycopg on Windows using MinGW

psycopg2 porting to Python 3: a report

New Psycopg Mailing List Online!

Psycopg 2.3.2 released

Psycopg 2.3.1 released

PostgreSQL notifications with Psycopg2 and Eventlet

Articles archive