Problems building a compatible Perl DBD::ODBC with unixODBC driver manager on 64 bit UNIX platforms

I maintian DBD::ODBC but I am wrestling with problems coping with 64 bit platforms. The problem is rather complex but I'm hoping that by describing it here someone will help come up with a logical answer to at least stop DBD::ODBC building in circumstances which are suspect.

Microsoft came up with ODBC long before there were 64 bit chipsets. The ODBC API talks about 32 bit values all over the place until recently when Microsoft changed their version of the ODBC API. Apparently the ODBC API was handed over to X/Open but it has not stopped Microsoft changing it. In the days of a 32 bit ODBC API a number of the arguments to ODBC APIs were defined as pointers to 32 bit values which when a particular ODBC API was called would write a 32 bit value back to that pointer address. Some of the APIs can return values like the number of rows or the maximum size of a blob and since 64 bit chipsets came along these values have increased in size such that a 32 value is not large enough to hold the value. The ammendments to the ODBC API by Microsoft define new types SQLLEN and SQLULEN which are 32 bit values on 32 bit operating systems and 64 bit values on 64 bit operating systems.

The first problem is that some ODBC driver writers took the original Microsoft API as gospel so when it said a 32 bit value it is a 32 bit value even though at the time 64 bit operating systems like tru64 existed (which had 64 bit longs in C) they kept to the API and made those values (and pointers to those values) 32 bit. This led to problems on 64 bit operating systems since you could download one driver which took the ODBC spec literally and another ODBC driver which decided the ODBC spec didn't mean that and some arguments to ODBC APIs would be 64 bit (or worse, pointers to 64 bit values). As a result some ODBC driver writers produce 2 versions of their driver for 64 bit operating systems 1) with 32 bit values and 2) with 64 bit values. That was the situation until Microsoft produced 64 bit operating systems and then they changed the ODBC API to introduce two new typedefs of SQLLEN and SQLULEN which are 32 bits on 32 bit operating systems and 64 bits on 64 bit operating systems even though in Windows a C long is still 32 bits. We'll skip this issue for now, it is just background and not the real issue I have a problem with.

The ODBC driver manager most used on UNIX operating systems is unixODBC and as an OS project is designed to be built from source via GNU libtool and configure. Since Microsoft introduced the new types SQLLEN and SQLULEN for certain arguments to certain ODBC APIs unixODBC followed. The problem is that whether you build unixODBC on a 32 bit operating system or a 64 bit operating system the C header files installed are exactly the same but SQLLEN/SQLULEN are different sizes depending on whether the C compiler knows whether the macro SIZEOF_LONG is set or not. When the macro SIZEOF_LONG is set to 8 SQLLEN and SQLULEN are set to a 64 bit value and when SIZEOF_LONG = 4 or it is not defined, SQLLEN/SQLULEN are defined as 32 vbit values. This works fine for unixODBC as the configure script used when building unixODBC sets SIZEOF_LONG depending on the platform but when you build another program (like DBD::ODBC) which depends on the unixODBC header files it is crucial the same compiler macos are defined or unixODBC could be built with SQLLEN = 8 bytes and DBD::ODBC could be built with 32 bit SQLLEN and corruption occurs whenever the ODBC driver attempts to write a 64 bit SQLLEN to an address provided by DBD::ODBC which only had space for a 32 bit value.

To sort this problem out unixODBC added the odbc_config program which when run as odbc_config --cflags it outputs the C compiler directives which were used to build unixODBC. As a result, any program building against the unixODBC header files can call odbc_config --cflags and get the compiler directives used to build unixODBC and thus SIZEOF_LONG is set and the program (DBD::ODBC in this case) has SQLLEN/SQLULEN set to the same size as unixODBC was built with. For some time DBD::ODBC has looked for odbc_config on its path and adds the --cflags output to its compiler line. However, unless someone builds a recentish unixODBC from source it is unlikely they have the odbc_config program and even worse, debian/Ubuntu packages (even if they are recent distributions of unixODBC) don't install odbc_config so DBD::ODBC has no way of knowing what compiler directives to add to the compile line. This means that if DBD::ODBC is built on a 64 bit platform with a 64 bit Perl if it does not find odbc_config, SIZEOF_LONG is not defined and the unixODBC header files default to SQLLEN/SQLULEN being 32 bit values but unixODBC was probably built with SQLLEN/SQLULEN as 64 bit values. Left unchecked this results in corruption and leads to all sorts of unpredictable problems.

For some time I tried to persuade the debian packagers to update unixODBC and include odbc_config but I never got responses to emails direct to them. Recently, when I had yet another issue with this I tried #debian channel on irc and am making some progress. The question remains as to what the most sensible action I should take when building DBD::ODBC on a platform where Perls Config says ptrsize is 8. Since ptrsize is 8 I can reasonably assume this is a 64 bit operating system but if I don't find unixODBC's odbc_config I cannot know how unixODBC was built (and few distributions install odbc_config). It is possible that even on a 64 bit operating system SIZEOF_LONG=4 (the person building unixODBC might have set this) but most likely SIZEOF_LONG = 8. I need to know for certain or a DBD::ODBC build will produce a binary which is incompatible with unixODBC and the ODBC driver. If I don't find odbc_config on a system where ptrsize=8 should I abort or let this go and leave it up to the installer to work out what is wrong when things go wrong or do I abort and say it is likely DBD::ODBC is going to be built in a way inconsistent with the unixODBC driver manager and ODBC driver.

As an aside, newer Unixes seem to put 32 bit shared objects and 64 bit shared objects in different directories but what about C header files - are there systems with a /usr/include32 and /usr/include64? If not then C header files which rely on C macros set at compile time are doomed to failure unless they are standard macros defined by the compiler which knows whether it is compiling for 32 bits or 64 bits - perhaps this is the fundamental failing in unixODBC's header files.

Update1

If you download unixODBC 2.3.1-pre or later the header files now default to a 64 bit SQLLEN/SQLULEN on 64 bit machines and 32 bit on 32 bit machines.

Update2

This issue has been reported on Debian bug tracking system at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=635668

Comments

Retracted debian issue

It turns out the details I've been sent from systems supposedly based on Debian build are out of date and Debian have already resolved this issue some time ago. This is no longer a debian issue and I've retracted the bug I started. The unixODBC project addressed this issue some time ago creating a file on compilation with the correct settings.