Compiling Gearman (or anything) for Zend Server CE on Snow Leopard

May 12th, 2010 § 7 comments § permalink

The first thing you need to know about Mac OS.X Snow Leopard all Mac’s and Macbook Pro’s is that this hardware is 64 bit capable. This may not mean you are running a 64 bit kernel, it simply means that the operating system is capable of executing x86 64bit executables. We won’t go into the details of kernel architecture, you can read more about that here.

What is important though is that both x86_64 and i386 based executables can run on snow leopard. What is not uncommon on OS.X is to have executables (and libraries) that have multiple architectures compiled in. To see what architectures are inside a particular file, run something like this:

[sourcecode language="text" highlight="1,7"]
/usr/local# file /usr/bin/php
/usr/bin/php: Mach-O universal binary with 3 architectures
/usr/bin/php (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/bin/php (for architecture i386): Mach-O executable i386
/usr/bin/php (for architecture ppc7400): Mach-O executable ppc

/usr/local# file /usr/local/zend/apache2/bin/httpd
/usr/local/zend/apache2/bin/httpd: Mach-O executable i386
[/sourcecode]

This means that PHP (supplied by apple), has been compiled with 3 architectures inside. What does that mean? It means there is basically 3 versions on PHP compiled into a single binary, and that when it is loaded into memory, only one particular version will be used at a time. To demonstrate, lets take a pretty common difference between 32bit and 64bit architectures: integer size. We know that 64 bit integer space is larger than that of the 32bit space. The following demo will show running different architectures from the same binary:

[sourcecode language="text" highlight="1,4"]
/usr/local# arch -arch x86_64 /usr/bin/php -nr ‘echo PHP_INT_MAX;’
9223372036854775807

/usr/local# arch -arch i386 /usr/bin/php -nr ‘echo PHP_INT_MAX;’
2147483647
[/sourcecode]

We know we are running same command though different architectures since we know PHP has different max integer sizes.

The next important thing to understand is the nature of the PHP stack. PHP is generally regarded as a glue language. That might mean several things to different people, but we will be looking strictly at this statement in the purest technical sense. PHP is made of the core language and features, but also a rich set of extensions. These extensions are typically written in C, and have interfaced with the C layer PHPAPI. Most of the really useful extensions are linked against libraries on your system, for example the openssl set of functions are not actually implemented in PHP’s source code, the openssl extension is simple a wrapper that calls out to libssl.so (or .dylib on mac, .dll on windows). This is what is meant by PHP being a glue language/platform.

Since PHP relies on existing compiled libraries, you further have to understand how things are linked and compiled. There are generally two options here: linking dynamically, or statically compiling. Either way, one thing remains true: you cannot mix architectures. This means that if your apache/mod_php and/or php binary are only i386, then all of the libraries on your system that will be used must contain the i386 architecture. Likewise, apache/mod_php and/or php binary are only x86_64, then all of your libraries must contain the x86_64 architecture. Failing to have this, you will get a message like this for example:

[sourcecode language="text"]
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/local/zend/lib/php_extensions/gearman.so’ – dlopen(/usr/local/zend/lib/php_extensions/gearman.so, 9): no suitable image found. Did find:
/usr/local/zend/lib/php_extensions/gearman.so: mach-o, but wrong architecture in Unknown on line 0
[/sourcecode]

Now that we understand that executables and libraries can have multiple architectures, let’s get to the task at hand: making sure new extensions can run with Zend Server CE.

Zend Server CE for Mac (as of this writing), comes compiled as an i386 executable only. This includes the PHP binary, php library, and apache binaries that come shipped with ZSCE. While ZSCE works great out the box with all the provided extensions, you might find that you want some additional 3rd party PHP extensions compiled/linked into this stack. That’s where things get a little confusing, and in this post, we’ll look at how to install the gearman extension.

PHP Extensions are basically wrappers around existing libraries, so generally, these extensions require the base library to already be on the system. In our case, we need “libgearman” compiled and on our system for us to be able to compile and use the PHP Gearman Extension.

At this point, I would generally instruct you to compile Gearman with multiple architectures and install (–prefix=/usr/local). (Note: to compile for multiple architectures, simply do the following):

[sourcecode language="text" highlight="1"]
export CFLAGS=’-arch i386 -arch x86_64′
[/sourcecode]

In the particular case of Gearman, this will not work as the Gearman makefile utilizes flags that are not compatible with multiple architecture targets. As such, we go to plan B.

Plan B is something I generally do to keep my system clean: statically building libraries. I have a personal rule of not keeping i386 only libraries installed in common places like /usr/lib or /usr/local/lib, in this case /usr/local/lib/libgearman.dylib. Since this is the case, I’ll build Gearman statically, compile it into the PHP Gearman Extension, and this will allow me to remove the temporary Gearman installation which will have to be i386 only.

[sourcecode language="text" highlight="4,11,12,13,17,21,23,26,27,28,34,35,36,37,42,43,44,50,56"]
# check to ensure we have a multi-arch libevent (if not go create it as
# normal with CFLAGS="-arch i386 -arch x86_64" and install to /usr/local)

/usr/local/src/gearmand-0.13# file /usr/local/lib/libevent.dylib
/usr/local/lib/libevent.dylib: Mach-O universal binary with 2 architectures
/usr/local/lib/libevent.dylib (for architecture i386): Mach-O dynamically linked shared library i386
/usr/local/lib/libevent.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64

# next compile gearman to a temp location

/usr/local/src/gearmand-0.13# export "CFLAGS=-arch i386"
/usr/local/src/gearmand-0.13# ./configure –disable-shared –prefix=/usr/local/gearman-tmp
/usr/local/src/gearmand-0.13# make && make install
[gearman installed now, this should only have static files]

# ensure we only have a .a library file for gearman
/usr/local/src/gearmand-0.13# ls /usr/local/gearman-tmp/lib/
libgearman.a libgearman.la pkgconfig

# make sure zend/bin is first on your PATH
/usr/local/zend/tmp# echo $PATH
/usr/local/zend/bin:/var/root/.bin:/usr/local/git/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin
/usr/local/zend/tmp# which phpize
/usr/local/zend/bin/phpize

# next, go to our zend server location, and pull down gearman extension
/usr/local/src/gearmand-0.13# cd /usr/local/zend/tmp/
/usr/local/zend/tmp# pecl download gearman-beta
downloading gearman-0.7.0.tgz …
Starting to download gearman-0.7.0.tgz (29,258 bytes)
………done: 29,258 bytes
File /usr/local/zend/tmp/gearman-0.7.0.tgz downloaded

# next, unpack, phpize, and statically compile
/usr/local/zend/tmp# tar zxf gearman-0.7.0.tgz
/usr/local/zend/tmp# cd gearman-0.7.0
/usr/local/zend/tmp/gearman-0.7.0# phpize
Configuring for:
PHP Api Version: 20090626
Zend Module Api No: 20090626
Zend Extension Api No: 220090626
/usr/local/zend/tmp/gearman-0.7.0# ./configure –with-gearman=/usr/local/gearman-tmp/ –disable-shared
/usr/local/zend/tmp/gearman-0.7.0# make
/usr/local/zend/tmp/gearman-0.7.0# make install
Installing shared extensions: /usr/local/zend/lib/php_extensions/

# Now go add extension=gearman.so to your php.ini file inside /usr/local/zend/etc/php.ini

# Now go check that php will have gearman support
/usr/local/zend# php -i | grep gearman
gearman
gearman support => enabled
libgearman version => 0.13

# Since we statically compiled it, we can remove our temp install of gearman
/usr/local/zend# rm -Rf /usr/local/gearman-tmp/
[/sourcecode]

At this point, you now have a 3rd party PECL extension that is compiled and working with ZSCE on Mac OS.X.

Where am I?

You are currently viewing the archives for May, 2010 at Ralph Schindler.