One of the hardest problems to solve when developing an ORM of any complexity is in deciding how to handle the retrieval of rows that satisfy a many-to-many relationship, also known as a M:N relationship. From the perspective of an object, there is no such thing as a many to many relationship. There are only two relationships an object understands. The first is the relationship of itself to another object, which is a one to one (1:1) relationship. The second is the relationship of itself to a group of other objects, or a one-to-many (1:N) relationship. It’s not until you look at the relationship of all objects in a system that the many-to-many relationship pattern emerges.

In RDBM systems, rows and their relationships are modeled through the use of foreign keys and foreign key constraints between a left table and a right table. Foreign key constraints, by themselves, can only model 1:1 and 1:N relationship of rows. To model M:N relationships, database developers must get creative. By employing the use of a “3rd party”, and by utilizing foreign keys that model a 1:N relationship, database developers can model a M:N relationship. This 3rd party comes in the form of another table that may or may not have any data model specific information attached to it. This table is generally known as a junction table, but has also been known as a cross-reference table, bridge table, join table, map table, intersection table, linking table, many-to-many resolver, link table, or association table.

Zend_Db_Table_Row And Junction Tables

Zend_Db_Table is a component in Zend Framework that implements the Table Data and Row Data Gateway patterns. In short, a row object attempts to create a single PHP object per actual row in the database table. Furthermore, Zend_Db_Table_Row objects can go as far as to describe, understand, and interrogate these various 1:1, 1:N and M:N relationships. This allows row objects to be able to find and return related row objects in the form of a rowset.

One of the primary tenets of Zend_Db_Table and Zend_Db_Table_Row is to be able to produce consistent row objects. This means that the properties of these row objects should be a complete and logical representation of how the row might look inside the table of the RDBMS.

Some time ago an issue (ZF-6232) was filed against Zend_Db_Table to report that columns from the junction table were being included in the resulting rowset’s row objects. This was causing issues for people who then attempted to save() the row object back to the database. If a developer mistakenly altered one of the junction table values that was accidentally included in the row, Zend_Db_Table_Row would throw an exception since the row object had more columns than the actual row in the database. Given that we want to create consistent, complete and logical row objects, a solution was devised to ensure that the junction table’s row information was not included in the resulting rowset’s rows. Consequently, this meant that anyone relying on this undocumented behavior would no longer be able to get data stored inside the junction table as part of the result set’s row object. This fix was incorporated into the 1.10.2 release.

Over the past several years of working on Zend Framework, I’ve noticed the developer population at large is really good at finding undocumented and previously unthought-of use-cases of Zend Framework components. These use-cases, while sometimes “inventive” to say the least- are also sometimes blatant misuses of a component. It suffices to say that these use-cases are not captured in a unit test and consequently are not protected by backwards compatibility.

Relying on Zend_Db_Table_Row to include junction data is not only an unintended use case but also a misuse of the findManyToManyRowset() functionality provided by Zend_Db_Table_Row. That said, I do want to provide a solution for developers that relied on this behavior of Zend_Db_Table_Row in Zend Framework previous to 1.10.2.

A Solution

While the motivation for creating this class is based on providing a solution to developers who relied on utilizing junction table data in Zend_Db_Table_Row’s many-to-many rowsets, this same technique can be utilized with any ORM or database abstraction layer that handles many-to-many result sets.

Basically, I’ve created a single class that effectively take the place of Zend_Db_Table_Row::findManyToManyRowset() for the purposes of creating an iterable rowset that allows access to both the target many-to-many rowset as well as the junction rowset. This solution is called a Composite Rowset. In this solution, both rowsets (iterators) are kept in sync with one another. This proves to be an ideal solution in a couple of ways. First, it will produce consistent row objects that are explicitly tied to a row in a database. Second, the cost of creating this composite rowset is at the expense of 2 queries: the original many-to-many query and a similar query to retrieve the junction rowset. This is ideal since previously, to get the junction data, findDependentRowset() would have had to been called on each row within the rowset produced by the Zend_Db_Table_Row::findManyToManyRowset().

The API for this Composite Rowset looks like this:

/**
 * @link https://github.com/gooeylabs/Gooey-PHP-5.2-Components/blob/master/library/Gooey/Db/Table/ManyToManyCompositeRowset.php
 */
class Gooey_Db_Table_ManyToManyCompositeRowset implements SeekableIterator, ArrayAccess, Countable
{

    public function __construct(Zend_Db_Table_Row_Abstract $row, $matchTableName, $junctionTableName, $matchRefRule = null);
    public function seek($position);
    public function current();
    public function currentJunction();
    public function next();
    public function rewind();
    public function key();
    public function valid();
    public function offsetSet($offset, $value);
    public function offsetGet($offset);
    public function offsetExists($offset);
    public function offsetUnset($offset);
    public function count();
    public function getRow($position, $seek = false);
    public function getJunctionRow($position, $seek = false);
    public function toArray();
    public function junctionRowsetToArray();
}

NOTE: Full class located here.

As you can see, the API mirrors that of Zend_Db_Table_Rowset to provide a something that is immediately recognizable. Below is an example of sample usage. For this example, assume there is a typically artist/genre data model that demonstrates a many-to-many relationship. Inside of the junction table we are attempting to track the date that the relationship was created. This examples shows this usage:

$aTable = new ArtistTable();
$artist1 = $aTable->find(1)->current();
echo 'Artist: ' . $artist1->name . PHP_EOL;
// instead of $genres = $a->findManyToManyRowset('GenreTable', 'ArtistGenreTable');
$genres = new Gooey_Db_Table_ManyToManyCompositeRowset($artist1, 'GenreTable', 'ArtistGenreTable');

// iterate
foreach ($genres as $genre) {
    echo '  Genre ' . $genre->name . ' added on ' . $genres->currentJunction()->added_on . PHP_EOL;
}

/**
 * Sample Output:
 *
 *    Artist: Foo Artist
 *      Genre Rock & Roll added on 2010-11-10
 *      Genre Hiphop added on 2010-11-11
 *
 */

Where To Get It & Conclusions

This code is available on my GooeyLabs github account, specifically inside of the Gooey-PHP-5.2-Components repository. (Gooey is my namespace and moniker for my open source code contributions.) Hopefully, those who have found they’ve had issues with the above mentioned fix for Zend_Db_Table_Row::findManyToManyRowset() and junction table data might find value in this class.

I come preaching true hope, against the fallacies.

I’ve heard the arguments for and against database abstraction layers (DALs) time and time again. I must say first, I agree with them all, both sides, equally. Interestingly, I can put the vocal proponents of each side of the argument in one of two boxes: a programmer guy box, or a database guy box. For some unknown reason though, they never seem to see eye to eye.

Honestly though, I like to put myself in the middle of that argument. I see both sides. I think fine tuning an application’s core business with vendor specific features is tremendously important, after all, that is why there are so many competing database vendors. Generally speaking of database driven projects, I feel like planning to use a specific vendor up front, knowing its pro’s and con’s, and tailoring an application to the chosen database’s strengths can only help in the long run. Also, I feel that building a database model first before any code, offers many performance and scalability advantages than does code first development.

That said, I also see value in using a database as a simple data-store when the actual database is not a key component of the overall application. That’s right, it is completely valid to say that the data-storage & database component of an application sometimes is not the key component; a database guy probably will never agree with you there. Just as there are programmers who swear by this code first, database later mantra, there are database developers that will swear by the database first, code later mantra.

The fact is, each project is unique. It’s this uniqueness of projects and their execution that ultimately shapes the perspectives of developers as well as the tools they write and consume. To say that one mantra is clearly a better choice over another is simply being ignorant.

The Use Case of Abstraction Layers

To be honest, I don’t really buy the “I might switch database vendors at some point” argument either, as Jeremy Zawodny points out. For larger projects (on the scale of the facebooks, the twitters, etc), switching the database underneath after a project has been in production is a monumental task- regardless if you have an abstraction layer or not. Chances are, you used some of the database specific features, not to mention, you now have a large set of mission critical data that also has to be ported. Long story short, its never as easy as swapping the abstraction layers database adapter out.

What I will buy though, is there are some problems that fall in thicker end of the Pareto Principle that can be solved with a database abstraction layer. For the uninitiated, the Pareto Principle is effectively the 80/20 rule. In software use cases, when applying this term- the 80% use case is the majority of use cases. These use cases are generally not that interesting in terms of database interaction. To give it a label, we can call these the CRUD, BREAD, or <<insert your favorite terminology here>> operations. That is not to say that these operations are not important, but they are not special. In fact, they are so un-special, that we can just about apply a standard query syntax (SQL 92) to them, and expect that the query is both portable between databases and common across applications that wish to use them.

This is where database abstraction fits in. As a developer, you’ll come across this problem time and time again. A large portion of an application are CRUD screens and the smaller more interesting part of your application is your reporting screens. With an abstraction layer, we are able to code against both a unified API as well as have a layer that will produce consistent and vendor compatible queries. This allows us to build more specialized data access layers (patterns) for multiple database vendors with great ease. You want Table Gateway- done, you want Row Gateway- done, you want Active Record- done. Each can be implemented to tackle the 80% part of the 80/20 rule when applied to the database centric business code of an application.

The Slow Path & The Fast Path

When I talk about this 80/20 rule in terms of the applications we write, I like to further refine the terminology so that it easier to visualize. The most prominent terms that helps developers visualize the 80/20 rule in their application is the slow path of your application, and the fast path of your application. Each of these terms has a set of characteristics that set each apart from one another:

Slow Path:

• Performance is not of primary importance
• Has an interactive nature
• Validation and verification of data are of high priority
• Application to data-store interactions are fairly trivial
• Does not comprise applications core business logic

Fast Path:

• Performance is of importance
• Limited interactive nature, information flow is fairly static (non-interactive)
• Flow of information consist of already verified and validated data (originates from the databsae)
• Application to data-store interaction can become complex (JOINs, SUB-SELECTS, VIEWS)
• Is the core business of the application

To get a better understanding of how the terms are applied, lets look at a typical web application. Generally speaking, there are a few web based forms that users interact with. These forms are the entry point of a code path that does not get a lot of throughput. This is generally because forms are submitted by people, and people can only type and submit forms so fast. In addition to this being a less traveled code path, it also has a few checks along the way- validation of data, and verification of data. Typically, the problems of verification and validation of data are not too unique to the application being executed. In fact, the web forms, validation and verification problems have been solved over and over again by various libraries.

On the other side of the equation, there is the aggregation and merging of the stored data (which inevitably came from the aforementioned web forms.) Since the unique aggregation and processing of this data is the core aspect of business of said application, it stands to reason that this code path will be more well traveled by users. This, is the fast path. The problems solved in this code path are generally unique and since they are unique, it’s hard to find an off the shelf solution to these problems.

Since this is where the money is to be made, it also stands to reason that developers should concentrate their efforts in the fast path of their application. This means they should solve the slow path problems of their application with existing tried and tested solutions- this includes generic forms solutions, validation and verification libraries and yes, database abstraction layers.

Getting Cozy With Zend_Db, a Database Abstraction Layer

Not that we’ve made a use case for DAL’s, what would one look like? Well, I’ll use Zend Frameworks Zend_Db as my use case.

The connection code:

$dbAdapter = Zend_Db::factory(array(
    'adapter' => 'Pdo_Mysql', // could be Pdo_Sqlite, Mysqli, Pdo_Mysql, Db2, or even Oracle
    'params' => array(
        'username' => 'test_user',
        'password' => 'test_pwd',
        'dbname' => 'test'
        )
    ));

You’ll note that since this factory takes a standardized array, it makes it trivial to swap out various connection information for different adapters.

Simple queries:

$data = array(
    'name'        => 'Remember the Milk',
    'description' => '2% Milk'
    'due_on'      => '2009-07-15',
    );
$dbAdapter->insert('todo_list', $data); // insert that data

// or
$lastInsertId = $dbAdapter->lastInsertId('todo_list');
$dbAdapter->update('todo_list', array('completed' => 'YES'), 'id = ' . $lastInsertId);

$dbAdapter->delete('todo_list', 'id = ' . $lastInsertId);

Here you’ll notice the generic and abstracted nature of this API. Since there are several tasks in database interaction that are consistent across the board, those such as INSERT, UPDATE and DELETE, it makes sense that we can create a generic API for handling such interactions. These interactions (INSERT, UPDATE and DELETE) represent the mutation methods of a database and as such, represent the most predominant way of getting data into a system.

For all intents and purposes though, simple SELECTs are fairly standardized too. They are standardized enough as to compliment the INSERT, UPDATE, and DELETE abstractions so that we can find actual rows to do these mutation operations.

Now that we have a simple and consistent API for doing simple SELECTs, INSERTs, UPDATEs, and DELETEs; we can implement something a little more interesting: the table & row gateway:

Zend_Db_Table_Abstract::setDefaultAdapter($dbAdapter);
$userTable = new Zend_Db_Table('user'); // ZF 1.9 feature
$userRow = $table->find(5); // find user by id 5 (primary key);
echo $userRow->username;

Immediately, you should see the inherent value in the above example. Rudimentary and common tasks can now be handled with a consistent and simple API. But what happens when you’ve started using this DAL, and you want to use a vendor specific feature? Well..

// assuming what you want is really REPLACE or INSERT IGNORE from mysql
$dbAdapter->query('INSERT IGNORE INTO configuration (name, value) VALUES (?, ?)', array($name, $value));

// OR
$dbAdapter->query('REPLACE INTO configuration (name, value) VALUES (?, ?)', array($name, $value));

As you can see, the query method of our database adapter will allow us to pass custom SQL into the database thus taking advantage of vendor specific features.

What if you want to combine both paradigms for ultimate flexibility?

// assuming Zend_Db_Table_Row, with a FriendshipReference rule
$friendRowset = $currentUserRow->findDependentRowset('User', 'FriendshipReference');

// collect friend id's
foreach ($friendRowset as $friendRow) {
    $friendIds[] = $friendRow->related_user_id;
}

$inClause = ' IN (' . implode(',', $friendIds) . ')';

$select = $dbAdapter->select();
$select
    ->from('user', array(
        'user_id',
        'related_user_id',
        'became_friends_on'
        ))
    ->where('user_id ' . $inClause);

// interact with driver directly
$mysqli = $dbAdapter->getConnection();
$mysqli->query('CREATE TEMPORARY TABLE friend ('
        . ' `user_id` int(11) NOT NULL,'
        . ' `related_user_id` int(11) NOT NULL,'
        . ' `became_friends_on` DATE NOT NULL'
        . ' ) ENGINE=MEMORY;'
    );
$mysqli->query('INSERT INTO friend ' . (string) $select);

// query new friend view
$friendTable = new Zend_Db_Table('friend');
$rows = $friendTable->fetchAll(
    'became_friends_on > DATE_SUB(CURDATE(), INTERVAL 6 MONTH)',
    'became_friends_on'
    );

While that above example is “a bit out there”, it does show that even with a DAL, if it’s flexible enough, you can code as close to or as far away from the database as you like. Ultimately the mantra here is: lets get the job done in the most effective, efficient and sound way possible.

Conclusions

Simply put, a database abstraction layer is just another tool in the toolbox. You don’t have to completely change your paradigm of programming, nor do you have to apply an all-or-none approach to using a DAL. When applied correctly, you can build out the slow path of your application in little to no time, while leaving extra time for developing and fine-tuning the fast path of your application. And to keep code from becoming unruly, simply apply some best-practices code organization to your project.