Shift8 Creative Graphic Design and Website Development

Let's take a step back for part two. Let's ignore complex (or any) objects and here we're going to go over some more basic (and common) queries that you may need to make. No, there's not really much order to the queries I'm going to go over in this series, sorry. It started as more of an "I'll post about what I'm doing as I do it." type of thing, but I'll try to go over some of the basics now. In fact, if you keep the MongoDB documentation site open for reference as you're coding, I guarantee that most queries you'll need won't be that hard.

Those of you coming from a strong love of MySQL will appreciate part two here. I'm going to go over what will replace your IN() and NOT IN() and I'll even brush on LIKE at the end of this post.

Equal To and In ($in)
So, probably your most basic query will be finding something by a specific value. We typically look up by id and we can do this in Lithium and it's been documented elsewhere. So if you've used Lithium and you've done this type of query, this will bore you...But it's good to point out that the following two conditions are basically the same.

$conditions = array('title' => 'BBQ Chicken');
$conditions = array('title' => array('BBQ Chicken'));

You can simply use a string value for the "title" key (the key being the field name in your collection in MongoDB). These aren't literally the same thing of course, the array() means to go get multiple. So you can add to that list and you've got yourself an $in basically. Here, these two are also work the same:

$conditions = array('title' => array('BBQ Chicken', 'More Chicken'));
$conditions = array('title' => array('$in' => array('BBQ Chicken', 'More Chicken')));

I suppose a difference here is that the first set of conditions would work if you were using MySQL or MongoDB with Lithium. The minute you start using specific operators for MongoDB (all those with dollar signs) then you're no longer coding in a database agnostic fashion. Which probably isn't a real big deal, unless you were trying to make some sort of application that could run with whatever database desired.

Not Equal To ($ne) and Not In ($nin)
Of course we have the opposite available to us as well. These will not be database agnostic, but you can run:

$conditions = array('title' => array('$ne' => 'BBQ Chicken'));
$conditions = array('title' => array('$nin' => array('BBQ Chicken', 'More Chicken')));

$ne being "not equal to" and $nin being "not in". The equivalent of <> and NOT IN() for MySQL users. You can probably start to see how all the operators are translated into this array format in Lithium's Model::find() method. Once you see a few you can pretty much assume the rest will be similar.

LIKE, as in Like Regular Expressions
So real quick I'm going to touch base on the equivalent of a MySQL LIKE in MongoDB. MongoDB is actually way cool in that it lets you run regular expressions! You can't simply put title => '/regex/'... That won't work because you're not technically defining a regular expression, you're defining a string. PHP has no RegExp() object class like JavaScript does. However, that PECL extension you installed for MongoDB does have a class to help you out here. So in order to use Lithium's find with a regex, you'll write something like this:

$conditions = array('title' => new \MongoRegex('/chicken/i'));

This is of course a very basic regular expression, but you do have the ability to create quite complex expressions and also take advantage of some flags; you can look here for the currently supported flags. Above you can see that I'm using the case insensitive flag. Those conditions would return me all of my chicken recipes.

Lithium also accounts for MongoRegex() as well so you don't actually need to create a new MongoRegex() object yourself. You could also write:

$conditions = array('title' => array('like' => '/chicken/i'));

For those of you not familiar with Lithium, let me glue it together for you. The Model::find() method is extremely powerful and simple to use in Lithium and it will help you make queries very quickly. You will write less code, things will look organized, and you won't be pulling your hair out. Not that I think using the straight up MongoDB extension is all too complex, but Lithium makes things a little nicer for you.

$recipes = Recipe::find('all', array('conditions' => array('title' => array('like' => '/chicken/i'))));

In a simple example, but keep in mind that aside from "conditions" you also have keys like "limit" and "fields" available to you in order to return a more specific set of data. $recipes here will be an object with all of my recipes with chicken in their titles. $recipes->data() would then return an array from that object for me to use nicely in my view template. That's one powerful short line of code. Typically to run the query you would be talking about establishing the connection first. You'd write more lines of code. Lithium makes things pretty easy, right?! Due to the speed of MongoDB this also might serve as your site's search engine. Not too shabby. Stay tuned for more, I'll get back to some of the more complex (and sometimes strange and probably not often used...but fun) queries you can make with Lithium and MongoDB.

Ok, so this will probably be a whole big long series because there are a real good handful of different types of queries you can make with MongoDB. Combine that with the complex objects that you can store in the database and you really can get quite complex. Of course I also urge you to think about database design even if MongoDB is so easy that you dive in and just simply forget to think and draw out a schema map. I mean you have regex and all sorts of crazy stuff available. Check out this MongoDB documentation page for more.

So how does one access all this using the friendly Model::find() in Lithium? If you're like me, you're probably coming from CakePHP and the find() will make a lot of sense. You'll be comfortable specifying the conditions key and order, etc. But you won't know what to make about some more of the complex things. So in this series I'm going to provide some simple examples of how to go and form those advanced queries using the Lithium framework.

The Task at Hand
So today I came across a situation where I have an array of data stored in MongoDB. I'll give you the exact context, this is for my Family Spoon project. This is the recipe record holding an array of data for all the families it's being shared with...Or requesting to be shared with upon admin approval. So this "families" array has multiple items each with keys of "family_id" and "approved" and "request_date" and "approval_code" and what not. All different data types right? Strings and booleans and timestamps, oh my! So on some overview admin page we want the user to see all these recipes that they need to either approve or reject. We also need to only show to other people only the approved recipes that are being shared on some other page.

So you're talking about your typical AND operator in MySQL. You're saying go get me records where the family_id is this AND the approved field is that. Great and using the JSON we use dots to get into those arrays and it translates to:

$conditions = array('families.family_id' => 123, 'families.approved' => true);

Or does it? Mwuahahaha....No it actually won't give you the results you expect. What that will do is give you all recipes where the family id is 123 and all recipes that are true. I know what that sounds like...BUT...not where both are strictly the case. So you're getting all these recipes that aren't approved, what gives? That's kinda more like an OR situation with MySQL right?

A little check into the MongoDB docs under the "Reaching into Objects" section should help clarify. It says blah blah blah, "subobjects have to match exactly", blah blah blah. Crap. So then what? 

$elemMatch To the Rescue!
Oh, we use this $elemMatch thingy. It allows us to specify these conditions for more complex objects how we would expect. The question is does Lithium have some array key called 'elemMatch' to use like 'conditions'?? No it doesn't and you don't need to make some custom query...You can use find(). So here's where the thinking cap went on...Er, the trial and error cap. Lithium DOES look for $ in conditions. So how do we write them? Like this:

$conditions = array('families' => array('$elemMatch' => array('family_id' => 123, 'approved' => true)));

Make sense? I don't think it's real obvious, but it's pretty simple looking at it. It's just that it's not well documented (yet). So this will build out the query the way it needs to be and return the results you'd expect, we wouldn't see any "approved" = false recipes in the results. Note that in this case we also are not using any dot notation. Keep in mind that running conditions like the following are also perfectly valid and work.

$conditions = array('families.family_id' => 123);

It just will give you simply all recipes with the family id of 123. So in some cases you will use the dot notation and in other special cases you'll need to see some of those operators with the dollar sign. So, fortunately, Lithium does allow us to use those operators and you do not need to make any sort of custom query and forfeit the use of the find() method. Hooray! Just keep one very important thing in mind...Your find() call may not be database agnostic anymore so if you're using multiple database types or want to make an app database agnostic, watch out. More to come soon.

I love CakePHP and I love Lithium. I imagine many other people will also love both or one or the other or parts from one or the other or neither or enough already. I also suspect that many people will find themselves moving from CakePHP to Lithium. I believe it will be a somewhat common and natural transition. However, we're missing something. Oh right, tons of helpers and other classes!

But wait! There already exists a few helpers for Lithium that act like CakePHP's helpers! So the whole point of this blog post is to say, "Hey! Migration might not be as bad as you think after all." Ok, ok, a few helpers is really not a solution for alllll of the tools CakePHP has but these are, well, helpful. I think Alexander Morland (alkemann) did an amazing job re-writing some helpers for Lithium. You can grab the helpers from his Github repository here. So I wanted to share my discovery and spread the word.

(before - click to enlarge)

li3_assets is a nice Lithium library that I've been working on for the Lithium php framework. It is a very easy to configure (and use) library that will combine and optimize your assets all into one file. How does this impact your web site? Well really, it's how traffic won't impact your web server. What happens is instead of say 5 or 6 JavaScript files if you're using jQuery and a handful of plugins, you'll have just one and it will be minified or packed for you. So you don't even need to be using a minified or packed version of the script. In my test here, I saw 12 requests go down to 5 requests and 302Kb page size drop to 187Kb. The numbers really can be staggering depending on how many scripts (and css files) and whether or not they were originally minified or packed or not. This helps overall performance in a website and helps when you have large amounts of traffic coming to your website. The server will serve out less requests and smaller amounts of data so it can handle more visitors.

(after - click to enlarge)

For JavaScript you have the choice of using JSMin or Packer and for CSS you have the choice of using CSSTidy or a simple whitespace and new line removal. This removal code is very simple, but also very safe. The reason I left a few options for both is because sometimes when you start minifying and packing code, things break. It's rare, but possible. Some of these tools have options and those can be passed as well in the configuration.

As an added bonus, this library also will run less files through phpless. Less is a CSS shorthand syntax that will be converted into actual CSS files. The idea is to speed up your development time. After the files are run through phpless and converted, they are then run through CSSTidy and combined into the single CSS file.

These single combined and condensed assets will live in a directory of your choice and be named with a hash string. This hash was generated from all of the assets used. This way every page that has different assets (JavaScript and for the CSS), will have a different combined hash and therefore different file. Updates? If you have an update to a CSS or JavaScript file, you'll have to remove this hashed file so it can be regenerated. In other words, you may not want to enable all this until you're live with your site.

What's this about images? Images can be "optimized" as well. All images placed with the Html image helper on any given template can be output as base64 data URI's if you so choose. The major drawback to using data URI's is that they don't work on IE 6 and IE 7. You can find some JavaScript solution to this problem, ignore this problem, or don't use data URIs. The benefit? Well, if you're caching your view templates, you've now just embedded all images within the HTML code of your view template. So you can probably take a page down to about 3 requests. One for the JS, one for the CSS, and one for the HTML code plus images within. Now you're really starting to save the server quite a bit of work.

So how easy is this to use? How do you use it? Lithium makes this simple. You will run the Libraries::add() method and add li3_assets. Within the same call, you'll pass all of the configuration options. Then in your layout you'll call $this->optimize->styles() and $this->optimize->scripts() instead of $this->styles() and $this->scripts(). So it runs based on the "scripts for layout" feature. Where any script you place within any of your view templates that contain the array option "inline" as false, you will have those be put into the layout template up top within your head section (or wherever you chose in your layout template). The li3_assets library's helper basically intercepts these assets and runs the magic. It then simply returns the HTML code for the assets to be embedded into the page. Simple. So this makes it nice when you want to turn off the optimization. It also makes it flexible so you can have certain layouts that optimize and others that do not. Here's the code all formatted. First, the library configuration.

Libraries::add('li3_assets', array(
  'config' => array(
       'js' => array(
            'compression' => 'packer', // possible values: 'jsmin', 'packer', false (true uses jsmin)
            'output_directory' => 'optimized', // directory is from webroot/css if full path is not defined
            'packer_encoding' => 'Normal', // level of encoding (only used for packer), possible values: 0,10,62,95 or 'None', 'Numeric', 'Normal', 'High ASCII'                                                      
            'packer_fast_decode', => true, // default: true
            'packer_special_chars' => false // default: false
       ),
       'css' => array(
            'compression' => 'tidy', // possible values: true, 'tidy', false
            'tidy_template' => 'highest_compression',
            'less_debug' => false, // debugs lessphp writing messages to a log file, possible values: true, false
            'output_directory' => 'optimized' // directory is from webroot/css if full path is not defined
       ),
       'image' => array(
            'compression' => true, // uses base64/data uri, possible values: true, false
            'allowed_formats' => array('jpeg', 'jpg', 'jpe', 'png', 'gif') // which images to base64 encode
       )
  )
));

This shows all of the options, but there are defaults. Then in your layout template put the following within the head section of your template:

echo $this->optimize->scripts();
echo $this->optimize->styles();

That should do it. Now any script in any template using that layout defined like this:

echo $this->html->script('scriptname', array('inline' => false));

...Will be sent to the layout and intercepted and added to the list of scripts to be processed. The same goes for CSS as well.

To convert all images within a template using the Html::image() helper method, you'll simple put at the top of the view template:

$this->optimize->images();

This doesn't output anything, but it does trigger a method that will apply a filter to the Html::image() method. Filters are a real nice feature with Lithium. Now any call to the image() method will be intercepted and the data will be converted to a base64 data URI and returned. I think the use of images as data URI's is limited, but can be a nice feature to have. I think it's especially handy on certain pages where you know a lot of traffic will hit and also helpful for mobile devices. Remember that making another request (connection) takes time and while we don't notice it on high-speed internet, mobile devices will notice it. It also changes when images get loaded and rendered in a browser as well. So I feel it's an under used trick of the trade.

Again, you can grab this library from the repository on the rad-dev site. http://rad-dev.org/li3_assets.

The Lithium framework is a very flexible framework and it gets along quite well with other classes. It has auto-loading features and utilizes namespaces in PHP 5.3. So any class or library of classes is typically pretty easy to use. Take for example the wonderful Swiftmailer project. This library makes sending mail via mail, sendmail, or any smtp server (including Gmail) very easy. 

So how does one get it working with Lithium? Quite easy. You can simply move all of the Swiftmailer files into a folder under your app's libraries folder, named something like "swiftmailer" ... Why not? Obvious enough. Under there you should have a lib folder with all of Swiftmailer's classes and it's auto-loader. 

Great. You're actually half done. The next half is to include the Swiftmailer library and auto-loader in your bootstrap. You could of course also just follow the directions on the Swiftmailer site and run require_once(...); somewhere in your method of some class, but we want to do things the Lithium way. So use Libraries:add('swiftmailer'); instead. Though, we will need to pass some options to bootstrap that auto-loader. 

Libraries::add('swiftmailer', array(
     'bootstrap' => 'lib/swift_required.php'
     )
);

That's all there is to it. Now you can use Swiftmailer anywhere in your application. You just should note that you're working with namespaces in Lithium and the Swiftmailer class doesn't use a namespace within your application. So you'll need to call its classes like so:

$transport = \Swift_MailTransport::newInstance();
$mailer = \Swift_Mailer::newInstance($transport);
$message = \Swift_Message::newInstance()            
              //Give the message a subject
              ->setSubject('Test Message')
            
              //Set the From address with an associative array
              ->setFrom(array('your@email.com' => 'Your Name'))
            
              //Set the To addresses with an associative array
              ->setTo(array('to@email.com'))
            
              //Give it a body
              ->setBody('Here is the message itself')
            
              //And optionally an alternative body
              ->addPart('Here is the message itself', 'text/html');
              $result = $mailer->send($message);

This would send a message using mail(). You can refer to the Swiftmailer documentation for other usages, but you'll just need to remember to add the first \ slash so you're in the proper namespace. You can also put the "use" command at the top of your class like this:

namespace whatever\may\have;
use \Swift_Mailer;
use \Swift_Message;
use \Swift_MailTransport;

class whatever extends something {
}

Then you won't have to remember the beginning slash to use the classes and when you follow (or copy and paste) from the Swiftmailer documentation, you won't have to really adjust much or think about it. I don't mind adding the slash personally, I don't like to put "use" up top in this case because you're talking about a few different classes that you will be using (the "Mailer", "Message", and "MailTransport") and I just find it a hassle and just makes me write more. Additionally, I may only have one method that actually uses those classes.

Anyway, I hope this helps someone out there with using Swiftmailer with Lithium. I'm sure it's fairly obvious, but that bootstrap feature is really cool and I wanted to put it out there for those not familiar with the framework or those who didn't take advantage of the auto-loader with Swiftmailer and Lithium. There are actually several ways you can use Swiftmailer (and other 3rd party classes for that matter) with Lithium. I just really like how Lithium keeps things consistent with the Libraries::add(); method.

What is aspect oriented programming (AOP)? Well, you could search Google and find tons of information about it and also read Wikipedia. Or you can essentially just understand that it's a very high level approach of handling program functionality, making it really modular in design. IBM has a great article that's a little easier to read.

So how does Lithium handle this idea of AOP? With it's filter system. Since it takes advantage of PHP 5.3 it has the benefit of lambdas and therefore its filter system was made possible. Essentially, any method you write or many (not all, but many) within the core framework can be filtered. What does "filtered" mean? Well essentially you call a method on the class passing the method as an argument, Class::applyFilter('method' ...); and you will inside the applyFilter() method also define and pass an anonymous function and pass along the chain. Within that function you can perform other logic and then return the chain so that everything can carry on. Or maybe you don't return the chain, maybe you stop right there and exit out with some data. This is basically linear event handling. Each filter is going to do its thing in order as called.

So now what's the big deal? Why is it so much better? Well, using the example of CakePHP's callback methods in the model...Let's say you have one tiny specific situation where you need to perform some logic before you find a record. You could use the beforeFind() callback method. Great, but then let's say you have to perform that in another model. Oh snap, well, we'll just add that into the other model. Then another model later on? Really? Come on the client must be beating you up. Then only at a certain time of day? Under other crazy conditions? Ugh. Ok, so you can understand that you are very quickly ending up with messy, borderline unmaintainable, code. You have to go back to many different places and add in new code. You may even need to write entirely new classes and access methods in those classes to get the job done. 

So if instead we applied filters from a different area in our code, we can keep all of our "advice" or ways of dealing with our "cross-cutting concerns" in one place. Furthermore, since we're in an event chain with Lithium's filter system, we can bypass things once we have what we need and structure things in an efficient order. 

The new CMS I'm working on, Minerva, will take full advantage of Lithium's filter system. Essentially, it will account for a very large piece of how the CMS is extended with plugins (Lithium libraries written specifically for Minerva). For example, the "Page" model is the main model that deals with records for, well, pages on the site. These pages can be basic simple pages or blog entries or photo galleries or anything really. What "flavors" or "decorates" pages? Libraries. So there's a "blog" library that gets dropped in and poof, there's a blog. In fact, there won't even be any "installation" or database setup required (thanks to MongoDB). Drop the files and literally - poof. Now the blog library has a Page model itself and it extends Minerva's Page model. It adds new schema so we can gain extra fields that we may need for a blog and, more on topic here, it applies filters. 

One filter it applies is to the model's find method. It has an index listing method in the controller and when called, it will make a call to the find method to get the records. The blog library's filter says, "Hey, since you're talking about me, go grab just my records." Now you have an index listing of blog entries instead of all pages. Sure, this is a very basic example and you could simply have the controller method pass along a URL parameter that changes the find query to only return records belonging to the library just the same. Let's say it was just a tiny bit more involved so that each time we added a new library, we'd have to go back and add in some more code so we can actually handle the new library.

Ok, so you can see in this admittedly poor example (use your imagination for more sophisticated cases), that you have to ultimately keep going back and altering a "core" piece of code each time you add in some new functionality. Maybe it's not the "core" framework, but still in our case the "core" app that we really don't want to touch because when there's an update, we want to ensure we can get that update without conflict. With the filter system, you don't have to touch the core app. It's a more hands off, more abstract way of handling the concern. Also look at where it's been set? It's been set inside the 3rd party add-on code. The main application doesn't even have a clue about what's going on and the core code base was never touched. It's not dependent at all. All this without having to setup some big API system that adds more beef to the app and has to have every little detail worked out. Then what if our "API" updated? Oh, well then we have plugins for the CMS that are for version 1.x and plugins for version 2.x etc. You'd need to update the plugins to run with the latest CMS. Minerva specifically aims to not fall into the same problems that most other CMS' out there do.

Of course you can also get away solving some of these problems with simple class extension, but you'll ultimately run into problems. Obviously maintaining the code. Say you do extend and override some methods then one day you pull down the latest framework code from the repository and poof your site no longer loads. You have a white screen and you have no idea what happened. Then you have to debug. Yuck. Hmmm, and wouldn't a nice logging system help you out here? Oh look, another great use for the filter system.

So Lithium's filter system is extremely valuable. It's something that I believe will distinguish the framework from others and will help developers build faster and more maintainable code. Stay tuned for some more examples and also more information about my new CMS, Minerva. You can actually dive into some of the code and check out some example uses of the filter system in Minerva's code found on github. Minerva is still in it's early stages and is subject to change, but it should accurately represent the direction it's heading and the blog library's Page model should give you an easy to understand example of the filter system.

Of course, I fall in love with a new framework the minute that I'm top of my game and have an answer for (almost) everything, every project and every need. So like the title says, Lithium is the gem that swayed me away from CakePHP. I'm going to continue to use both. CakePHP is still an extremely good framework that allows you to very quickly produce sites/apps.

I originally planned to get all super technical and prove how wonderful CakePHP and Croogo are. They are. It is worthwhile running load tests? Yes, I think we are lacking a lot of statistical data out there on modern open source CMS'. More than just the statistical data, I was also going to throw in actual real world data and (as delicately and legally as possible) explain which and why certain things failed. I'll get back to it, I promise. I've just taken a little detour.

So this framework...Lithium. What is it? Well, it's not super new, it's been around for some time, but it's definitely gaining some steam and is starting to sway some CakePHP people. It is what was going to be Cake3. Basically an MVC framework (with similar conventions to CakePHP) but geared strictly for PHP 5.3 and above. It takes advantage of the new features in PHP 5.3 and I think that's just honestly where I want to keep my head right now.

That's not to say CakePHP won't either and that Croogo or CakePHP are bad. No, I'm still using Croogo and will still be building add-ons for it and following the community tightly. Croogo and CakePHP are absolutely positively the first thing I grab for when building new sites 90% of the time. I don't get huge sites to work with every day you know. 

However, what about those big sites? Ah, the ones that gave me headaches? The ones I slept at the office (no joke) in order to build? Sleep deprived stumbling across downtown Manhattan trying to catch a bus to get some real sleep all the while trying not to get hit by a taxi....Oh right right. Yea, those good old sites. I believe...No, I know that they don't have to end up being horror stories. I need to keep my sanity and I don't want to be in that kind of position again. So I think by learning Lithium and maybe even reviving an old personal project...I'll be able to have a solution for those that keeps me sane and happy. Just like Croogo and CakePHP are my solutions for small to medium sized sites and custom apps...I think Lithium will be my solution...or my shield to the monsters out there. Of course this is all assuming that I have a choice in how the monsters are built. Sometimes I do, sometimes I don't. 

I will also probably write up some information, discoveries, and tutorials for Lithium as well...Right now their documentation isn't the best, but there's an extremely helpful and knowledge able community. You can find out more about Lithium here.

I just finished up my cycle plugin for Croogo CMS. It requires the very latest version of Croogo (from github) and also the MeioUpload plugin (a wonderful CakePHP upload plugin/behavior). 

The cycle plugin allows you to upload images and include with them a title, caption, and link. You can then display the cycle on any node in various locations. You can display the same cycle in many nodes and you can share cycle records across cycles. Each cycle on each node can have a different visualization and size. 

I've included five jQuery plugins to display the cycles/galleries. You can easily add your own by creating elements within the plugin's elements view folder. You can also modify the default visualizations I've included by copying the element to your Croogo theme's elements folder (same pathing) and you can have altered visualizations based on each theme in your CMS. This should come in handy for obvious reasons.

The plugin will cache the queries made to retrieve the cycle records (not that it should ever really be a complex query) and it also caches the view element. This plugin uses my image version component and helper in order to generate the thumbnails (it requires GD library, which most web servers should have) and that component will only generate new thumbnails for modified source images. So again, some more caching features.

I've included in this node a little demo for you to enjoy. You can find more information and download the Croogo Cycle plugin from github.