Shift8 Creative Graphic Design and Website Development

I'm very excited to see a few more admin GUIs for working with MongoDB out there. I'm on Windows so I decided to try MongoVUE. Works great! I really haven't used it for a lengthy period of time, but my first reaction is that it's really good. For the OS X users out there, I think MongoHub might be worth taking a look at. 

So how to connect? Well, if you're working locally, it should be pretty trivial...But what if you're trying to connect to a firewalled server? Tunnel time. However, MongoVUE (I don't know about MongoHub) doesn't have built in tunnel support. That's probably the only thing I don't like about it so far. Not to worry though, you can simply make an SSH tunnel using PuTTY (or KiTTY, which I like better). Here is a really good article on how to setup a tunnel, it takes no time at all. After you're done, connect to localhost on the port you defined for the tunnel and you should be set!

MongoVUE gives you several ways to view the data in your collection. You can use the "text" view which shows you a JSON string that you're probably used to see in the mongo console, but you can also choose a tree and table view which is probably what you're hoping to see with an admin GUI for any database. The only other thing I might like to see is an actual console within this tool so you can manually enter commands. While I don't see that (if it's there somewhere) I do see that it will return the commands used while browsing and performing actions with the GUI. This is nice because you can then copy and paste those commands to an actual terminal window or something. I'm definitely looking forward to seeing feature enhancments for this tool.

I wrote a previous article on using MongoDB's non-locking updates and it's $inc operator in order to allow for a simple counter that would create minimal impact on your database. Related to that, let's suppose we want to make a rating system. Say it's a star rating; however, it can be any kind of voting or polling system really.

So here's the trick. You're storing this star rating on a piece of content (a document) and you already queried that so you can display the data on the page. We're in NoSQL land so we don't use JOINs and we really don't want to query another "rating" collection of some sort. There's no point. So we keep under this document a "ratings" array. Now when you load the document you can simply count and divide to get your average with the language of your choice. You could also make another count() query to MongoDB but again we want one query.

So how do you store ratings? You want to do an non-locking update to the database and much like incrementing, we want the impact on the database to be low. It's not that MongoDB isn't an amazing database that is more than capable of handling these updates, even if you need to make a read first, but we want to be efficient. Let's assume we're making millions of queries and this is all adding up to save us more than a few pennies in hosting costs.

Ok, so let's take a look at the $set operator. It's a simple one, nothing magical. It's just some cleverness with what we are setting. Under this "ratings" array we're going to set each users IP address (substitute for user id if you have some sort of authentication system you'd like to rely on instead) into this ratings array with a value. 

$set will set the value or update it. So you don't have to worry about repeat voters! They simply will change their vote and not skew your results. If you don't want the users to be able to vote twice then you can simply disable the link to vote on the page by looking for the IP in the array of ratings. Which, yes, is a simple deterrent and the user could technically lift the URL to hit to make the request to vote again...But again, your results aren't skewed and if you need further protection you can come up with something else. 

Let's apply this to my favorite framework here, Lithium. Here's what your query might look like:

$query = array(
  '$set'=> array('rating.' . str_replace('.', '-', $_SERVER['REMOTE_ADDR']) => $rating)
);

$conditions = array('_id' => $some_id);

$result = Page::update($query, $conditions, array('atomic' => false));

Do you see the gotchya? It's the IP address. You can't have a key name with dots in it. When passed to MongoDB it's going to translate to object hierarchy. So you'd end up with something like: rating" : { "192" : { "168" : { "126" : { "1" : "3" } } } } ... No good. So if we replace the dots with dashes, underscores, or hash it, etc. then you'll end up with something like this instead: "rating" : { "192-168-126-1" : "5", "192-168-126-2" : "3" }

Now if the same IP address voted again it would simply change the value in the "rating" array on the document. This way a user couldn't skew the rating by contstantly clicking on a star rating widget, submitting a form, etc. Of course you can further limit the impact to your database and server by setting a cookie.

Now it's just simple math to get the average. If you're using PHP, you'll run a count() on the ratings array and then add up the values and divide. Simple!

What would you have to do alternatively? Well, you might store another field with all IP's or user id's that voted in addition to a rating field... But now you've disassociated users with their votes. You could also store a different field for each option and then use $addToSet to add the IP to each option. Great, but then you allow a user to vote once for each option. You won't be completely skewed, but you wouldn't be as accurate. What about MySQL and relational database land? Well, you probably don't need to hear about all the ways you can use JOIN and how many rows you'll be scanning through to get your results and what kind of indexes you need to make sure you build.

Hope you found this little schema example useful in a schemaless world.

Minerva, my CMS built using the Lithium framework for PHP 5.3+ is really coming along. Or at least I think it is... I want to keep increasing awareness of it, but I don't want anyone getting the wrong idea. It's still just a starting point for a CMS and not actually a finished and usable product. However, it is getting pretty close.

I just hope that enough time goes by that gives people the chance to weigh in and give me their feedback and thoughts before I get too far along and commit to something irreversable (or something difficult to reverse). I've used many CMS' over the years, far too many to list (or remember), so I've drawn from that. I've added what I believe are the best parts of the CMS's I've seen and used...I've also simplified.

I truly believe there is no such thing as a 100% single serving CMS. People choose different CMS' for different reasons and to achieve different goals. Then you have to edit them and either code or fine add-ons in order to make it complete...That's just for those who actually use a readily available open source CMS. There's also the custom applications that get built. I've been there too. I've done it all and; in the end, it's all just a good waste of time. I'm hoping to strike a middle ground between a jam packed fully featured CMS and a collection of a few class that you put together using a framework. Using the Lithium framework, I think there's a flexibility for the CMS that can't be found elsewhere. 

If you aren't familiar or haven't read a blog post of mine in the past about it...It's a CMS for developers. It's not for just anyone to grab and use and I doubt there'd ever be a "for dummies" book out there for it. That said, I hope that I'm not the only the one that will benefit from it, though I definitely think I will benefit from it so that means it has some value. 

I've revamped it a bit and consolidated a lot of code. It has an admin interface and system for adding templates to help you build out the front end as well as gracefully modify the admin interface. There's also access control so technically it has most (if not all) the features you'd want in for a CMS excpet some of the "friendly" things like a bunch of helpers and tinyMCE. Again though, it's under heavy construction.

Give it a look on Github and check it out clone it. It's usable to the degree that you will get the idea and understand the general direction of it. I'm really hoping to get some feedback...Otherwise maybe it'll only end up being of value to me, but I hope not. Speak now or forever hold your peace. 

It's 2011, I firmly believe that we should have systems smart enough to automatically tag our content. Our tweets, our FaceBook wall posts, our blog posts, etc. There's even face detection APIs available out there. So surely getting context and keywords from a copy should be easy, right?

There's a few major players in this arena and sure enough there exists some free APIs for getting keywords and context from copy. Reuter's OpenCalais, Alchemy, and Yahoo term extractor are all great services to use. However, how current are they?

I have a new project where I want to auto tag content (I also plan to add auto-tagging to my Minerva CMS), but there's a few issues. First, the content is going to be very very recent so these services may not have had a chance to pick up some of the terms. For example, "iPad" doesn't come back when using the Alchemy service. It doens't know that "iPad" like "iPhone" (which does come back as a hit) belongs in the Technology category and should be a keyword. Obviously this will change as time goes on, but as of Jan 2011, it's apparently not currently in their index. Yahoo's keyword extractor does seem to pick up "iPad" interestingly enough. Maybe you have to use multiple services for the best coverage.

So since you've found my site, you may or may not know that I'm in love with PHP, the Lithium framework, and MongoDB. Well, I am. So I took the Alchemy API for PHP and converted it into a Lithium library. Basically this involved namespacing the classes, breaking them out into separate files, and making sure the could depend on each other via "use." I then added two methods in the API classes (there's a CURL class and a "normal" one) that help out a bit. One grabs config information set when you call Libraries::add() so that you don't need to set your API key everytime you go to make a call and the other method is just for convenience so you can call the API statically. It's really not completely static of course, I didn't go through and re-write the API, it's just a static wrapper. Now you can simply do something like:

use alchemy\AlchemyAPI;
$data = AlchemyAPI::call('TextGetRankedKeywords', array('The iPad will feature a slot for reading SD cards unlike the iPhone', 'json'));
var_dump(json_decode($data));

With that, you'd then get back a nice array of data from the service. Again, I just like calling things like this statically, but you could also say $alchemy = new AlchemyAPI() and yada yada yada. If anyone wants this Alchemy library for Lithium, let me know because right now I'm unsure if I'm going to use it myself. It wasn't a lot of work to modernize these classes, but it might save you some time. I'll probably also end up making something for OpenCalais and Yahoo work as well for Lithium.

But back to our issue at hand. How do I go about getting relevant keywords? I'm half ready to spend an insane amount of time trying to use an artificial neural network or something (cool fun fact, there's a few for PHP actually)...Perhaps keep a list/dictionary in MongoDB and search against it? I'm not sure. There has to be something clever that MongoDB can help out with. Hopefully someone out there might know and leave a nice comment.  

For now, I hope that maybe I gave you a few ideas and some resources to go look at if you've never attempted this before (or even if you have and were unaware of a few of the mentioned services) and keep an eye out for some possible Lithium libraries for this stuff and definitely expect to see it in the CMS I'm working on.

As I was pointed out today in the wonderful #li3 channel, MongoDB has an $inc operator, awesome! One I didn't see before (there are oh so many handy ones). So how do you write this in Lithium land? Well because we like being efficient and all, let's also look at how to do this without first reading the record (ie. an "update()" call).

Model::update(
// query
array(
  '$inc' => array(
    'field_name' => 1
  )
), 
// conditions
array(
  '_id' => $id
), 
// last but not least
array(
  'atomic' => false
)
);

There you have it. To decrement you'd put -1. Now also note that in my case (and it's worth mentioning) the value I had to increment was within an array actually. To get at that field you can simply use dot syntax to jump down into it. So the query would be more like:

array(
  '$inc' => array(
    'field_name.value_within' => 1
  )
)

Putting it altogether you can in pretty much one line, one call, very nicely increment or decrement values. Since MongoDB is so wonderful this gives you a very easy way to keep say a real time hit count or something for content within your Lithium application.

So if you're using Lithium and MongoDB you can set in your model the $_schema property which will allow you to set data types for your fields. So things may look like this: 

protected $_schema = array(
  '_id' => array('type' => 'id'),
  'created' => array('type' => 'date')
);

With that, the "created" field will be of type "date" which will be a MongoDB date object. This date object isn't going to display properly when you got to print it back out in your view templates. You'll end up with some weird number value. The timestamp is actually within this date object that's returned under the "sec" key. So it would be $document->created->sec in your view template (assuming $document was your result from the database).

How about formatting to a pretty date? If you're coming from CakePHP or some other frameworks you may be spoiled and have a time or date helper. Lithium is pretty lean and leaves that as one of your responsibilities. Fortunately, alkemann has created some helpers that mimic some from CakePHP for Lithium. These include a Time helper. So in your view template, you can use:

echo $this->time->to('nice', $document->created->sec); 

That will output something more readable like, "Fri, Jan 14th 2011, 02:29" instead now. Don't have or want those helpers? You may be crazy, but you can also use:

echo date('Y-M-d h:i:s', $document->created->sec); 

However, I would definitely give those AL13 helpers a look, they are pretty nice. The important thing to remember here though is that when using MongoDB date objects, your timestamp is under this "sec" key. Of course, you can always store timestamp integers or string values, but then you're really crazy.

I'm pleased announce, after a few days of really focused work, I've released a new version of Agile Uploader. For those who haven't seen it before, it's a Flash client side resize before upload, multiple file upload tool. Why another multiple file upload tool? Well, swfupload and flash file uploader and uploadify and plupload and all the others (too many to mention) work great and all (and believe me, their authors do a great job and it's not easy and the tools do work well), but they didn't quite do what I needed. First off, several of those I just mentioned don't have client side resizing for image file types. What I mean by that is Flash can scale and re-encode new jpeg files that can then be uploaded to your server rather than having your server do the resizing. Obviously this helps with hosting costs, site scalability, and for those using shared hosting. Second, Agile Uploader is fairly lean and extremely customizable. Agile Uploader allows you to customize how things look and function with regard to what type of files you allow and how you want it to resize images (if at all).

Agile Uploader asynchronously resizes and encodes images so you can keep attaching more files while it's working. The tool has a bunch of other options as well as a tight communication with JavaScript. I wrote a jQuery plugin to accompany the tool, but you could write your own JavaScript should you choose (or alter the jQuery plugin). Flash will pass to a defined JavaScript function various event data as it does its job. In fact, you can control just about every facet of the upload tool right from JavaScript. The only thing you can't use JavaScript for is the "browse" or "attach" button, it's against ActionScript's security rules sadly. However, you can customize the button that appears in the swf by specifying the path to your own image(s). You can also send the form via JavaScript if you wanted to automatically or on some event (like a button or link click). 

Documentation for this new version is on the way, there's been a lot of changes since version 2.x. It's become a lot more organized and simplified now. One of the long awaited features includes the ability to select multiple files at the same time. While the tool always uploaded multiple files at the same time, the user had to click the button to browse for each file one at a time. So the user experience has improved quite a bit along with a few minor bug fixes. 

You can see more about this tool, and a working demo, on the Agile Uploader project page.

One of my New Year's resolutions is to post more to my blog. Even if it's short posts. That's fine. So to that effort, I'll leave you all with a little quick tip for the Lithium framework. If you want to redirect a request to an external server or any URL you can do so in one of two ways.

Redirect Controller Action
The first method is how you might do it with the CakePHP framework...Simply route all your URLs to a controller that has an action which will call $this->redirect() and pass the argument (the URL you wish to redirect to). That might look like this:

Router::connect('/about', array('controller' => 'Redirects', 'action' => 'go_to_url');

Then you'd need your "RedirectsController" to have a method something like this:

... 
public function go_to_url($url="/") {
    $this->redirect($url);
}
...

That's great and will work but there's actually a bit more involved. First off, you have to have some extra controller (which could handle all sorts of "legacy" issues if migrating from an old codebase, so that's fine if you want to do so) and that means a bit more code than necessary in most cases. You might also have to tell it not to use a template, etc. 

Router Closure
If you didn't know already, Lithium allows for closures in its Router::connect() method. It's very very nice. So here's what you can do instead: 

Router::connect('/about', array() function() {
    header('Location: http://www.site.com/about.html');
    exit; 
});

That's all there is to it. With this method, you are also bypassing a good chunk of the framework, so it should perform a bit better too. You can get a little clever there and add some code to check if the URL exists and if not, re-direct to a 404 page on your server or something too. Go wild

So of course me being me I'm working on a million projects all at the same time. Lately I've been trying to push a lot on my Lithium CMS, Minerva. As a byproduct of all that I am happy to release li3_access, an access control class for the Lithium framework. Don't get too excited, it won't handle your ACL the way you may be looking for out of the box, but you can definitely use it to handle your ACL/RBAC. It extends the adaptable class so expect adapters for RBAC in the future.

Not to go against the flow, there's actually a spec for an ACL system for Lithium. I built my library to fall in line with it so that hopefully there will be less wheel re-inventing. Not that it is a huge class or anything, but it was well thought out and it does come with test cases -- bonus! :)

So of course it doesn't really do much without an adapter and while I don't have a robust RBAC adapter that may use some sort of tree system from the database, I do have a "rules based" adapter that comes bundled with the library. 

You can basically think of this rules based adapter like validation. There's even a method within this adapter called "add" and it does exactly what the validation class does. It adds rules to check for access. If any come back false then it returns an array with some data; including a message explaining why access was denied and a redirect URL. You can then check multiple rules at once. It's very quick and easy to use. Many sites will be able to use this alone and won't require some larger database ACL system. This really involves no queries (well unless you needed data from the database to determine true or false).

Using this adapter you can get very detailed control over access rules; for example, you can lock out specific users during specific times of the day for specific content if you wanted. That's simply something you can't do with a traditional RBAC system. You would have to use your RBAC system and then add to that extra code to catch those special conditions. This allows you to neatly organize and check against your rules much like validation.

For more information and the code, check out li3_access on Github. Remember to keep your eyes open for new adapter in the future, or feel free to contribute one! Thanks, hope you enjoy.