You all know not to put your passwords into the database in plaintext. Catalyst and DBIx::Class::EncodedColumn make doing this super easy and completely secure.

First off, you might want to check out the wikipedia article about cryptographic hash functions. The gist of it though is this: a password stored in plain text is obviously compromised if the passwords file gets into the hands of evildoers. You can “hash” the passwords and they are now harder for the attackers to transform into plain-text. If your password is good it is nearly impossible, but basically what can happen is that the attacker uses the algorithm to generate hashes for every word in the dictionary or whatever and now they basically can crack all the basic passwords.

You can take it a step further and “salt” your passwords (wikipedia salt article.) A simple way of doing that is just to concatenate some string onto the end of all of your passwords. This will make dictionary attempts useless unless they know your salt. Typically when using a salt the salt is kept secret.

And then you can have a unique salt per password. Imagine a scheme where the salt is $username$id. It would require the attackers to basically generate a dictionary per user!

The scheme we’ve settled on uses Eksblowfish. The DBIC Component for it actually uses a 16 character randomly generated salt for every password. Nice!

Ok, so how does one apply such sweet, sweet code? First, (always) set up your model. This is a slightly trimmed version of ours:

Easy peasy! The encode_check_method option for password basically puts a method in your result class that you can call with a plaintext password and it returns true or false if the password is legitimate. The nice thing about that is that if you decide to switch to some other kind of hashing, your controller stays the same. Model code for the win!

Next up, the Catalyst configuration. This was what took me a while to find, but thanks to mst I finally found it yesterday. The package we use for auth is the same one everyone uses in Cat: Catalyst::Plugin::Authentication. The docs that I was looking for specifically were the ones for Catalyst::Authentication::Credential::Password. So after reading those docs, the following is the catalyst config snippet one would use for these nice passwords:

Note: the password_type of self_check is what tells the controller to just call $result->check_password($plaintext).

So there you have it. That’s all the code you need for secure passwords with Catalyst. If you make a new project and your users passwords get compromised it is your fault.

Have a nice day

Today at work I had to do some validation that we haven’t yet had to do for my project at work. I’ve always thought that for validations exceptions are the way to go. I’ll explain everything I did so you guys can benefit/critique.

First off, I used Exception::Class to create my exception classes:

Also note the use of Moose::Util::TypeConstraints; we’ll come back to why I did that in a bit.

The following code is a method from a DBIx::Class Result class. Nothing too surprising here. It basically creates an exception if someone tries to use a nonexistent bin or a box that the bin doesn’t contain.

Next up is the Catalyst action which calls this method. This is the first part of the code I’m excited about. I’m using TryCatch for the syntax sugar here. Note that I get to do a catch based on type of exception. This is why I had to use Moose to define the class_type’s above. You’ll also note that I recast the Exception as a “UserException.” I’ll note why next.

And then this is the final (server side) method that wraps it all together. This belongs in the Root controller of our Catalyst app as it takes care of all of our errors. Basically what’s going on here is that if there are errors we want to set a 500. We show the error raw if the server is in debug mode or if it is a user error. There is a small subtlety that there can possibly be more than one error. For simplicity’s sake we show all the errors if we are going to show one of them.

I won’t show the javascript right now as it’s messy and most readers of this blog aren’t hardcore Ext users. But basically what happens is that we have a global listener for all connections that fail (aka, don’t return with 200 OK) and I have some special code for various cases. For example, I have an unauthenticated case which allows the user to login (and then it seamlessly retries the query,) I have an unauthorized case if somehow the user tries to do something they are not allowed to do, I have a user error case, which will basically display the error verbatim, and then I have a server error message, which will output the raw exception (or the vanilla message above.)

The beauty of all this is that now that I’ve written the code sufficiently generically I should only need to do an ACD::Exception::UserException->throw(message=>…) to show the user an error window at any point in the program. Pretty sweet huh?

Ok, this is just too crazy to not record and relate. By now anyone who has read much of my blog or interacted with me should know that I use a significant amount of javascript on my current project at work. Because I like to keep everything nicely organized, 95% of the time each class has it’s own file. That means I have to tell the server every time I add a new class. No big deal really.

So yesterday I created a new form, and I added it to the list. When I refreshed the page the form failed to load. Well, that could have easily been an error in the syntax or even runtime logic of the form, so I start looking for firebug errors and whatnot and don’t see any. So I look at the debug output of Catalyst::View::JavaScript::Minifier::XS to make sure that it tried to include that file. It didn’t! Ok so clearly I didn’t save the file with the list of JS files. So I ensure that I’ve saved it. Still no luck. So I change another part of the list, a file that is loading, to see if that gets taken out of the list. Maybe I misspelled the filename you know? Nope. List remains the same. Ok, so then this must not be the canonical list. So I change a fundamental part of the list, to see if everything still works after I change that. Nope, now nothing works, showing that this list clearly isn’t nothing.

Bizarre. So I go home, assuming that I was just tired. Today at work, after working on a bunch of other stuff, I get back to it. So first I output the list itself, to make sure that it’s what I think it is. It is. Then, I open up the code behind Catalyst::View::JavaScript::Minifier::XS. I added some debug stuff where it loads the files and look at the output. Nothing prints out. That’s weird… So I change it from a warn to a $c->log etc. Still no output. So I’m editing the wrong file, obviously. I go to rename the system version of this file so I know it’s using my local copy (I have changes that haven’t been accepted by upstream yet.) Oh wait…it’s already renamed from yesterday…

So that’s weird. Ok, so I put some debug statements right next to where it already has some debug statements… Lo and behold they output! Ok so clearly I am missing something. I put debug statements before the block where I am already outputing debug statements from: no luck. I put them after. No luck. I change the code so that all the filenames get “frew” added to the end to see if CVJMX will throw an error or even change the output messages. STILL NO CHANGES.

I should point out that I have checked that I am using my local server and editing local files numerous times by the way.

Ok, so I am clearly insane at this point, as that’s more likely than file changes being scoped to a 3 line block. On a whim I decide to restart my computer. Start the Catalyst Dev server with the same command (from history) as before, without file changes. Everything worked.

All I have to say is: jfkasl;fkdasfkojqwklmdcszkljcvsxlkv m,w;ejriopjewiojc4weojejoifevjoirjivoi

The project I am working on at work is going to be deployed soon, so today I worked on some of those things that need to be taken care of before the deploy. One of those things was changing our gigantic list of javascript files into a single file with minimal hassle. I actually tried to implement it myself, but that was silly. A simple search on CPAN for catalyst javascript yields two promising results.

The first is Catalyst::View::JavaScript. While this is an excellent package which does caching and automatically minifies depending on whether the server is in debug mode or not, it expects javascript to be in memory already. Maybe that’s something that’s common and I don’t know about it, but we almost never write javascript on the server.

So onto the next result: Catalyst::View::JavaScript. It does not do caching, and it always minifies. But heck! That’s not too bad. Before installing it, note my RT that it does not have correct dependencies. Hopefully that will be fixed soon. I also sent another RT. Since I technically write more lines of javascript than perl, I need to be able to turn off minification. So I patched the module to only minify when the server is not in debug mode. Hopefully that will be accepted as well.

So after doing that I ensured that apache had gzip turned on to reduce the amount of bandwidth required by the clients to download our javascript. Then I got curious. How much of a difference do these things make? So I measured it:

Wow! Minification really doesn’t help that much, whereas gzip’ing is huge.

Now, just to be clear, this isn’t a great solution for an externally facing site, because the clients don’t currently cache the minified javascript, because it gets served by Catalyst and it’s probably not worth my time to set that up all the headers to fix that for a site that runs mostly on a LAN. Once the customer does start allowing external access we’ll want to set some last-modified headers and whatnot.

I also did a few more cool tweaks using debug mode. For example I have the debug version use ext-all-debug.js, which has extra stuff for error messages and whatnot, and ext-all.js, which is preminified and has error checking removed for performance reasons (for the tests above I used ext-all.js the whole time.)

Furthermore I have the debug switch my server side error messages from actual exceptions to a simple message asking the user to get in touch with the devs and tell us what happened and what time it was when it happened.

All in all I felt really good about the code I wrote today. It will help me a lot as time goes on and it makes the dev server way faster (1 small download is way faster that 60~ really small ones). Too bad the customer will probably not notice a significant amount of it Anyway, hopefully these tips will help give you some ideas for your webapps.

I’m probably preaching to the choir here, but it must be said: code reuse is most excellent!

Today I got a somewhat complex feature working for our customer, and almost all of it was features I’d already written, and due to the organization of our system I could easily reuse most of the code.

Our customer fixes airplane parts. When they fix a part they need to document every single thing they did to the part. We have each operation (more or less) that can be done already defined so that they can at least save those keystrokes (they are actually operation templates.) But there are a lot of operations and a typical work order will be around 50 operations, so choosing the same operations over and over is a waste of time. So we have a feature that lets them look at other work orders that were fixing the same type of part and copy operations (and materials) from that.

It was really easy to use the existing view for operations and materials because the front end is entirely comprised of JS classes. I even used an instance of the work scope grid to list all of the work scopes that are for the given part. The nice thing was that so far I’ve written no new server side code yet. And for the classes I didn’t use inheritance; I used a role style object modification by doing what Moose people would see as an after method on new (called a plugin in ExtJS). With the plugins I could simply change the store to ask for a given part-type’s work orders, hide extra columns, and add listeners to update the operations and materials when a user clicked a row.

Don’t think this is all just JS praise; Perl and Catalyst were help too. But really the benefit here was the use of any web framework in general. Because I’m using a framework I can easily find server side actions that do what I need (which the grids were already tied to in their base classes, but still.) In our other projects I’d be hard pressed to give you a list of all of our “actions,” whereas with CGIApp I can easily make a list of runmodes myself, and with Catalyst the server will make a list for me.

Excellent!

Blah blah blah perl marketing navel gazing wasting time blah blah blah perl is alive blah blah blah.

Ok, now that we’re done wasting time, here’s how to do something that (hopefully) will be useful!

I am working on a small Web Application in my increasingly rare spare time, and I decided I’d like to use OpenID for the authentication. Because of the structure of Catalyst applications this isn’t exactly easy as pie, but if you read this post it will be for you!

First off you have to install Catalyst::Authentication::Credential::OpenID (and dependencies) but wait! there are some issues you have to deal with first!

Catalyst::Authentication::Credential::OpenID depends on LWPx::ParanoidAgent, which has a few issues. See this RT for a patch that will solve it. When you run cpan (perl -MCPAN -eshell) run the command:

Wherever the directory is, save the yaml from the RT page to that directory.

Then install Catalyst::Authentication::Credential::OpenID the usual way and everything should work nicely. After the install is done you’ll need to configure Catalyst to use the module. Here is my config:

Note that I also have a realm for DBIx::Class. This is because I need to store actual data about the user and not just the fact that they have logged in with OpenID. I currently have my database set up such that a user can have more than one OpenID. I don’t have a UI for this, but I hope to add one eventually. Here are my DBIC Models:

That should be fairly obvious what it does, especially if you know DBIC already. Normally I leave my DBIC models pretty bare, but in this project I am generating the DB from the model, so I should make it as complete as possible.

Not really a lot going on there either.

This method, complete_openid_login, is really the flesh and blood of this login system. We call it after a person logs in with OpenID (see next section). The main thing to point out is that authenticate method call. It will set the $c->user object up to hold the correct User object, which is a good thing. A secondary thing to point out is that I am using perl’s vomitous exception handling for flow control. It’s just as valid to use regular if-else stuff, but I like the way this works conceptually. Hopefully as time goes on the string error messages can be replaced with actual Error objects. More on that in the coming months.

Most of the above is fairly unremarkable. We’ve got the login method which hopefully needs no explanation, and we’ve got the create method, which is what creates the DBIC model of the user. The vast majority of it is error handling. A lot of that could be taken care of with something like FormFu, but I haven’t crossed that bridge yet, so I can’t bring you across it either.

The one thing I should probably mention from the above is the captcha_check bit. I use ReCaptcha to ensure that my users are humans (for now I will discriminate against the machines!) It’s pretty easy to set up and it works quite nicely. I may post on that at some point too.

And lastly, here is what I came up with for the OpenID login screen. I used this Simple OpenID Selector to help with the logging in for the user. I’ve modeled mine off of what was used on StackOverflow.

That’s pretty basic. Nothing very special except for the research of where to get the js to do what I did.

And that’s it! You don’t have to email users to activate their account, you don’t have to worry about storing passwords securely, and they don’t have to remember another pair of credentials to use your site!

So some of you may have heard about RESTful interfaces. What I am going to describe here is my vague interpretation of REST through a web developers glasses, with respect to Catalyst and ExtJS. But first some background.

I am working on a relatively new project at work (6 months as opposed to 10+ years) and I’ve been striving to use the best tools for the job through and through. I was initially going to try for Rails, but fortunately my boss curbed that interest by saying no, (subtly!) So instead of using Rails I looked at Catalyst briefly and I already knew CGIApp from years before. Catalyst seemed like a no-go as there had previously been issues with it and windows at our work, so CGIApp seemed like the best route to take. It was still light years better than a giant bag of non-persistent CGI scripts with extremely little code reuse.

Next up is the front-end design. Say what you will about requiring users to use Javascript; if it’s an internal website, I see no reason not to do a pure Javascript site as long as you can at least try to pull off 508/ARIA. And if a website is pure Javascript, with just enough html for a single div and a header, it is probably using ExtJS. There may be other ways to pull it off, but Ext really is pretty nice to use. I very rarely have browser compatibility issues with Ext, whereas with prototype it was almost bare metal. So with Ext, instead of previously, where we had a giant bag of standalone scripts, we have been doing an inheritance based (and prototypical, duh) OO interface. It’s been excellent for UI design and code reuse. I think that some of the things we have done with Ext rival UI design I could do (as a mere programmer who’s read a couple books about UI design) with any other framework, and that includes flash and non-web GUI’s. It’s very elegant due to Javascript’s design (events, OO out of the box, etc).

Now that we have theory out of the way, lets talk more concrete issues. For Ext you have a Grid class. You can think of a Grid as basically somewhere between a table and a spreadsheet, plus lots of other sexy goodies. To set up an Ext Grid you need a store and a column model. Really the meat of the Grid is in the column model, so I made a subclass of the grid that generates the store automatically based on an action and a controller. This is really helpful because basically all of your dispatching is done in one place. That means if I change my server side dispatching I only have to change one thing in the javascript (and I have done this.)

I also add a bunch of CRUD-y buttons to the top toolbar of the grid. And of course, because they are generated automatically I generate the urls for Creating, Reading, Updating, and Deleting these records automatically. I define controller and action (currently this compiles to /controller/action, it used to be /Controller_controller/action…whatever) and I define server side actions for create_action, single_action, update_action, and delete_action. It works fairly well, except when I’m dumb and call action something like single_report. Then I end up with reasonable things like, delete_single_report…and silly things like single_single_report. Yeah that happened.

Recently I was working on a really basic Grid. The user won’t even have windows for the rows as usual, and will instead edit the Grid directly. One of my coworkers has been working on something like this lately anyway, so I asked him how he did it and went at it. It involved a REST based Ext store, a per-row editor for the grid, and the server has to support REST somehow. So far so good.

I implemented my Ext class by copying my original Ext subclass. I didn’t subclass it because this is slated to replace it. I added a temporary dispatching feature to it so that instead of dispatching to /controller/action it will dispatch to /controllerREST/action. That way I will slowly migrate things from my ghetto dispatching to REST and then when I want to switch everything at once I just move the controllers and change the dispatching in my one subclass. Easy peasy.

For the Catalyst side (and I specifically waited till I finished the Catalyst switch for this) we use Catalyst::Controller::REST. At first this is a small hassle to set it. It’s (seemingly) an all or nothing setup, so instead of making single actions RESTful (possible but not as nice) you just make RESTful controllers. The Action class takes care of the basics of RESTful dispatching. That is, PUT’s get routed to /controller/action_PUT. The controller actually does some really sexy stuff. It will look at the headers sent from the browser (Accept) to see what format the message sent across the wire was, automatically decode it, and put it in $c->request->data. You do whatever you want based on that and then you put data in (manually or using a helper function) $c->stash->{rest}, and it again uses the Accept header to decide how to format the data.

Now you may have missed it, but this makes it stupid easy to have your server support JSON, YAML, XML, and a bunch of other serializers baked in for free! Well, as long as you can set the Accept header. (I set it with a global override in Ext.) But more importantly for me it makes it quite clear what my interface is, and a simple way of automating the usage of that interface. Instead of adding single or update to some url, I just do a PUT for creation, or a DELETE for deletion. It’s simplified our server side and client side code significantly!

Also note: using REST you can simplify a lot of other things, like instead of everything being a text blob, you can have integers, booleans, and other complex data structures. This means that I can have non sucky checkboxes. This is a big win.

Now that I’ve finished writing and editing this blog post it is a whopping 1059 words long, so I should probably stop there. But I may blog more about this REST stuff in the future. What about you? Do you use a RESTful interface? Do you hate REST? Why?

Sorry about that guys, I didn’t use links to make it clear which book I was talking about. Usually I do that kind of stuff but the internet was sucky (fixed!) so it hurt to look up links. Enjoy?

I am just getting through chapter four of the Catalyst book and there are already a whole lot of things worth mentioning. My internet is currently at 50% packet loss because our wifi router is busted so this is pretty painful for me. So we’ll keep it short.

Moose

The book has a nice (very short) introduction to Moose. Not only is this good because Catalyst is now based on Moose, but also I would say you probably want your OO code to be based on Moose. There are times when you probably don’t want to use Moose, but there are also times when you don’t want to use strict. As a rule, use Moose for OO code.

CPAN

There is also a very good introduction to usage of CPAN. A lot of us think that CPAN is our programming platform. Knowing how to use it is extremely important. It includes not just finding stuff on CPAN, but also ascertaining the quality of those modules, and how to install them. Very good information for a perl programmer.

Tesing Methodology

In chapter 4 mst discusses how he writes tests (which can be slightly supplemented with his latest blog post) and it’s actually quite helpful. Some people write tests after writing their code and run the risk of forgetting to test at all (that’s me!) Other people are all hardcore TDD and write tests first, but that assumes that they have already thought through the interface for what they are writing. mst posits that it’s better to write your code, and “test” it from test files as you go. And test in this case means warns, Data::Dumps, whatever. After it works how you expect, you then take those warns and whatnot and translate them into ok’s, is’s, and cmp_deeply’s. It’s really much nicer than the alternative: build it all and see if it works. Try it!

Diffs

Lastly, I really like how they represent code as diffs instead of monolithic code. Writing large swaths of code doesn’t work that well in real life. It works much better to do tiny changes and make sure they still compile, do what you want, etc.

But the book certainly isn’t perfect! There are some weird code layout issues, (p34, 36, 39, etc) and I am pretty sure I saw at least one syntax error (END__ instead of __END__).

So far though, I would say that the book is better than most programming books. Really, a lot of programming books need to be more like this, instead of focusing entirely on the arcana of one framework they should help you be a better programmer overall.

So this week, as previously alluded to, I convinced my boss to let me switch my current app from CGI::Application to Catalyst. I had gotten the book in the mail and I showed it to him to make the point that it’s a serious framework. Fortunately the switch has been mostly painless. The first reason being that our controller is pretty bare right now aside from validation, which took about a day to get entirely ironed out.

The interesting thing for me, most of all, is that I have gotten pretty good at writing regular expressions with vim to search and replace for CGIApp-isms to replace with Cat-isms.

Here are a list of some of the big ones:

Simple replacements:

More complex stuff

If you know anything about regular expressions you know that the \1 means the first back reference. Now, vim’s regex flavor is a little strange because it is optimized for searching for plain text, so *most* characters default to literals. That’s why I have to escape the parentheses to make a matching group. Also note the following unusual construct: .\{-} . That’s the same as .*? in Perl. That’s actually surprisingly important.

Anyway, this switch has been fairly fun and exciting. The best part being the inimitable structure of a Catalyst application. For example, the fact that we have a dev server with lots of affordances for (duh) developers other than little setup is great, and built in config file reading is something that I have always wanted. We always ended up rolling our own solution in other projects, but this is really supreme since it’s in one place and not just Perl code.

An there are lots of pleasant things like how it’s really easy for our app to have both JSON and TT support. This will be really good later on when we start to do pdf printouts and whatnot. Instead of adding methods for those things into the controller, like in CGIApp, we will just add another View module.

The main thing that has weirded my out so far is that in CGIApp the App is the controller. In Catalyst you have an App, which also seems to be an instance variable, with accessors for CGI parameters and whatnot, and you also have Controllers. Anyway, I need to wrap my head around all that. Hopefully reading through the book will help with some of these issues.

How about you? Are you still happy with CGIApp? Are you adventurous enough to use Reaction?