The project I am working on right now is rewriting a large, mostly CRUD application. The current app (second generation) is all VB6 and Stored Procedures. We are making the app entirely web based with DBIx::Class for the brunt of the backend and ExtJS for the UI. There are a few other technologies involved, but they should remain fairly light and unobtrusive.

As we’ve designed our code I’ve made an effort to only look at the inputs and outputs of the original code, to avoid using any existing design mistakes that have already been made. Generally this methodology works well. But sometimes the very format of the input/output leads me astray. Here’s an example that I encountered today.

Our customer typically uses composite primary keys to allow for public facing id’s. That makes sense. Typically serial numbers follow actual reason and composite primary keys work for that use case. In some places these keys are also the natural ordering for a set of items. For example, the company will have a list of operations that were performed to fix a part. Those operations are listed in order (for obvious reasons) and that order must be maintained. The id makes sense initially. Yet sometimes people need to change the order of some of these things. The most specific part of the composite pk always starts at one and increments by one. So when the user reorders the items in the list we ddo something like this (from memory):

Works great!

But today I was considering what this would be like if I were to remove the composite pk’s from the system. How would I order the items? I would no longer change the id because they would be completely unique and reordering would be a big hassle. Solution? Real numbers! If you have 1 and 2 and you want 3 to be between them, you set it to 2.5! or if you want to displace 2 with 5 you set 2 to 1.5 (or 2.5) and just set 5 to 2. Of course you’d need some code to find the midpoint ((x+y)/2). But that’s no big deal.

Now, I’d like to think that I would have come up with that (simpler) solution originally if I hadn’t already assumed the database format that we have. Although the first form was fun to come up with it is inferior to this. Id’s should really be forever, and ordering shouldn’t change the id of a thing. Anyway, keep that in mind when you do your rewriting and reverse engineering. Think of the data as it would be if you’d made it originally, and simpler solutions may come to you out of the blue.

When you want to get help on the internet, it’s not just what you say and how you say it; it’s also where you say it. I use three different communication mediums on a day-to-day basis to get help with the various toolkits I use and only a couple of them overlap in medium. There are inherent benefits and drawbacks to each medium, but generally you don’t have a choice in which medium to use for a given project.

At $work we use the ExtJS framework for UI stuff. We use their paid forums. There are a group of ext users who use IRC but I never found the channel to be any help. So with Ext you use a forum if you want help. Forums are nice because there can be built in code formatting, all the archives are right there, you only have to see the messages when you really need them, and you an have built in categorization.

I used to use CGI::Application. The best way to get in touch with those guys is to use a mailing list. They have an IRC channel, like the ext guys, but in a week there are about as many conversations in as many days. So mailing lists! Mailing lists are kinda intimidating to get into. Partially because if you stop using a toolkit you keep getting the mail. Or when you don’t have a problem you keep getting the mail. But ultimately this is a really good long term solution, because you tend to have people who use the toolkit who, dun dun dun, check their email! A drawback of this medium is that there is no built in archival (although typically it is archived) and any source code formatting is done in plain text.

Both DBIx::Class and Catalyst primarily use IRC for support. They have mailing lists, but you can get information and simple answers much faster on IRC. The beauty of IRC is that it is realtime. So you have people having regular conversations as well as getting answers to questions. One of the major issues with IRC, in my mind, is that code must be shown separately, in a pastebin website. There also is no built in archival, or even persistence. Nonetheless, this is very common in largish open source projects.

And there you have it! fREW’s useless list of mediums that go with a handful of projects!

I’ve been using Catalyst at home for nearly six weeks now and I guess two weeks at work. I feel that now is a good time for me to list some of my impressions.

The angle that I am coming from is mostly CGI::Application, which means very bare bones.

The first thing that I got for free with Catalyst was configuration file support. Less than a week after switching to Catalyst our customer asked me if we could change the database connection easily. Previously I had that set in an environment variable in the Apache config. Although that’s still something my customer could easily do, it’s certainly not as easy as other forms of configuration.

Chained Actions are an interesting way to allow code reuse in Catalyst controllers. Basically they allows one to automatically call a number of methods in a given order automatically based on the path. One would typically set various per-request values (stash) in the chained methods. The following is an example.

So basically how that works is that /groups/1 will call load_req and then view whereas /groups/1/add will call load_req and then add. You can see that because load_req starts the chain (‘/’), names itself groups (PathPart), and then says it takes a single argument (CaptureArg). But it cannot be called directly, because it’s not an endpoint. Any argument with CaptureArgs is not an endpoint. This isn’t a tutorial, so I won’t try to explain it in depth. Either way, it can significantly increase code reuse.

Flexibility is another great thing about Catalyst. Catalyst::Action::REST allows one to easily use Catalyst in a RESTful manner. It automatically handles serialization, deserialization, and dispatching. Here’s the cool thing; when there is an error in my app at work it needs to return a 500 as well as valid JSON. A REST controller will return an html error if there are errors. I don’t want to override the end method, because it’s what does the deserialization. Moose to the rescue! Basically what I do is define a before method that will put data in the stash if there are errors. Then the real end method runs and outputs the errors as JSON. Again, this isn’t a Moose tutorial, so I won’t show all the details. Maybe another post.

Of course because of the increase complexity Catalyst is harder to learn, but I haven’t actually found it that much harder than CGIApp. It’s nice and regular, so once you know where to look in the doc for what you need it’s not that bad. Although I guess that’s the same as most large projects

So I have some cool posts enqueue but they are not done and longish, so I figured I’d post about this bug in the interaction between Perl 5.10′s switch statement and List::Util‘s first method. Here is a test script:

Here’s the output:

So keep this in mind! This almost drove me crazy recently when I was working on some code at work. I had replaced a grep with a first as a performance optimization, but the code stopped working. Fortunately, Graham Barr had mentioned this issue at one of the Perl 6 meetings, and I remembered it when I found that first was at fault.

So some of you may heard that PerlMonks got hacked recently.

Before I get into my (not entirely unique) solution, I want to express how upset I am at PerlMonks about this. I am not going to blame them for getting hacked. But storing passwords in plaintext? I would have thought better from a developer community, especially one as entrenched in web applications as the Perl community. I am dumb for using the same password in a lot of websites, but I’m upset that one of the ones I trusted (level 2 out of 3 password) violated that trust. I’m sure they will fix the issue, but I am really upset. Blah. If you have a pm account, you should change the password and any other passwords that were the same.

Initially I was going to install this vimscript and use it, but then I’d have to somehow install openssl on a usbkey. I looked on PortableApps and found KeePass. That works, but KeePass 2.0 is even better! It will work with .net 2.2 and mono 2.0. Srsly guys, I tried it!

First off, get KeePass. I’d get the zip version for your USB key and the Linux computer, and then on Windows install the executable. Make sure you get the 2.0 version.

Now, if you are using Linux you need to ensure that you have Mono 2.2 or greater, and the Windows Forms libraries. For Ubuntu that means adding a few extra items to your sources.list. See here for which sources to use. Make sure you update after adding the source. After that you should be able to install mono-2.0-gac (the version will actually be 2.4) and libmono-winforms. There is a 1.0 and 2.0 winforms, but I just installed both. After that you can just do “mono KeyPass.exe” and it should work with rainbows and unicorns!

A few config tips that would have helped me initially. You can set a default username in the Database settings, which is nice. In Options you can set it to autolock after a given period of time and set default expiry times for the passwords. If you aren’t going to trust the internet with your passwords you might as well expire them. I chose 90 days as that’s not too big of a hassle with a tool like this.

And just a really basic note on usage issues: when you add a new password to database it will generate a new password for you. I changed the generator to use symbols in addition to the default alphanumeric set. But this is really nice when you decide to just not know any of your passwords like I am doing. Also, the default behavior will allow you to double click passwords in the db, copy them to your paste buffer, and then clear the buffer after 10 seconds. I find this to be really convenient!

Anyway, hopefully this will ease the transition burden for someone else too. Enjoy!

Blah blah blah perl marketing navel gazing wasting time blah blah blah perl is alive blah blah blah.

Ok, now that we’re done wasting time, here’s how to do something that (hopefully) will be useful!

I am working on a small Web Application in my increasingly rare spare time, and I decided I’d like to use OpenID for the authentication. Because of the structure of Catalyst applications this isn’t exactly easy as pie, but if you read this post it will be for you!

First off you have to install Catalyst::Authentication::Credential::OpenID (and dependencies) but wait! there are some issues you have to deal with first!

Catalyst::Authentication::Credential::OpenID depends on LWPx::ParanoidAgent, which has a few issues. See this RT for a patch that will solve it. When you run cpan (perl -MCPAN -eshell) run the command:

Wherever the directory is, save the yaml from the RT page to that directory.

Then install Catalyst::Authentication::Credential::OpenID the usual way and everything should work nicely. After the install is done you’ll need to configure Catalyst to use the module. Here is my config:

Note that I also have a realm for DBIx::Class. This is because I need to store actual data about the user and not just the fact that they have logged in with OpenID. I currently have my database set up such that a user can have more than one OpenID. I don’t have a UI for this, but I hope to add one eventually. Here are my DBIC Models:

That should be fairly obvious what it does, especially if you know DBIC already. Normally I leave my DBIC models pretty bare, but in this project I am generating the DB from the model, so I should make it as complete as possible.

Not really a lot going on there either.

This method, complete_openid_login, is really the flesh and blood of this login system. We call it after a person logs in with OpenID (see next section). The main thing to point out is that authenticate method call. It will set the $c->user object up to hold the correct User object, which is a good thing. A secondary thing to point out is that I am using perl’s vomitous exception handling for flow control. It’s just as valid to use regular if-else stuff, but I like the way this works conceptually. Hopefully as time goes on the string error messages can be replaced with actual Error objects. More on that in the coming months.

Most of the above is fairly unremarkable. We’ve got the login method which hopefully needs no explanation, and we’ve got the create method, which is what creates the DBIC model of the user. The vast majority of it is error handling. A lot of that could be taken care of with something like FormFu, but I haven’t crossed that bridge yet, so I can’t bring you across it either.

The one thing I should probably mention from the above is the captcha_check bit. I use ReCaptcha to ensure that my users are humans (for now I will discriminate against the machines!) It’s pretty easy to set up and it works quite nicely. I may post on that at some point too.

And lastly, here is what I came up with for the OpenID login screen. I used this Simple OpenID Selector to help with the logging in for the user. I’ve modeled mine off of what was used on StackOverflow.

That’s pretty basic. Nothing very special except for the research of where to get the js to do what I did.

And that’s it! You don’t have to email users to activate their account, you don’t have to worry about storing passwords securely, and they don’t have to remember another pair of credentials to use your site!

I saw CKY this past Friday and it was a most excellent concert.

I saw two of their three openers. The first one was vanilla boring metal. Only worth mentioning for completion.

The second band was Graveyard. They were very interesting. I think they are classified as Blues Metal. It sounds ridiculous, but the music was alright! They were reminiscent of a Metal version of the Allman Brothers. Certainly worth checking out.

Then cKy, the headliner, started playing. They were great performers. They didn’t have any kind of setlist and almost entirely played requests, which was pretty cool.

One thing that was interesting about them is that they refuse to allow silence in their concerts, so if they aren’t playing, the audience needs to be cheering. When silence did occur they would play really cheesy eighties music as punishment. That was pretty hilarious. They even came out to Billy Jean, which was pretty awesome.

There was one point at which they asked for songs to play and after a few minutes of people calling out songs, the guitarist said they could only play half of the requests. So they started playing one of the songs and at some point in the song the guitarist said, “Wait wait wait! I said half!” so then they switched songs. They did this a few songs and ended the medley with Michael Jackson’s Beat It. That was pretty excellent.

The last thing worth mentioning was that the parents Margera were there and April Margera came up to sing along with the last song. That was pretty cool.

Anyway, if you are local and want to come to a concert (J-Dot) here is a link to my concert calendar.

So some of you may have heard about RESTful interfaces. What I am going to describe here is my vague interpretation of REST through a web developers glasses, with respect to Catalyst and ExtJS. But first some background.

I am working on a relatively new project at work (6 months as opposed to 10+ years) and I’ve been striving to use the best tools for the job through and through. I was initially going to try for Rails, but fortunately my boss curbed that interest by saying no, (subtly!) So instead of using Rails I looked at Catalyst briefly and I already knew CGIApp from years before. Catalyst seemed like a no-go as there had previously been issues with it and windows at our work, so CGIApp seemed like the best route to take. It was still light years better than a giant bag of non-persistent CGI scripts with extremely little code reuse.

Next up is the front-end design. Say what you will about requiring users to use Javascript; if it’s an internal website, I see no reason not to do a pure Javascript site as long as you can at least try to pull off 508/ARIA. And if a website is pure Javascript, with just enough html for a single div and a header, it is probably using ExtJS. There may be other ways to pull it off, but Ext really is pretty nice to use. I very rarely have browser compatibility issues with Ext, whereas with prototype it was almost bare metal. So with Ext, instead of previously, where we had a giant bag of standalone scripts, we have been doing an inheritance based (and prototypical, duh) OO interface. It’s been excellent for UI design and code reuse. I think that some of the things we have done with Ext rival UI design I could do (as a mere programmer who’s read a couple books about UI design) with any other framework, and that includes flash and non-web GUI’s. It’s very elegant due to Javascript’s design (events, OO out of the box, etc).

Now that we have theory out of the way, lets talk more concrete issues. For Ext you have a Grid class. You can think of a Grid as basically somewhere between a table and a spreadsheet, plus lots of other sexy goodies. To set up an Ext Grid you need a store and a column model. Really the meat of the Grid is in the column model, so I made a subclass of the grid that generates the store automatically based on an action and a controller. This is really helpful because basically all of your dispatching is done in one place. That means if I change my server side dispatching I only have to change one thing in the javascript (and I have done this.)

I also add a bunch of CRUD-y buttons to the top toolbar of the grid. And of course, because they are generated automatically I generate the urls for Creating, Reading, Updating, and Deleting these records automatically. I define controller and action (currently this compiles to /controller/action, it used to be /Controller_controller/action…whatever) and I define server side actions for create_action, single_action, update_action, and delete_action. It works fairly well, except when I’m dumb and call action something like single_report. Then I end up with reasonable things like, delete_single_report…and silly things like single_single_report. Yeah that happened.

Recently I was working on a really basic Grid. The user won’t even have windows for the rows as usual, and will instead edit the Grid directly. One of my coworkers has been working on something like this lately anyway, so I asked him how he did it and went at it. It involved a REST based Ext store, a per-row editor for the grid, and the server has to support REST somehow. So far so good.

I implemented my Ext class by copying my original Ext subclass. I didn’t subclass it because this is slated to replace it. I added a temporary dispatching feature to it so that instead of dispatching to /controller/action it will dispatch to /controllerREST/action. That way I will slowly migrate things from my ghetto dispatching to REST and then when I want to switch everything at once I just move the controllers and change the dispatching in my one subclass. Easy peasy.

For the Catalyst side (and I specifically waited till I finished the Catalyst switch for this) we use Catalyst::Controller::REST. At first this is a small hassle to set it. It’s (seemingly) an all or nothing setup, so instead of making single actions RESTful (possible but not as nice) you just make RESTful controllers. The Action class takes care of the basics of RESTful dispatching. That is, PUT’s get routed to /controller/action_PUT. The controller actually does some really sexy stuff. It will look at the headers sent from the browser (Accept) to see what format the message sent across the wire was, automatically decode it, and put it in $c->request->data. You do whatever you want based on that and then you put data in (manually or using a helper function) $c->stash->{rest}, and it again uses the Accept header to decide how to format the data.

Now you may have missed it, but this makes it stupid easy to have your server support JSON, YAML, XML, and a bunch of other serializers baked in for free! Well, as long as you can set the Accept header. (I set it with a global override in Ext.) But more importantly for me it makes it quite clear what my interface is, and a simple way of automating the usage of that interface. Instead of adding single or update to some url, I just do a PUT for creation, or a DELETE for deletion. It’s simplified our server side and client side code significantly!

Also note: using REST you can simplify a lot of other things, like instead of everything being a text blob, you can have integers, booleans, and other complex data structures. This means that I can have non sucky checkboxes. This is a big win.

Now that I’ve finished writing and editing this blog post it is a whopping 1059 words long, so I should probably stop there. But I may blog more about this REST stuff in the future. What about you? Do you use a RESTful interface? Do you hate REST? Why?

Sorry about that guys, I didn’t use links to make it clear which book I was talking about. Usually I do that kind of stuff but the internet was sucky (fixed!) so it hurt to look up links. Enjoy?

I am just getting through chapter four of the Catalyst book and there are already a whole lot of things worth mentioning. My internet is currently at 50% packet loss because our wifi router is busted so this is pretty painful for me. So we’ll keep it short.

Moose

The book has a nice (very short) introduction to Moose. Not only is this good because Catalyst is now based on Moose, but also I would say you probably want your OO code to be based on Moose. There are times when you probably don’t want to use Moose, but there are also times when you don’t want to use strict. As a rule, use Moose for OO code.

CPAN

There is also a very good introduction to usage of CPAN. A lot of us think that CPAN is our programming platform. Knowing how to use it is extremely important. It includes not just finding stuff on CPAN, but also ascertaining the quality of those modules, and how to install them. Very good information for a perl programmer.

Tesing Methodology

In chapter 4 mst discusses how he writes tests (which can be slightly supplemented with his latest blog post) and it’s actually quite helpful. Some people write tests after writing their code and run the risk of forgetting to test at all (that’s me!) Other people are all hardcore TDD and write tests first, but that assumes that they have already thought through the interface for what they are writing. mst posits that it’s better to write your code, and “test” it from test files as you go. And test in this case means warns, Data::Dumps, whatever. After it works how you expect, you then take those warns and whatnot and translate them into ok’s, is’s, and cmp_deeply’s. It’s really much nicer than the alternative: build it all and see if it works. Try it!

Diffs

Lastly, I really like how they represent code as diffs instead of monolithic code. Writing large swaths of code doesn’t work that well in real life. It works much better to do tiny changes and make sure they still compile, do what you want, etc.

But the book certainly isn’t perfect! There are some weird code layout issues, (p34, 36, 39, etc) and I am pretty sure I saw at least one syntax error (END__ instead of __END__).

So far though, I would say that the book is better than most programming books. Really, a lot of programming books need to be more like this, instead of focusing entirely on the arcana of one framework they should help you be a better programmer overall.