Woohoo! git!

I am so happy to announce that DBIx::Class has migrated to git!

If people latch on well, this should benefit is in a number of ways. The first thing is that most people should appreciate is the ability to check in to source control without needing to commit to the remote repository. Not only does this make things way faster, it also means that you can work sanely offline. A lot of people did this before with SVK, but SVK is slow and a hassle (I think) to install and setup.

Another thing that this should help is our history. I don’t mean to point out a specific person, I did this all the time with svn (it’s just the nature of the beast,) but take a look at some of these commit messages. The SVK merge messages are pretty obnoxious, but that’s not what I’m referring to. The ones that bother me are the: “Oops,” “Typo,” “Thinko,” etc. With git those can be cleanly squashed into another commit. In fact, I would recommend cleaning up your history before you push every time. This is how I do that:

That will rebase your current branch (all of it) onto master, and give you an editor that will let you fix commit messages, merge commits, and more. If people do not do this I fully intend to do it myself before I merge a branch in, if I do. And that brings me to the next point which we shall discuss.

Workflow

We toyed with the idea of setting up git such that everyone would just use github or gitorious or whatever and ribasushi and I would be in charge of merging remote branches into the DBIC master, but mst veto’d the idea in favor of a more communal approach. Basically the workflow we will use looks like the following:

1. Get a commitbit (liberal policy)
2. Clone the repo (dbsrgits@git.shadowcat.co.uk:DBIx-Class.git)
3. Make a topic branch based on what you are doing, do work on it, whatever
4. Push branch to repo
5. Ask a fellow DBIC’er to review the branch, that includes me, ribasushi, Caelum, and really anyone else with a commitbit
6. If the reviewer thinks everything is sane, the reviewer will merge the branch in; to be a bit more clear, you should not merge your own branch into master, that is the reviewer’s job
7. Delete your branch locally and remotely after everything is merged in, and work on other stuff!

A note about merges, don’t merge master into your branch. It makes for yucky history. Instead, rebase your branch onto master. Currently the thought is that anything except for master can be rebased. Of course if you and another dev are working in the branch you might want to keep that to a minimum, but at the very least you should be rebasing to squash silly commits before you push, and then when you do the final merge into master you should rebase first, so that history remains sane. Of course, the best time to rebase to fix silly history is before you push, and the best time to rebase to make it so that you are fastforwarding master is right before the final merge, so try to only do that then.

At some point I plan on writing a DBIx::Class::Manual::Contributing in the spirit of Moose::Manual::Contributing, but ours will be significantly more lax. In the meantime, just swing by #dbix-class, get your commitbit, and help out!

This is almost entirely so that I remember how to do this. A big thanks for arcanez for showing me this in the first place.

The Problem

In the Perl community, numerous important git repositories are hosted at shadowcat, but of course if you went to that url you would not be able to see all the work that I have spent on each of those projects. I like the fact that github has a nice concise view of my work.

The Solution

The following is an example of a section from my .git/config in my DBIx-Class-DeploymentHandler:

This lets you push to origin, github, or all. I tend to only pull from origin and push to all. Hopefully this can at least be a reference for people

I am writing this post to address a problem that I could see appearing in our community. If it offends you feel free to let me know. If you comment on my blog as a troll, I will delete your comments. Feel free to put them on your blog where they reflect on yourself

Recently a certain member of the community has posted a few blog posts that boil down to “Open Source developers should support their open source work as if it were a job.” I will not make a link because I would rather people not read his posts. Normally I feel like the best plan of action for people who make silly statements is to just ignore them, but a good friend of mine pointed out that it would be bad if people got confused and actually bought into this line of thought.

So in foolish manifesto style, I will write the rest of this post with how I think things should be done, not who got what wrong.

Patches Welcome

As a developer of CPAN modules and more tentatively a leader of a small part of CPAN, I will continue to say “Patches welcome.” If people are using the software that I have written for CPAN they are developers and can help implement the code that they need for their given use case. I doubt a non-developer would be able to figure out how to install a CPAN module, let alone use it.

Patches are not just for code though! That is where novice developers come in. In the DBIx::Class world (and I imagine Catalyst and Moose) standard practice is to help a newbie on IRC and then in return ask them for a documentation patch that will have rendered the help session needless. Of course sometimes the documentation needs massaging or you need to guide the newbie as to where the documentation should go, but that’s fine! This is how you get new blood into your contributor pool.

And of course the beauty of the “patches welcome” statement is that by definition only people who actually want the feature will send you patches! I would never ask a random user of my module to implement a feature that they do not need. It is because I needed correct sorting in SQL Server that I got involved with DBIx::Class. I did not ask: “How can I make this happen?” I actually asked someone else to fix it for me, and instead this person (ribasushi) guided me on how to do it myself. To restate the point: patches should be written by the people who will use them.

Meritocracy

One of the beauties of the open source world is that ultimately things are (to an extent) a meritocracy. That means that the people who get stuff done have the most say. As a Perl developer I am much more likely to listen to RJBS or Miyagawa about how to do something or even defer to them over some random developer I have never heard of. Of course there are times when things are done democratically. Recently some of us DBIx::Class developers had a vote about something. mst has even stated to us that when it comes to DBIx::Class he is willing to defer to the other developers if everyone else agrees. But ultimately, because writing software takes skill, the people with that skill make the decisions. Not the people who are the prettiest or richest or have the best marketting. I personally like it that way and will do what I can to keep it that way.

I say this because really, open source programming ultimately is not like regular work. Sure, developers fall off of the face of the planet and stop maintaining their software. But if you need the software that they wrote, you have at least two options; the first is that you contact the author and ask if you can take over the project so that you can get it back to a point where you can use it. If that sounds like too much work you clearly do not really need it. Your second option, if the author does not give you the OK to take over the project, is to fork it. Generally this should be avoided, since it can breed bad blood and often your software and the authors can have incompatibilities, but it is an option nonetheless.

I have done both of the above and it was never a huge problem. So again, in Open Source software, the onus is on the user not the developer; but that is why I use it, personally. If a developer dies, I as the user will be ok because I can fork the project myself.

Social norms

As I already said, the main reason for this post is so that developers inside and outside of the community will not be misled. I do not want new developers using my software expecting that they can just make demands and that I will do what they ask. I am not a hostage and neither is my software. I do not work for free. Anyone who expects me to work for free is either lying to themselves, insane, or is a looter stealing from me. I do not respect the opinions of thieves, liars, or lunatics when it comes to responsibility and I do not plan to respect their opinions.

I strongly believe the above, and I hope that we can all have some healthy discussion about it and hopefully come to agreement. I also have some thoughts on how to help increase the communication between various dissenters and those of us who like things the way they are, but I will leave those for another day.

Update

Turns out that this turned into a meme, so I’ll try to collect all of the related links to this topic here. I’m sure that I left some out, but I tried to include all of the ones that popped up in my feed reader. (Note: I’m not subscribed to all of Iron Man, so I might have missed some)

• chromatic’s thoughts
• dagolden’s thoughts
• perigrin’s thoughts
• jjnapiorkowski’s thoughts
• nperez’ thoughts

I’m proud to announce a new version of DBIx::Class::Journal after almost three years of different people working on different parts!

It’s certainly not complete. The main issues for me are:

1. It only versions tables with single column PK’s
2. It has no simple way to have related data in the journal

The former is a SMOP, the latter, on the other hand, is a very serious architectural issue which I don’t think can even safely be solved. It *might* be as simple as just replicating all of the relationships in the original result and then adding in another column to the relationship which points to a version of that result. Or it might not. I need to consider it and look at things, but I think it can be done.

Honestly, if I had all the time in the world, I’d rewrite ::Journal from the ground up and make it a lot more malleable. Unfortunately I don’t have all the time in the world and personally I don’t have much use for it. Parts of my job do, but they only pay so much for features

Anyway, enjoy it! It should be pretty solid. Just make sure you read the limitations before use.

Yesterday I was reading this post by chromatic and I finally understood what state does. If you look at the perldoc for state you will see why. There is quite a dearth of examples there.

Anyway, here’s a real world example from our code base which uses state in a slightly different way from what is probably typical.

Before:

After:

I agree with chromatic on this one; it’s not lifechangingly better, but given enough usage I think it could make things much more clear.

Ok, so I just had to refer to this unposted post since I upgraded to perl 5.12 and I figured I’d finally post it.

Here’s everything I did to get ODBC working and connected to our MSSQL server at work:

Note:
driver=FreeTDS refers to /etc/odbcinst.ini
this is how it finds the .so

And this is our DSN:

“dsn”:”dbi:ODBC:server=10.6.0.9;database=ACDRI;port=1433;driver=FreeTDS;tds_version=8.0″,

Hope this helps someone!

update: For some reason I had to replace unixodbc-dev with libiodbc2-dev, so you may need to do that as well.

Yesterday Ovid posted this little snippet to get his top 10 used commands.

I had to modify it a little for my zsh settings:

I’m sure that my home computer would have the git and svn switched. I’ll update this post with that computer’s history if I remember.

update Here’s my home computer:

DBIx::Class::DeploymentHandler is nearly ready for prime time, so I’m going to discuss a pattern mst described to me that I’ve found very helpful in developing this project.

Roles

If you don’t already know what roles are you probably don’t read very many perl blogs etc. chromatic has written a series of blog posts where he discusses the various merits of roles vs whatever your poison is. Maybe read that. This isn’t really about that. What this is about though is that roles aren’t always the answer.

Delegation

One of the assumptions of roles is that all of the methods in a role share their namespace. So if you compose Role1 and Role2 and they both implement a sleep method you will get an error at compile time saying that the method collides. This is a Good Thing and helps us not shoot ourselves in the foot. When it’s a problem is with private methods that the end user typically shouldn’t be calling, but happen to collide. As far as I know there is no way to have a role that partially composes with a class. I’m pretty sure that’s against the whole spirit of a role.

So instead of using a role to compose in the interface for whatever it is you are doing you can instead use delegation, where object A has-a different object B and uses the public interface of object B. That way private methods of B stay that way and don’t collide. The problem is that this can make code a lot more verbose. So instead of

one must do:

That make things a lot more verbose, it gives away the inner workings of $dh, and most importantly it makes overriding parts of $dh harder.

Roles with Delegation

So basically the pattern goes like this:

The Public Interface:

The Delegate:

The Role:

Usage:

And to use that you’d do:

Of course that’s totally contrived, but it gets the general pattern across. If you want to see examples in action check out some of the roles from DBIx::Class::DeploymentHandler!

So basically you define your public interface, which isn’t a bad idea anyway, and the “handles” key for the delegate’s attribute takes the role that defines the public interface. This automatically delegates all the methods required by the role (and probably any defined by the role too.)

If you have private methods you want to reuse make another role and compose that into the delegate’s class, but don’t put it in the handles section.

I feel like this pattern, even though it yields a lot of boilerplate, helps to make very clean interfaces. Because of this pattern I’ve made well decomposed classes and testing them is dead easy. I imagine that to test roles normally you make stub classes using the roles. Here I just test the actual delegates alone and then I have an integration test that ensures the class that uses all the roles does the right thing.

Hopefully you’ll find this as useful as I have.

In February of 2008 I figured out how to switch our servers from IIS to Apache. The main reason I did that was because if you print to STDERR in Perl while running under IIS the server would crash hard. In general it just took some research and motivation. All was well with the world…. For six months.

After switching to Apache we needed a way (previously accomplished with PerlEx from ActiveState) to run certain scripts persistently. I did some research and discovered that using mod_perl in win32 was feasible and you can indeed turn it on for parts of your site. Yet again, all was well with the world.

Unfortunately as time passed and we started using a deeper stack of Perl (we originally were just using DBI, the Perl database layer, and sometimes using Template::Toolkit, one of the most major Perl templating systems,) we started seeing Apache crashing or leaking memory. Unlike IIS crashes the cause and time till crash was unpredictable, but after some work the issue was found and fixed.

During this time I and all but one of my coworkers switched to the most excellent Strawberry Perl, the Windows Perl one might say. Logging in to the servers to install packages with PPM quickly soured for me and I spent probably 3 days worth of my time (half of which was unpaid!) trying to find a way to use Strawberry persistently. If we could use Strawberry on the servers installing dependencies would boil down to

But I couldn’t seem to get mod_perl to build, mod_fcgid and mod_fastcgi were both unworkable (for me anyway,) and I couldn’t get lighttpd + FastCGI to work. So I gave up on that endeavor.

Nearly four months have passed since the crashing Apache issue was originally solved and just days ago we deployed our first Catalyst project (we have two more in the pipeline now!) We deployed onto mod_perl and Apache on Windows. I would never recommend deploying onto Windows, but I also realize that there are business reasons to do so and sometimes it’s just what you have to do.

And all was well with the world…for two page requests. It turns out that somewhere in our stack of Perl, Apache, mod_perl, and Windows there was an issue that made the server consistently crash after nearly every other request. I did some research, and even built up a replica of our deploy on my machine (linux) to see if the issue was generic mod_perl. If it were a problem in Linux it would be much easier to get free help from the community, but alas, it ran perfectly on my machine.

While I was driving to a friend’s recently I had a thought; why not just use the Catalyst development server or some other Perl based server and just proxy to it with Apache? Heck, it’s actually very similar to one of the recommended ways to deploy Catalyst in the Catalyst book.

Before I get into the details of this I need to point out that we are not using HTTP::Prefork, thanks to Windows. If you have a large site you really should not use Windows, for numerous reasons. That was the conclusion that my boss and I came to anyway.

First off, here is the Apache configuration we ended up with:

So basically all we do in this configuration is have Apache serve the static files and then proxy the requests to a couple of catalyst dev servers. I used Srvany.exe and a couple of .bat files to start the catalyst dev servers. It works much better than using mod_perl, and each server sits at about 90M a piece. If we ended up getting a huge site and for some strange reason needed to keep our outfacing server windows, we could actually serve the catalyst parts on a linux server and have apache proxy to those, so it scales very nicely.

Anyway, here’s to the next 6 months of serving!

I really should have posted this sooner. Certainly before I began my next project. Oh well.

I am proud to announce the next bit of mstware! Log::Contextual is a small module for making your life easier when it comes to logging. Instead of bringing yet another logging infrastructure into the mix (see Log::Log4perl and Log::Dispatch), this module is a thin wrapper around any logging system you choose to use. (Note: we are working with authors of major logging packages to work seamlessly with L::C, but at the time of writing most need some form of adapter.)

There are a few major features worth noting. First off, ridiculously convenient interface. Once you’ve set up your logger (presumably in the startup of your app or whatever) all your logging code will look like the following:

Another great thing is that, like Devel::Dwarn, all of the logging functions are identity functions; that is, they return their arguments. That means you can do cool things like the following:

Of course, in Perl you may be passing around complex references and the above will get cumbersome fast, so we added shortcuts specifically for logging out data structures:

The automatic stringification is done with Data::Dumper::Concise, so you will get reasonably indented output for free.

In a separate package we are going to provide a module to basically turn the logging functions into no-ops at compile time, thus giving you the ability to have your code run just as fast if it never had the logging functions in in the first place. I’ll post more on that once it’s released.

So what are you waiting for? Go log stuff!