At work we have a tiny set of classes and relationships that we’ve reused for a few projects now. The idea is that it’s a package deal of users, roles, permissions, and a way to map permissions to parts of the application. I’m actually pretty fond of it, but its usage is a little awkward and not very flexible. If I could I’d put it on CPAN as that would mean tests, docs, and more importantly, a way to make it more useful for disparate projects.

The way it works right now is as follows:

for all of the seven-ish results. That’s a little obnoxious, but you need a place to add extra columns and relationships. The problem is if you decide that you want to, say, not use roles the way we decided to, you already have all these bogus relationships tying everything together. So what I’m thinking is that I’ll instead put the relationships in a more intelligent schema component with a bunch of options and whatnot. So you could do all of the above, and then in your schema do something like:

auth_setup could take a number of arguments toggling which results will get which permissions. Of course for a simple system one could just subclass the user result, which already has the password storage stuff that I’ve blogged about before. For a supercomplex system (maybe users can have roles and permissions?) all you’d need to do is add the relationships to the user and permission classes, and of course you could do that from the user, from the permission, or from the schema directly, since DBIx::Class is so flexible. Anyway, feel free to comment with other ideas, but I feel pretty good about this direction and hope to have a 0.00001 release soon.

I keep running into people at parties or whatnot who mock me for using Perl and claim that “only .NET is a real programming language” (sic.) Most of the time they are trolling, but I figure I might as well make measurements for what I think of as a reasonably useful programming language. I’ll break this up into two groups of things. The first group is stuff that I want when programming at home for fun. The second is stuff that must be there if I am to use the language seriously at work; the second group contains the first group.

Note that if you have some neat thing to play with for a weekend, that’s fine. I had a good time playing with Factor but I don’t know of it meets all of these requirements. I wouldn’t do a big project with it or use it at work though.

Stuff Needed For the Language To Be Reasonably Fun

Anonymous Subroutines

Functions and objects are not the only abstractions that I need to get stuff done. If your langauge doesn’t have anonymous subroutines (anonymous classes do not count) I don’t want to hear about your language.

Dynamic Scope

Dynamic Scope is a powerful tool. You shouldn’t use it unless you really need it, but there are such times.

Thriving OSS Community

Are there open source, community maintained Web Frameworks, ORM’s, and other commonly needed tools in your language? If such tools must be bought or only come from a single source (that is, a very small community) then it isn’t good enough.

Extension Distribution Framework

If I cannot run a command (or maybe some GUI tool) and install new libraries (or packages or whatever they are called in your language) and their dependencies you are asking too much of me as a developer. If you don’t respect my time I don’t respect your langauge. Get out.

Strong backwards compatibility

Does your language try hard to maintain backwards compatibility? If your langauge breaks my code (no matter how bad the code is) every two or three years you (again) don’t respect my time as a developer.

Stuff Needed For the Language To Be Useful At Work

A Good ORM

Your ORM should support the big databases out there: MySQL, PostgreSQL, Oracle, SQL Server, and SQLite (for development.) I should be able to deploy the database from the ORM and also generate ORM files from the database. I should be able to automatically deploy serious stuff like foreign key constraints and unique constraints. As much as possible should be able to be overridden. I should be able to add some extension to a given table to automatically populate certain columns or whatever. I should also be able to make predefined searches that I can extend without going crazy.

A Good Web Framework

This isn’t hard. I use a pretty powerful web framework, but you don’t even need all of that. I need reasonably flexible dispatching, a well-defined “flow” (so I can hook in at different levels and do validation or whatever else,) sensible MVC helpers, and a development server so I don’t need to install a server on my laptop get work done.

There’s more depending on what you do. For example if you do a lot of event driven programming you need a solid framework (or at least language level programming.)

Obvious Stuff

Cross Platform

I can’t think of a lot of languages that violate this, but I might as well put it down.

Garbage Collection

My time is worth more than the computer’s. I don’t want to waste time with memory allocation.

What languages (aside from Perl and Javascript) support these features that you use?

I just released Log::Sprintf and Log::Structured to CPAN. They are both very simple modules, but they allow some powerful stuff.

Log::Sprintf will convert a hashref into a string given a specification almost conformant to Log::log4perl’s log specs. The example from the SYNOPSIS is as follows:

Also it was made with subclassing in mind from the start, so it is easy to add more flags as needed.

Log::Structured is a more generic tool but arguably more powerful. All it does is generate a “simple” (easily serializable) data structure and call a coderef that you give it with the data structure. What I hope to do with that is log to standard error using Log::Sprintf, but then log to a file using newline separated JSON documents. That means I can parse the log file DEAD easily and do what I want with it. Here’s the SYNOPSIS (after Log::Sprintf-ification) for that:

Anyway, hope you find a handy use for these!

At work I was recently given an external hard drive for backup purposes. Most of my coworkers are using some windows program to get the job done, but of course I can’t use that since I am using Linux. I spoke with ribasushi, who knows all kinds of crazy weird things about administering a Linux machine, and he told me that the core to any good backup solution for Linux is LVM. He uses it for no hassle MySQL backups too, so it’s not limited to full system backups like I did.

LVM

The high level explanation is basically that LVM abstracts away hardware to an extent that the actual used partitions span multiple harddrives or multiple partitions or whatever. You don’t really have to worry about it. The layers introduced by LVM are

• physical volumes (PV) – what would normally be considered partitions; actual partitions on the harddrive
• volume groups (VG) – a collection of one or more physical volumes
• logical volumes (LV) – the new, kernel level partitions which are partitions of a volume group

Thanks to the extra abstraction that LVM gives us, we can get atomic snapshots of any LV. Additionally, I don’t use this regularly but we will when we set this up, you can migrate a live LV from one PV to another PV, as long as the VG that the LV is on is contained on both PV’s. What a mouthful! Ok, let’s just get started

Set up the LVM

The first thing I did was format the external usb to be entirely a Linux LVM drive. I used cfdisk for that. The partition type for Linux LVM is 8E.

The rest of this is a bunch of commands, so I’ll do all of them with comments.

At this point you need to fiddle with grub and fstab to get your computer to boot into the new PVs. This part isn’t exactly hard, but I found it to be a hassle. Make sure to have a boot CD handy so that if you mess it up you can boot into the cd, mount the pv, chroot into that, and fix stuff from there. If you didn’t already know the commands to do what I just said I think the basic dance is:

Now here’s the crazy part:

bup

bup is a program to make backups. It uses the git format but not the git porcelain, so it is even more performant. I won’t go into all the awesome things about bup. If you want to read about those, click the link. Before you use bup you need to install it; the instructions are here.

The first thing you need to do is create a partition for your backup to reside on; you don’t want to back up root to itself. Also, the default bup location is ~/.bup, which I think is silly. We’ll take care of both of those things here:

Lastly you may want to use the following tiny script that I run daily to create the backups:

I run that with cron once a day at 6AM. At some point I hope to set up something else to profile the space usage and maybe make a handy graph of it. Anyway, hope you enjoyed this and find it useful!

Moo was just released! As mst says, Moo is almost, but not quite, two thirds of Moose. Or maybe Minimalistic Object Orientation. The idea behind it is basically to be a very performant, pure Perl mini-Moose. It supports lots of Moose features already and even more are on the way. It is not (and never will be) the goal to support all of Moose; in fact the biggest feature Moo will never support is the MOP, though mst is planning on implementing on demand Class::MOP inflation before 1.0.

I’m already working on a module which leverages Moo and it’s pretty neat! Sadly I’ve not had a lot of opportunities to use Moose at work, and on CPAN unless I’m working on something big (like DBIx::Class::DeploymentHandler) I try to avoid it for compile time speed reasons. So now I get to use builder and handles and method modifiers and roles and all that other cool Moose stuff that I like.

The really interesting thing about Moo is that if you read some of the (non Moo namespaced) modules in the package you’ll find that at it’s core Moo is really just a handy code generation library. Don’t believe me? Run the test suite with the environment variable SUB_QUOTE_DEBUG set and read the handy generated constructors and accessors!

I am hoping to actually go through the codebase for Moo and do something with it. I don’t want to promise anything yet, but I have some fun ideas that I may start as early as this week, so stay tuned!

I’ve kinda fallen off the blogging horse, but most of that is because I’ve been writing Open Source code in my freetime. I think generally that’s a worthwhile tradeoff, but I like blogging in general, when I have stuff to blog about, so I’m gonna try to mix in more blog posts; at least about what I’m doing.

The first thing I did was make a subclass of my DBIx::Class::Schema. It looks something like:

This is generally a good idea even if you aren’t using Catalyst. Having a test sub-class of your schema allows you to put various helper methods on your schema just for testing, as well as handy ways to deploy the schema. Since I’m so early on with this project I’ve been redeploying the database a lot, so it’s been very helpful to have my tests all just call init at the start of the test.

Speaking of redeploying, I’ve also been using an in memory SQLite for my tests, which is very handy. At some point later I’ll switch it to the deployment database. To connect to an in memory database use “dbi:SQLite:dbname=:memory:” as your dsn and you’ll be golden.

The next thing I did was made a special catalyst config file just for tests. This is what it looks like right now:

First off, the changed schema_class means that now $c->model(‘DB’)->schema is my test schema, so I get all those extra methods. Also note the xdsn key in connect info. JSON doesn’t natively support comments, so this is my hacky way of putting in an alternate dsn which I use when I want to look at the database after the test has finished (and probably failed.)

Now to get the above to apply, first I save it to “lynx_sms_testing.json” and then, before loading up my catalyst app in the test, I put

And then after loading catalyst I’ll do something along the lines of:

To generate and populate the database.

Of course a lot of this probably needs to start using some kind of fixture solution and put into a small module in t/lib so that I’m not copying around that BEGIN block in all of my controller tests, but this works for now and I’ll factor it into that once I have more than one controller test

Hello all,

I’m proud to announce DBIx::Class 0.08124! It’s been a VERY long time since 0.08123 and a this release brings lots of goodies.

My favorite is color-coded, correctly indented SQL, with placeholders filled in. Try it! Just do:

DBIC_TRACE_PROFILE=console DBIC_TRACE=1 ./foo.pl

There is also the exciting new “-ident” pseudofunction for SQL:

$rs->search({ foo => { -ident => ‘bar’ } })

which is the same as

$rs->search({ foo => \’bar’ })

but more introspectible!

Also, I’m a big fan of the new doc-map, which is at DBIx::Class::Manual::Features. Check it out!

The entire changelog is as follows:

0.08124 2010-10-28 14:23 (UTC)
* New Features / Changes
– Add new -ident “function” indicating rhs is a column name
{ col => { -ident => ‘othercol’ } } vs { col => \’othercol’ }
– Extend ‘proxy’ relationship attribute
– Use DBIx::Class::Storage::Debug::PrettyPrint when the
environment variable DBIC_TRACE_PROFILE is set, see
DBIx::Class::Storage for more information
– Implemented add_unique_constraints() which delegates to
add_unique_constraint() as appropriate
– add_unique_constraint() now poparly throws if called with
multiple constraint definitions
– No longer depend on SQL::Abstract::Limit – DBIC has been doing
most of the heavy lifting for a while anyway
– FilterColumn now passes data through when transformations
are not specified rather than throwing an exception.
– Optimized RowNum based Oracle limit-dialect (RT#61277)
– Requesting a pager on a resultset with cached entries now
throws an exception, instead of returning a 1-page object
since the amount of rows is always equal to the “pagesize”
– $rs->pager now uses a lazy count to determine the amount of
total entries only when really needed, instead of doing it
at instantiation time
– New documentation map organized by features
(DBIx::Class::Manual::Features)
– find( { … }, { key => $constraint } ) now throws an exception
when the supplied data does not fully specify $constraint
– find( col1 => $val1, col2 => $val2, … ) is no longer supported
(it has been in deprecated state for more than 4 years)
– Make sure exception_action does not allow exception-hiding
due to badly-written handlers (the mechanism was never meant
to be able to suppress exceptions)

* Fixes
– Fix memory leak during populate() on 5.8.x perls
– Temporarily fixed 5.13.x failures (RT#58225)
(perl-core fix still pending)
– Fix result_soutrce_instance leaks on compose_namespace
– Fix $_ volatility on load_namespaces (a class changing $_
at compile time no longer causes a massive fail)
– Fix find() without a key attr. choosing constraints even if
some of the supplied values are NULL (RT#59219)
– Fixed rels ending with me breaking subqueried limit realiasing
– Fixed $rs->update/delete on resutsets constrained by an
-or condition
– Remove rogue GROUP BY on non-multiplying prefetch-induced
subqueries
– Fix incorrect order_by handling with prefetch on
$ordered_rs->search_related (‘has_many_rel’) resultsets
– Oracle sequence detection now *really* works across schemas
(fixed some ommissions from 0.08123)
– dbicadmin now uses a /usr/bin/env shebang to work better with
perlbrew and other local perl builds
– bulk-inserts via $dbh->bind_array (void $rs->populate) now
display properly in DBIC_TRACE
– Incomplete exception thrown on relationship auto-fk-inference
failures
– Fixed distinct with order_by to not double-specify the same
column in the GROUP BY clause
– Properly support column names with symbols (e.g. single quote)
via custom accessors
– Fixed ::Schema::Versioned to work properly with quoting on
(RT#59619)
– Fixed t/54taint fails under local-lib
– Fixed SELECT … FOR UPDATE with LIMIT regression (RT#58554)
– Fixed CDBICompat to preserve order of column-group additions,
so that test relying on the order of %{} will no longer fail
– Fixed mysterious ::Storage::DBI goto-shim failures on older
perl versions
– Non-blessed reference exceptions are now correctly preserved
when thrown from udner DBIC (e.g. from txn_do)
– No longer disconnecting database handles supplied to connect
via a coderef
– Fixed t/inflate/datetime_pg.t failures due to a low dependency
on DateTime::Format::Pg (RT#61503)
– Fix dirtyness detection on source-less objects
– Fix incorrect limit_dialect assignment on Replicated pool members
– Fix invalid sql on relationship attr order_by with prefetch
– Fix primary key sequence detection for Oracle
(first trigger instead of trigger for column)
– Add various missing things to Optional::Dependencies
– Skip a test that breaks due to serious bugs in current DBD::SQLite
– Fix tests related to leaks and leaky perls (5.13.5, 5.13.6)

* Misc
– Entire test suite now passes under DBIC_TRACE=1
– Makefile.PL no longer imports GetOptions() to interoperate
better with Catalyst installers
– Bumped minimum Module::Install for developers
– Bumped DBD::SQLite dependency and removed some TODO markers
from tests (RT#59565)
– Do not execute t/zzzzzzz_sqlite_deadlock.t for regular module
installs – test is prone to spontaneous blow up
– DT-related tests now require a DateTime >= 0.55 (RT#60324)
– Makefile.PL now provides a pre-parsed DBIC version to the
Opt::Dep pod generator
– t/52leaks.t now performs very aggressive leak detection in
author/smoker mode

In the past I’ve only touched on the fact that I am a z shell user. I figured I’d make a post about some of the tweaks I made to my config (mostly my prompt) yesterday, in addition to why I use it at all.

First off, what are some features zsh has that make it work using for me?

Various bundled “modules.” For example, the zsh-mime-setup module enables me to “run” files with extensions and have the mime setup use the right program to open them. Great for stuff like pdf’s or whatever that.

Ridiculous amount of features and options:

auto_cd: you can type .. or foo or any other path and leave off the cd. Brilliant.
auto_pushd: cd implies pushd, which maintains a stack of directories. What that means is that popd (which I alias mk to) becomes “back” for navigation.
no_case_glob: simple as pie to allow case insensitive globbing

Oh and history is an amazingly powerful feature as well! First off, I save 10,000 lines in my history. That’s about a years worth of commands. Or at least it was in college. I think it’s probably less now, but still. It’s a lot. My history options allow me to leverage that to an extreme extent:

append_history: don’t overwrite my history at the beginning of a session. Append it.
various “dups” options: The main takeaway from this is that my history is a *stack*, not a list. I can’t stand doing ls, cd, ls and the pressing up twice to get back to ls.

Speaking of history, zsh has it’s own super configurable line editor, unsurprisingly called ZLE, which I use to bind / and ? (in cmd mode) to searching forward and backward in my history.

On top of that I have super configurable completion. Bash got this a while back, so it’s not as exciting. But one really cool thing you can do is fix up the completion such that kill lists all of your processes with nice levels and everything. It’s awesome. You should see it in action.

Of course, everyone has awesome prompts nowadays, and mine is fairly plain compared to many others, but it *does* contain the current branch, mode (merge, rebase, etc), and whether or not there are changes or staged changes. Oh and because it uses a generic module to do all of that it works for git, svn, hg, and lots of others.

Anyway, I actually spent a good part of yesterday reorganizing my configuration and adding in the version control bits to my prompt. I did a lot of research and I think it not only looks good but it works fairly efficiently. Check out my conf if you are interested.

Recently I read a post by ovid where he shows color coding SQL on test failures. I really wanted to steal his code for DBIx::Class‘s trace output. For MSSQL it would be especially helpful since our pagination involves two subqueries. ribasushi had pointed out in the past that all we need to do this (and do it correctly) was to refactor a bit of the test code and we’d have a proper parser and deparser.

Anyway, I got it into a state that I think is actually quite usable! Currently it’s in a dev release (1.67_01) of SQL::Abstract because we want to iron out any interface issues before blessing it. It’s also a little bit of a hassle to use at this point, but that will get worked out when it’s cored. To use it, just put the following in your MyApp::Schema:

Now if you set DBIC_TRACE you’ll get color-coded, indented, correctly nesting sql!

If you want to install it the easy way use App::CPAN::Fresh as follows:

And lastly, but arguably most importantly, a screenshot.

It’s certainly not perfect, I’d like to add some kind of width parameter so that it wraps more nicely. There are lots of configurable bits that aren’t documented yet; at some point I’ll get that taken care of. Anyway, enjoy!

This year at YAPC::NA nearly all the talks were filmed, including mine. I watched them so I could glean a bit more ideas for how to make talks in the future better.

Two things jumped out:

I feel better now than I felt after doing the talks

This is great. I feel like they went really well now.

The diversions into code weren’t that great

On the other hand, actually showing the underlying code for DBICDH was probably not worth the time spent.

And if you want to watch the videos, here they are:

• DBIx::Class::DeploymentHandler [slides]
• DBIx::Class [slides]