I got The Processing Book for Christmas and I finished it this morning.

It’s been a lot of fun going through the book and I learned a ton of different things. First off, book review:

The authors did exceptionally well at balancing six different topics:

• Programming
• The Processing API
• Introduction to Algorithms for Software Based Art
• Introduction to Software Based Art
• Synthesis sections which contain complex Processing examples
• Survey of existing art made with software

The programming stuff was pretty basic (just touched on higher level OO near the end) but the creators of Processing (who also wrote this book) did a good job at making the easy stuff easy. For example, unlike Java (which Processing is based upon,) you are not required to have a main function and all that jazz. You just wanna draw a rectangle?

Or you to get more complex you basically define a setup and draw method. setup gets called once and draw gets called for every frame. You don’t have to worry about the loop to call the draw method or all that jazz. It’s taken care of for you. So the following works fine:

I also thought the writers did extremely well introducing the API for Processing, but not stopping there. Too often technical books are just giant swaths of examples using the API. This book supposedly only goes over about half of the API, but I think they do well at explaining how to use the parts they’ve shown.

And then there is the algorithm stuff, which is what’s really the most important to me. A basic example would be the explanation of the easing technique, which is where something transforms to something else at a natural looking rate; think deceleration. I never would have though to do research on something like that. I plan on doing a post just showing examples of all the different algorithms, just for my own reference later.

They also do a really good job of pointing out creative ways to do things. Like that it’s interesting to use the location of speed of the mouse as something other than the location of something. Like maybe the faster you move your mouse the bigger something gets. Whatever. Stuff like that was peppered throughout the entire book, and then the reader is treated to a heavy dose of it in the Synthesis sections.

Next up, a little bit of introspection:

In college I never really had to read that much. I went to a technical school (LeTourneau University) so if I did do reading it was typically programming stuff, which meant fewer words than all those fictional works which I read before bed every night. The point is I’ve never had to be a fast reader. But since I’ve subscribed to the Iron Man feed I’ve gotten way better at knowing when and where to skim and where not to. I think this is what allowed me to finish the book in about four days. Check out what I made by the way! Make sure to press r and h to see the rgb and hsb breakdowns of the color the mouse is over. Pressing other buttons on the keyboard, right-clicking, and left-clicking should do some basic fun stuff too.

And lastly, just like I discovered during Thanksgiving, programming “bare metal” is a lot of fun! You may think, “Oh fREW, that’s not bare metal!” Look at it this way: if I want a button, I draw a square, manually run code that I wrote to check if the mouse is over the square, change the background of the square (for a mouse over) and then check if the mouse clicked the button; again, all my own code. So yeah, for graphics stuff there is a lot done for me, but for UI type stuff it’s pretty low-level. But it’s a lot of fun! Anyway, my hope is to make a really basic game with this stuff. Since Processing is just java stuff I don’t think I’ll have much trouble finding libraries to do what I need….

Anyway, hopefully this weekend I’ll do a post about the algorithms I learned, of which there are myriad. In the mean time I am going to do some research into this game idea….

If I were to pick one of the modules that I’ve written so far to be my legacy DBIx::Class::Helpers would be it. Maybe later it will be DBIx::Exceptions, but as of now that’s technically vaporware.

So over the Christmas break I’ve been working on updating it a bit. First and foremost I added the exciting DBIx::Class::Helper::ResultSet::Union helper. The best use case (I can think of) for a union is when you want to get data from multiple tables as if they were one table. Here is how one might do something like that with the new module (example ripped from wikipedia):

I’d argue that if you have tables like the above you are Doin it Rong, but there are other times when it might make more sense, like if you wanted to maybe have some kind of autocompleter that works for more than one table (Artist, Album, Track) where all of the things have names and ids.

As you should be able to see from the name of the Component I have gone from naming things DBIx::Class::Helper::* to DBIx::Class::Helper::$namespace::*. Mostly that was because my esteemed DBIx::Class janitor, ribasushi, suggested it. I almost had done it before, but his asking for it was enough to get me to do it. The old names will be available for all of the 2.* series and the 3.* series with a warning. After that they will be removed.

Speaking of version numbers, I have also switched from using RJBS versions to using a more normal versioning scheme. Again, I wouldn’t take the credit for this one as it is mostly due to mst and ribasushi’s prodding. Basically the deal is that although RJBS versions make it dead easy for me to release code, they make it hard for the user to see what’s going on. How much has changed between releases? All a user knows is if a release breaks backcompat or not. So I’ll be using the more normal x.yyyzzw where x breaks backcompat, y means major new features, z means bugfixes, and w means minor fixes like pod or dist fixes. I imagine that even that could be automated by looking at current and previous release… Maybe next time

Next up is a much more complete DBIx::Class::Helper::ResultSet::Random which allows the selection of an arbitrary amount of rows, instead of just one. As mentioned in the POD I’ve only tested it on a large table with SQL Server (2005.) To be more explicit, if you are using mysql and it is as slow as people always say, show me a benchmark and we can work together to make it fast. The only thing I can think of to make it fast is to use RowNumberOver and do an IN for a list of shuffled numbers generated with perl. We’ll see though. It works fine for me right now so I don’t care

While I was updating all these little bits I went ahead and pregenerated the DDL for the schema, so I no longer depend on SQLT, which is actually surprisingly heavy. So that’s pretty sweet.

Lastly, String::CamelCase had a new release that no longer fails tests! So I’ve added it as a dep and removed the conditional load code. This isn’t as big a deal as most of the other stuff, but it is certainly a nice change.

Because of all of the major changes I’ve released the new version to cpan as a developer version, so please check it out. At the very least if someone finds a missing dep or something that would be good. If I somehow messed up the doc or something that would be good to find out as well. I plan on releasing for realz before (or on?) New years.

As I mentioned in my last post I rewrote one of my personal apps (WebCritic) to use Web::Simple over the Thanksgiving holiday. It was exciting, writing one of the first apps to use the brand new Web::Simple. Of course, that also meant that I had to read incomplete doc, deal with examples that didn’t work, and in general just deal with the whole hassle of an immature project. Of course that was rewarding though, because I got a chance to help beef up the doc some, fix the broken examples, and convince mst to add a basic feature that will allow me to run the server standalone.

There are two cool, unqiue features of Web::Simple that I’ve used so far. The first is the use of HTML::Tags, which is undocumented so far, but super easy to use. See my last, and already linked to, post for an example of that one. Or if you are curious see my use of it. The nice thing is that I can write my html all nicely formatted or whatever, and the server outputs it without extraneous whitespace.

The other feature I’ve used allows really dumb, static serving for a Web::Simple app. Normally that would be discouraged, but because my app is meant to be run by devs from the commandline I kinda need this. Check it out:

So basically the static matcher just reads in a file under the static location and spits it back out. I haven’t checked what happens if you put ‘..’ in the path or anything like that, but again, this is for local usage, so I won’t stress much over it.

And because Web::Simple isn’t really geared to be a standalone server, I also redid the view part, which was entirely ExtJS, to use jQuery. Basically to use the Ext Grid I was loading the largish Ext javascript, the Ext css, and numerous images. jQuery really solves different, more basic problems, whereas Ext is an entire UI framework. I love Ext and think that all commercial, large scale projects should consider it. On the other hand, Ext has a weird license which makes me nervous including it in an OS project. Technically that’s ok, but if someone were to use my OS app to make money, I think they might have to pay the Ext people, which doesn’t sit nicely with me. Also it’s a huge framework that I was using probably 5% of for my project.

So instead of using the Ext Grid I just have a sortable table with columns that can be toggled. I do still miss the qtip (nice mouseover text) and general organization that I got from using the Ext framework, but I think the former can probably be solved with some research, and the latter is just my lack of knowledge coding “bare metal” javascript. Of course it’s not really bare metal since I’m using jQuery, but it’s certainly much closer.

In general this has been a lot of fun. Normally I’m a fan of large frameworks. At work we use (and I love!) ExtJS, Catalyst, Moose, and DBIx::Class. All of them (except maybe DBIx::Class) are probably the largest frameworks in their respective fields. But I get some perverse pleasure (and I do mean perverse) from using such a minimalistic toolset. I’d say that the switch to jQuery was warranted as Ext can significantly slow down the browser. The switch to Web::Simple over CGIApp was pretty much just for fun, but I learned a lot, and that’s certainly worth something.

Lastly, since revisiting this I realize that I should release it to CPAN. Once it’s complete (which depends on the next release of Web::Simple) I’ll release it. I’d expect my end of that to be done before the end of the week. As for Web::Simple, I’m not sure what else needs to be done to consider it release worthy, but I’ll be doing what I can to make that happen as well.

Guys! Web::Simple got released today! I fully intend on porting my personal CGIApp projects to Web::Simple immediately after writing this post. It really allows for a lot more possibilities, and not just the super sexy dispatching that is documented.

One thing I found interesting is that mst didn’t document the tags stuff anywhere at all. There used to be examples, but they seem to be gone. For a taste of those, see the tests.

For those unwilling to click the link, LOOK:

Also not included is Zoom, which I can’t seem to get to since shadowcat servers are DOWN. Zoom seems like something that should at least be referenced by Web::Simple. Zoom seems like the future of HTML templating to me; basically it uses xpath or css selectors to fill in values, instead of just putting text in text. This is good because it doesn’t just encourage the production of good HTML, but it also understands the actual structure of HTML, so it allows you to do cool things like OO generation of templates, which you can’t to with plaintext nicely.

Anyway, here I go! Hope you enjoyed this small survey of the new msTechnology

Surely you, dear reader, already know what this post is about. But on the off-chance that you don’t, make sure you read these posts from people smarter, more connected, and more balanced than I.

You may remember the beginnings of the technical posts on this blog. They were mostly sub-par because I had not begun read my articles before posting. I also had the bad habit of posting more than once a day. A lot of those posts were about perl6 and those posts were what got me more involved in blogging. I now have high hopes that I can use perl6 at work one day because of that time for me. perl6 has so many cool features that perl5 just didn’t seem to have and will certainly never have in the core…

But then something happened. I convinced my boss to open the IRC port at work so I could get help with DBIx::Class. While I was in #dbix-class I got a lot more connected with the community. It was there that I learned of the IronMan competition (how people ever found this blog before that I’ll never know.) And from that channel (and also the IronMan feed) I learned about all these amazing technologies that perl5 has.

Moose, Devel::Declare, and Catalyst just to name a few. I remember asking a question in #dbix-class and mst responding with something like,

I thought he was programming perl6! Devel::Declare brings some of that nice perl6 syntax to perl quite nicely, and the above is Devel::Declare (with MooseX::Declare or Method::Signatures::Simple probably) in action. Of course few people would say the above is perl6 specific, it’s really like most languages nowadays.

The point that I’m trying to get across here is that I learned a lot when I played with perl6. Certainly not as much as I learned when I flirted with ruby (for probably two years straight,) but I was much less experienced then. I love the fact that part of the perl mindset is stealing from other languages. And I love that we are stealing from perl6. Maybe one day I can say, “I love that we are stealing from perl5,” but perl6 (or really rakudo) just isn’t there yet, and in the meantime I have a job to do and cool projects to work on, and I’m glad that there is a push for us to discuss more technical merits of perl5 and perl6 and deal with less of the drama and politics. I am all in favor of doing more of what a programmer should be good at (technology) and less of what we are all good at (trash talking, etc.)

perl6 and perl6, two languages, same family, different lineages.

I enjoy updating this blog. Part of it is that I like writing, and part of it is that I kinda feel famous with all these great coders reading the words that I write. But I like programming better. That is why lately I’ve been posting less and coding more.

In posting modules to CPAN I’ve learned a lot of different things. First off, testing is easy. But before you test you have to set up your test environment. It’s not really hard, mostly it’s just a hassle. The obvious good thing about testing is that when I add features (my coworker asked for one for CGI::Application::Plugin::DBIx::Class just today) I know with pretty good certainty that I didn’t mess anything up, even if it’s a trivial change.

One problem I do have is that I consistently forget to add dependencies. The only think I can think to do is to have a vanilla perl install that I always test the release on. The problem with that though is that it is time consuming. If anyone has tips on how to deal with that let me know.

Another thing that I’ve learned is that Dist::Zilla is an invaluable tool for building releases. I love that it sets my version number, fills in my POD, sets up all my metadata, and even uploads my module to CPAN for me. It lets me spend less time learning about my build tool and more coding.

So I haven’t posted for kindav a long time. I have a lot planned to write about, but first, the reason it’s been so long and the solution to that problem.

You may remember my post regarding printing etc. When I told my boss about the large response I got, I think I miscommunicated and somehow heard that he wanted us to try a few more solutions before we hired someone else. I think he did want us to try more things, but not for three weeks with zero results! So I mentioned to him at some point that Chris J. Madsen, who gave us a business card at YAPC::NA, responded with a demo and everything. My boss said get it going and I got in touch with Chris and we planned a time to meet and plan things out. I think that was five days ago.

Since that meeting Chris has built us an extremely flexible module to generate postscript reports called PostScript::Report. I haven’t read the entirety of the code, but I do think that it’s designed well and documented well. So far I’ve made two reports based on his example and one from scratch, and it’s been a joy. I love his MooseX::AttributeTree and the use that it allows for something like this to set defaults in an elegant manner.

Chris has also been extremely responsive to issues I’ve had with the code. For example today we discussed some design issues and he solved a lot of them quickly and handily.

I take away three things from this experience.

First, it would have been cheaper if we had hired him immediately after we discovered that printing is hard and we have no experience in the problem domain. My original idea was to use TT to generate the PostScript for the reports, and that would have been pretty terrible.

Second, this is a really good Open Source business case. There is no way that we could sell this module to other shops. Our job isn’t to generate reports. Or job is to generate 50ish reports for one customer. By paying Chris to release it on CPAN we increase the business opportunities of the entire community and may get more features from that. I’d also like to think that because Chris put his name on it, as opposed to making it our code, it is of higher quality because the whole community will see it as a reflection of him.

Lastly, I’ve learned that Chris is a really great programmer. If anyone reading this blog has a job opening doing Perl development (and I think he would prefer in the Dallas area) I think he might appreciate the work. He may be working full time when you read this, but he’s still a great contractor and can do excellent work. See his website for details.

So with that behind me hopefully I’ll bring some content to this blog again

Recently we were doing something at work where we needed to get to a location deep in an HoH. We already had a solution that worked alright, but it was copy pasted in a couple places, it wasn’t tested, and it wasn’t documented. So I looked around on CPAN and found Hash::Path. It did exactly what we wanted, but the code was recursive instead of iterative (like our solution.) Because we weren’t going too deep I just installed it and figured I’d look at the actual differences later.

Well, I think last week I felt the urge to see what the difference actually was, empirically speaking. The following is my test case:

Ours stayed pretty close to 0.001 seconds, whereas the other version went a little slower (I think up to like, .010 s) but ran out of stack before I could test much deeper. So I put the testcase on RT in the hopes that the developer checked his email. He does and he updated the module just a couple days later! Pretty cool, huh?

I’ve been pretty busy/distracted lately. Normally I try to post 3 times a week about the cool things that I’m doing, but I haven’t this week because what I am doing isn’t that cool!

Basically I started working on the printing subsection of our app at work and it’s not looking like a lot of fun. First let me explain what makes it different than anything we’ve done before. In normal web dev you have the server generate html or maybe a pdf, send that to the user and let them print it. You might do some js to invoke the print dialogue, but beside that your work is done. Unfortunately the situation we are in now is that the users do something with the web app and the server prints out some stuff on a predefined printer. This will either happen immediately or via a cron-job, but either way it means we have to interact with printers much more directly than we normally do.

Now printing to a printer is actually not as hard as I thought it would be. We are on a windows system, and the way to do it in windows is basically as follows:

Done! That works just fine. The problem is that postscript is (as far as I can tell) no fun and pretty complicated. Of course going from a pdf to postscript is pretty easy using Ghostscript, so basically what it looks like is we should generate a pdf or a postscript file. I looked at using LaTeX a little bit but a 600 megabyte package really seems like overkill. And even then I don’t know LaTeX beyond math stuff I did in college. The perl offerings are good, but I am just out of my depth here. Here are the modules I looked at:

• PDF::API2 – Very Complete, but I’d need to know how PDF’s work to use this
• PDF::ReportWriter – Good, but I don’t know if I have enough control. And I’ll still need to learn a ton of XML directives.
• PDF::Template – Pretty much the same as above
• PDF::FromHTML – Again, this (and others like it) are alright, but there are just too many things that I may not be able to do that our layout requires.

I tend to be a fan of doing a lot of stuff in house, mostly because I love to learn and I tend to find the fun parts in things. But this is one of the times when my boss mentioned that we could pay someone else to do this that I am totally behind it. Even if I learned all of the stuff I’d need to know to do this myself, it would still take forever.

So here is my plan. I’d like to solicit people from the Perl community for some contract work. We currently have these printouts defined in terms of VB6 Data Designer … things. I would like to give someone a couple of the definitions, along with example printouts (scans), and screenshots. Conversion to PDF or PS are both fine. I was thinking that we’d use TT to fill in various parts of the new template. If the programmer does something like code in the PS or PDF to loop over data defined in the file, that’s ok, as long as it’s well commented as none of us know PDF or PS. Also if the coder does it in pure perl with one of the libraries above (except probably FromHTML) I’m fine with that, as I think they are all pretty quality modules. My ultimate goal is to make a conversion tool for the other 32 reports we’ll need printed out. If the programmer gets that started that would be awesome. I’m not at work right now, so I can’t post the examples, but I will post a couple links to examples tomorrow to the Catalyst ML along with contact information. I think the people who use Catalyst tend to be good coders. (I’m also going to email a few people I’ve met in person directly.) If you are interested, I will probably have posted everything by 2PM Central Time, so check the ML Archives then.

I hope to hear from you!

For the past 6 months or so I’ve been doing a lot more design and a lot less coding (due to design and a few other things) and it’s interesting to me what the results have been.

I remember when I got excited by grokking the concepts behind map and reduce. Don’t get me wrong, I am most happy with map and reduce, but to me they are great ways to be terse and clear. (Also fewer intermediate variables etc.) You use them when you build code but you don’t make large systems with them (yeah we could be talking about something more macro like MapReduce, but bear with me here.)

After getting comfortable with those I took the next logical step (in my mind) which was to start doing more functional programming. That was a huge step. When I first started doing that (dispatch tables) it was so that I could react appropriately to a fairly complex search without resorting to large, overly complex and opaque if blocks. I didn’t know that what I was doing even had a name, so I just called it a hash of anonymous subroutines. I knew that I didn’t come up with it as my boss had used one in the past in C land on a piece of hardware.

Awesome. It was about at this time that a coworker and I (Maestro to those from #moose) wrote the first Priority ordering system. Our first try wasn’t bad! I’ll explain the problem so that you can understand why we did what we did etc. Imagine that you work at FooBar Corporation. At FBC you go through bugs in a bugtracker and keep programmers on the ball. Unfortunately FBC can’t afford just anyone, so they tend to just get people who will start at the top and go down. Well, my company made a tool for FBC which would accurately prioritize the bugs in the tracker. (Note that it’s not really a bug tracker, I’m just trying to speak our language.)

For our first tracker my coworker made a special function which would take a single ticket and correlate all related issues (a ticket has many issues, it is not an issue.) We then iterated over the issues and depending on type of issue and the data inside the issue we would divide the priority if the data means good, or multiply if it means bad. Because we wanted a real scale of 1-10 and not 1-∞, we initialized the number with a log and then only multiplied by numbers between 1 and 0. Those are just details. It worked is the point. The code was kind of messy though, and we ended up doing some special casing in the driver for the dispatch table, which is a drag.

Fast forward 8 months. I’ve gotten much better at OO programming thanks to ExtJS and DBIx::Class and a little better at architecture thanks to Catalyst. I’ve also gotten much nicer syntax and features for Perl 5 OO from Moose, which makes writing and understanding OO Perl 5 code much nicer.

The same coworker and I work together again to write another priority system. This is because our initial try really wasn’t reusable, even though we tried for it to be. So this time we decide (after some implementation effort and discussion) to go with a triune system. Two-thirds of this was obvious to us and the third part was inspired by Catalyst. Here are (from memory) our three parts:

• Prioritizes Role
  • requires good dispatch table
  • requires bad dispatch table
  • requires Context instantiation method
  • requires apply method which knows how to apply the Context to the given dispatch table
  • provides generic dispatch driver and framework
• consumer of Prioritizes
• Priority::Context object
  • has priority and related methods
  • has a stash (thanks again catalyst) of random data that comes from the controller for use in the dispatch table methods

Now at first this system wasn’t clearly better than what we had before except that it was easily reusable. There aren’t a lot of places I could see reusing it, but once it’s written well it isn’t too unreasonable to start putting it in other projects just to add value… But then something happened that made all of our work worthwhile. Our customer wanted a certain “issue” to yeild a priority with a minimum of five, no matter what (they have a good reason, but the analogy can’t handle it, so you have to trust me.) With our original system we’d be forced to do some stuff after the dispatch table had been exhauseted. Now all we did was add a priority_minimum method to the Context, and then we added a trigger to priority which would ensure that the priority never dipped below priority_minimum. That alone made the whole design so much more elegant.