fREWdiculous!
6 Jul
Ever since I heard about SD (Simple Defects) I’ve been enamored with the idea of distributed issue tracking. Unfortunately SD is mostly unmaintained, undocumented, slow, and has lots of deps. I could probably get over the latter two, but the first two are deal breakers.
Fast forward eighteen months and I saw genehack’s App::GitGot. It’s a little sluggish (1.35s to merely list repos on my SSD) but it’s exciting because it can easily list my repos that are dirty or ahead by X commits. genehack is very receptive to changes, so this gave me an idea. First, what if we also listed how many unmerged brances there were? Next, what about unclosed issues? Obviously both of these ideas require some basic configuration, but it’s totally worth it.
I figured that step one would be to make a minimalist bug tracker, storing the information in the current git repo. The idea being that if I have a git tracker I can easily keep track of what I have to do next, so I won’t forget what else to do. I’ve been working on it every day a little bit since YAPC and it’s been pretty fun! The only thing is that there are already a few other implementations of what I am doing. I might as well list and evaluate them now.
First off, it’s implemented in Python, which is an immediate deal breaker, as I don’t have python installed everywhere I have git installed. On top of that it uses XML, which is not exactly easy to parse and modify. Finally it is basically dead, so it’s unlikely to get more features added in the future.
Implemented in Ruby, which is a problem, just like with git-issues. It is extremely full featured, so if you are working on a Linux system all the time I’d say this is the route to go. Unfortunately it uses rubygems; unless something has significantly changed in the past couple years that means it’s gonna be a little sluggish for a CLI app.
Again this is a Python app. It seems very full featured like ticgit, so maybe it has the same benefits as ticgit. It seems like it has some really neat features, but I just don’t like the python dep.
This is implemented in bash, so it immediately jumped out at me as something that is going to work anywhere that I have git installed. Unfortunately it is unmaintained for about three years now. I emailed the author to see what’s up, so we’ll have to see what happens there.
So my current dilemma is this: should I start afresh with a Perl app, possibly modelling after or even making it compatible with some of these other apps (be or ticgit) or should I pick up where bart left off with git-case? Before you jump to the former, realize that I don’t want to use any CPAN modules because I want this to be super simple to use anywhere one uses git; that is, all I should dep on is 5.8 and Git.pm (which is mostly useless.)
Currently I’m leaning towards git-case, maybe with a name change as I probably want to make some backwards incompatible changes. Any ideas? Hopes? Dreams?
24 Jun
A coworker sent this to our internal mailing list, partially to goad me into responding to the stupid comments. I don’t have an ars account, so I’ll just hope that trackbacks work.
Here are the arguments that someone actually put effort into making:
a) Magic ‘this’. This is this, except when this is that. JavaScript pushes you to use anonymous functions all over the place, except they always end up losing the proper context for the ‘this’ variable, so you end up having goofy code like “var _this = this” all over the place and then using that inside your callbacks or other functions. Some days I swear that the number of functions I manage to write that don’t use a renamed ‘this’ are actually smaller than the number that do.
If you think it’s magic you clearly don’t understand it. That’s not really an argument. I agree that sometimes it’s confusing, but sometimes programming is hard. Sorry?
b) 1 + “1″ – 1 = 10. Also, “1″ + 0 = “10″. Yes, this actually has caused bugs for our applications, where data that’s expected to be a number was loaded from a JSON file as a string due to a bug in another application, and the result was not good. All our loading code had to be updated to add a ton of type conversions all over the place. When I need something to be a number I really freaking absolutely want it to be a number, not a string or an object or null or anything else. Lua, which is very similar to JavaScript in most respects, fixed this problem by just not being retarded enough to use the same operator for addition and string concatenation.
I agree, this sucks, but I hit it rarely enough that it doesn’t discount the language.
c) Global by default variables. So even if you take the argument that dynamic typing is just “easier” because you don’t have to think about variable declarations, JavaScript throws that argument out the window by making you put ‘var’ in front of new identifiers all over the place. And then it silently screws you if you forget to.
Again, I disagree with this, but maybe because we use Perl and Perl works the same way (with the advantage of use strict, which some JavaScript implementations recently added but doesn’t really handle that issue as far as I can tell.)
d) Prototypes instead of classes. There are very few large-scale real-world JavaScript applications in existence that don’t plug in their own class system to work around the inherent uselessness of prototypes in large application architecture. Those same apps make minimal use of prototypes to extend the base JavaScript types, and only because JS was so poorly designed that even the two interesting built-in types it comes with are lacking half the features you’d expect them to have.
Wow I can’t believe how much I disagree with this. Prototypes give you more flexibility, so sure, they have caveats, but I totally think it’s worth it. Like a lot of things, you need to exercise discipline when using them, but I can think of numerous times in our real world, medium sized app (14K lines of JavaScript) where it’s helped to have.
e) Inability to create pass-by-value types. This is a frequent problem in just about every language aside from C++/D, actually. For those using JavaScript to write WebGL apps, take a look at all the linear algebra libraries for JavaScript. In 3D apps, you almost use vectors more often than you do scalars. Imagine if every integer in your app was passed by reference, so that “a = 1; b = a; b++” made both a and b equal to 2. Every little three component vector is a complete full object. They are passed by reference (the source of almost a half of the bugs in our WebGL game so far, in fact). They exist in great quantity, are heap-allocated, and are garbage-collected, which puts an intense amount of pressure on the GC which can and does result in GC pauses in even simple WebGL games, unless the developer jump through ridiculously complicated hoops to avoid creating new vectors in all the places where it’s logical to create new vectors. You can’t have operator overloading, so you have very large and ugly expressions to do basic operations. Accessing individual components is slow. The objects aren’t natively packed and hence are incredibly slow to push into a vertex buffer, unless you implement them as a Float32Array instances, which confuses the crap out of the optimizers of both V8 and SpiderMonkey currently. Did I mention they’re passed by reference?
I can’t relate with this. I just have never needed to pass anything other than a scalar by value, and those are pass by value. I can see that it’s a real issue for games and whatnot though. Also realize this argues against the poster’s language of choice.
f) No built-in include or require functionality. Seriously, still. Third-party libraries exist but almost all of them have some kind of bug or another, not least of which is a confusing caching problem in at least Chrome making doing actual development a pain in the butt.
Using a third party library should be expected. JavaScript isn’t Python, it’s not made to be “batteries included.” If there are bugs in a library you use you should fix those bugs.
g) Dynamic typing. Yes, I’m willing to start that argument. You start noticing it the most the second you stop writing little Web apps or Web pages and start writing large apps where you actually have data that persists for longer than a single mouse click or request/response cycle: add the wrong kind of object to an array to process later and get a crash later from a missing method or member in a completely different bit of code than where the actual mistake was. Fun times. Yes, Java makes static typing seem evil. No, Java/C#/C++ are not the one and only way to do static typing. Type inference, implicit interface binding, etc. give you all the “easy to deal with and not a lot of keystrokes” advantages of dynamic typing without all the bugs. The second most popular Web language — ActionScript 3 — is statically typed, in fact, despite otherwise being identical to JS/ECMAScript. As an aside, I get more crashes from the Python apps on my Fedora desktop than I do from the C/C++ apps (actually, none of the C/C++ apps on my desktop crash, now that I think about it). Missing member exceptions == so much easier to develop and maintain apps, right?
I don’t think I’ve had a type issue in our “large [app] where [we] actually have data that persists for longer than a single mouse click” for longer than a typo.
h) Speed. Yes, there has been some ridiculously immense amounts of effort by a large number of super bad-ass developers put into the language runtimes to make JS almost half as fast as a low-grade C compiler that a single college Junior could write in a few months. And LuaJIT is in the same boat as JS in terms of fundamental language limitations but manages to do better than every JavaScript implementation anyway. People who don’t understand what all the JS optimizations in V8 or such actually _do_ like to claim the JS can do amazing things speed-wise, but the reality is that all those optimizations are basically just “try very very hard to analyze the code to figure out types for variables and then compile it like a slightly retarded statically-typed language’s compiler would do it.” Oh, and there’s tracing, but then tracing also works on statically typed languages (and works better due to a lack of need for type guards in the generated machine code). Not a single one of those whizbang optimizations was invented by or for JS, in fact; most were taken from research JVMs (Java is evil!) or classical OOP languages (prototypes are awesome!).
Nice! So the poster says: “JS is fast, but not as fast as C and the things that made it fast were taken from other VM’s so it doesn’t really count.” Ok Mr. C#, I’m sure you’re correct since C# is entirely original research
Also, is C# as fast as C?
i) No IntelliSense even possible. Want to see what methods exist on that variable you’ve got there on line 187 of foo.js in your text editor? Too bad. Go trace through the code until you figure out where it was initialized, then go trace through the code to find out what its prototype has on it. And then hope there’s no code dynamically changing the prototype behind your back. In fact, just run it in a browser and set breakpoints, because finding out anything useful about the value any other way is basically impossible for any codebase larger than the toy_web_app.html sites that JavaScript apologists use to glorify the ease and simplicity of JavaScript. Some code editors try _really_ hard to do better, and almost kinda sorta succeed for the really simple cases, sometimes, once.
Yeah, that’s a real thing I guess, but I’m not the target user of IntelliSense anyway.
j) No advantage. JavaScript isn’t even special compared to other dynamically typed language. It is not capable of doing anything interesting at all all that can’t also be done by Lua, Python, Ruby, etc. None of the JS implementations are any faster than LuaJIT or PyPy or various other advanced JIT-ing implementations of other dynamic languages. JS has no plus sides to it compared to other commonly available languages. Oh, except it runs natively in a Web browser without a plugin. Which is the only reason in the world why it’s so popular. In fact, it’s the only reason it event _exists_. If someone 10 years ago had just thought, “heck, let’s drop an existing well-designed and well-established language into our browser and get the other guys to do the same instead of making everyone use this goofy little hackjob that NetScape came up with,” the Web would look a lot different (better) today. Just imagine the future if Chrome dropped Python into Chrome as a supported language. Or actually, imagine this: Google drops C/C++ into Chrome as a supported language (http://code.google.com/p/nativeclient/).
The fact that it’s the most widely deployed VM in the world isn’t an advantage? The fact that there are three major VM’s leading to healthy competition and improvements isn’t an advantage? The fact that it’s in all browsers right now (unlike some crazy future where browsers need to implement C or (good lord!) C++ safely) isn’t a feature?
Maybe I’m just a tool for thinking this guy just has confirmation bias. I’ve done the same thing of course in arguing against C#, so maybe I’m just as bad, but I can hardly take these arguments as reasons not to use JavaScript.
24 May
Most open source git repositories that I interact with are hosted at git.shadowcat.co.uk. A few typical repo urls (read/write) hosted here looks like:
1 2 3 4 | dbsrgits@git.shadowcat.co.uk:DBIx-Class.git catagits@git.shadowcat.co.uk:Catalyst-Runtime.git p5sagit@git.shadowcat.co.uk:Devel-Declare.git gitmo@git.shadowcat.co.uk:Moose.git |
A handy trick is to make a file at ~/.ssh/config with the following in it:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | host catagits user catagits hostname git.shadowcat.co.uk port 22 identityfile ~/.ssh/id_dsa host dbsrgits user dbsrgits hostname git.shadowcat.co.uk port 22 identityfile ~/.ssh/id_dsa host p5sagit user p5sagit hostname git.shadowcat.co.uk port 22 identityfile ~/.ssh/id_dsa host gitmo user gitmo hostname git.shadowcat.co.uk port 22 identityfile ~/.ssh/id_dsa |
Now that that’s done the repos before are simply:
1 2 3 4 | dbsrgits:DBIx-Class.git catagits:Catalyst-Runtime.git p5sagit:Devel-Declare.git gitmo:Moose.git |
I actually did that for my work and github stuff too, since I just don’t have the time to type those stupid domains all the time.
Enjoy!
17 May
I have now converted something like 25 repositories from svn to git. I can fix undetected merges, correctly import tags, and clean up ugly (svk) commit messages.
With this knowledge I hope to write a small, non-free eBook (7.50 USD I think.) But first I’d like a chance to convert your repository! The more repositories that I convert the more ground the ebook can cover. I’ve converted a number of repos for CPAN modules and I’d love to do more. My first thought was to convert the modules in the Catalyst repo, but sadly I’m not sure which ones I should do.
So if you are interested in having your repo converted, I am totally willing to do it. All I need from you is an email, comment, ping, etc saying you are interested and I need to you be willing to check the converted repo to ensure that it’s good. I’ve gotten pretty good at this but I’m not perfect and I’m not taking the blame if you miss something
If you’d like to try your hand at doing this, I put all my conversion scripts online, so feel free to take a peak.
13 Apr
I’m just releasing my first new release of DBIx::Class::DeploymentHandler in six months! For the most part the release is just a few doc tweaks, but it does have one important new feature, the “_any” version.
If you didn’t already know, DBICDH has a handy little directory structure for how your deploys work. If you haven’t seen it, take a look. This new release allows you to use _any in place of a version or version set, which will run the given files no matter what version you are deploying to.
Enjoy!
6 Apr
I’ve been using this for a couple years now, and I figure I’ll repost it so that it’s easy to find (for me.)
| Sugar, White Granulated | 1 Cup |
| Instant Non Fat Dry Milk | 1 Cup |
| Non Dairy Creamer | 1/2 Cup |
| Instant Tea | 1/2 Cup |
| Cinnamon, Ground | 1 Teaspoon |
| Ginger, Ground | 1 Teaspoon |
| Salt, Table | 1/2 Teaspoon |
| Nutmeg, Ground | 1/2 Teaspoon |
| Allspice, Ground | 1/4 Teaspoon |
| Cloves, Ground | 1/4 Teaspoon |
| Cayenne Pepper | 1/8 Teaspoon |
And for convenience, here is the recipe x 4, which is how much I like to make:
| Sugar, White Granulated | 4 Cup |
| Instant Non Fat Dry Milk | 4 Cup |
| Non Dairy Creamer | 2 Cup |
| Instant Tea | 2 Cup |
| Cinnamon, Ground | 4 Teaspoon |
| Ginger, Ground | 4 Teaspoon |
| Salt, Table | 2 Teaspoon |
| Nutmeg, Ground | 2 Teaspoon |
| Allspice, Ground | 1 Teaspoon |
| Cloves, Ground | 1 Teaspoon |
| Cayenne Pepper | 1/2 Teaspoon |
Enjoy!
15 Mar
Thanks to some idle chatting in the #dbix-class channel on irc.perl.org I came up with DBIx::Class::Helper::Row::RelationshipDWIM. The gist of it is that you get to type
1 | __PACKAGE__->has_many(addresses => '::Address', 'person_id' ) |
instead of
1 | __PACKAGE__->has_many(addresses => 'MyApp::Schema::Result::Address', 'person_id' ) |
That yields a total sugar (with candy) of the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | package Lynx::SMS::Schema::Result::MessageParent; use Lynx::SMS::Schema::Candy; primary_column id => { data_type => 'int', is_auto_increment => 1, }; column account_id => { data_type => 'int' }; column type_id => { data_type => 'int' }; column caller_id => { data_type => 'int', size => 11, is_nullable => 1, }; column message => { data_type => 'nvarchar', size => 1000, }; column when_created => { data_type => 'datetime', set_on_create => 1, }; column voice_id => { data_type => 'int', is_nullable => 1, }; belongs_to account => '::Account', 'account_id'; belongs_to voice => '::Voice', 'voice_id'; belongs_to type => '::Type', 'type_id'; has_many children => '::MessageChild', 'message_parent_id'; 1; |
Pretty nice.
9 Mar
I’m extremely proud to announce a fairly major release of DBIx::Class::Candy, 0.002000. Not only are the tests much more complete as well as the underlying code much more comprehensible, but the usage of the Candy can now be even sweeter.
To get the full features of DBIx::Class::Candy you’ll want to first create the following base class:
(Of course you can call this sugar if you hate my naming scheme or rainbows if you love it.)
1 2 3 4 5 6 7 8 9 | package MyApp::Schema::Candy; use parent 'DBIx::Class::Candy'; sub base () { 'MyApp::Schema::Result' } sub perl_version () { 12 } sub autotable () { 1 } 1; |
Now a basic id, name table would look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | package MyApp::Schema::Result::Permission; use MyApp::Schema::Candy; primary_column id => { data_type => 'int', is_auto_increment => 1, }; unique_column name => { data_type => 'varchar', size => 30, }; 1; |
id got set to the pk, name got a unique constraint, the table was named permissions, perl 5.12 features were imported, the base class was set to MyApp::Schema::Result. How awesome is that! Not that you can do the same thing as above without a subclass if you like still:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | package MyApp::Schema::Result::Permission; use DBIx::Class::Candy -base => 'MyApp::Schema::Result', -perl5 => v12, -autotable => v1; primary_column id => { data_type => 'int', is_auto_increment => 1, }; unique_column name => { data_type => 'varchar', size => 30, }; 1; |
I should give credit where credit is due. Getty had lots of ideas for improvements, but the first one I implemented (due to how easy it was and how much I liked it) was primary_column. mst had the idea of automatically generating the table name and using a subclass of candy to avoid boilerplate. Enjoy!
1 Mar
I really like git. It has an excellent suite of tools bundled with it from the start and it gets lots of updates and active development. Today I was looking at the latest git version (1.7.4) because I was installing it on a new machine and, as usual with new versions of things, I perused the
release notes. What really caught my eye was this:
1 2 | * "git log -G<pattern>" limits the output to commits whose change has added or deleted lines that match the given pattern. |
I don’t know about you guys, but I fake that feature 2 or 3 times a month by just doing git log -p | grep foo -C50. It’s not nearly as nice as it catches other things, breaks color, etc. Anyway, I decided that instead of waiting for my already non-standard ubuntu repo to catch up, I’d just build it.
First, I checked out git with my installed git:
1 | git clone git://git.kernel.org/pub/scm/git/git.git |
That will take a while, so while that’s going on install
1 2 3 4 5 | build-essential autoconf asciidoc libcurl4-openssl-dev gettext |
Note that asciidoc tries to pull in a ton of TeX junk. Don’t let it, you don’t need that at all.
Once that’s done do something like the following:
1 2 3 4 | make configure ./configure --prefix=/opt make -j5 all doc sudo make install install-doc install-html |
I used the prefix because I’d rather not install on top of my existing stuff; you might want to install to home. Anyway, after doing that ensure that $prefix/bin is in your path and enjoy a brand new git!
17 Feb
My fiancĂ©e and I have not yet picked out a date for our wedding, but we do know that we want it outdoors. We have scoped out a number of locations that can handle indoor and outdoor weddings just in case there is bad weather, but we’d prefer to have perfect weather.
After some searching I found NOAA’s NSSL, which has ridiculous amounts of data. Instead of most websites, which give you the average high temperature and average low temperature for a given day of the year from the past three years, this gives hourly measurements for basically anything back to 1910. Of course some stations are newer and whatnot, but it’s a lot of data.
Their website only lets you get one day of data at a time, so I wrote a screen scraper using the excellent Web::Scraper. Here’s most of it:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | #!/usr/bin/env perl use Modern::Perl; use JSON; use URI; use Web::Scraper; # http://www.unidata.ucar.edu/cgi-bin/gempak/manual/apxA_index my %data_to_grab = ( SMPH => 'wind-speed', TMPF => 'temperature', RELH => 'humidity', ); my $data_str = join ';', sort keys %data_to_grab; my $weather = scraper { # there isn't a class, so we find the table with width 90 process "table[width=90] tr", "datas[]" => scraper { process "td:nth-child(2)", 'when' => 'TEXT'; my $i = 2; for (sort keys %data_to_grab) { $i++; process "td:nth-child($i)", $data_to_grab{$_} => 'TEXT'; } }; }; sub moar_data { my ($y, $m, $d) = @_; my $res = $weather->scrape( URI->new(sprintf 'http://data.nssl.noaa.gov/dataselect/nssl_result.php?datatype=sf&sdate=%4i-%02i-%02i&hour=00&sdate2=%4i-%02i-%02i&hour2=23&outputtype=list¶m_val=%s&area=&area=@DFW', $y, $m, $d, $y, $m, $d, $data_str)); warn sprintf "%4i-%02i-%02i\n", $y, $m, $d; sleep 3 + rand(2); grep { # undefined when means there wasn't actualy an observation defined $_->{when} && # ignore headers $_->{when} ne 'YYMMDD/HHMM' } @{$res->{datas}} } my @end = ( map { my $year = $_; (map { moar_data($year, 9, $_) } (1..30)), (map { moar_data($year, 10, $_) } (1..31)), } ( 1990..2010 ) ); print to_json(\@end, { pretty => 1 }); |
The scraper object grabs a bunch of the data from TD’s in the table, skipping the first TD. I made the moar_data function which just takes year, month, day so that I could get more data. It outputs all the data as json, my prefered data format.
If you did the math at home, you realized this is a ridiculous amount of observations; something along the lines of 14 thousand observations. That means you can’t just look at it. So I also wrote a little tool to slice and dice the data. Check it out:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | #!/usr/bin/env perl use Modern::Perl; use JSON; use List::Util qw(min max); use Statistics::Basic qw(mean stddev); my $field = $ARGV[0]; die "please choose a field to research" unless $field; die "$field is not a valid field!" unless grep { $_ eq $field } qw(wind-speed temperature humidity); # expected format: # [ # { # wind-speed => 123, # temperature => 123, # when => 'YYMMDD/HHMM', # }, # ... # ] my $data = from_json(do { local $/ = undef; open my $fh, 'weather.json'; <$fh> }); # final format: # MMDD/HHMM => [{...}], my %by_day; for (@$data) { my $when = $_->{when}; $when =~ s/^\d\d//; # remove the year part $by_day{$when} = [] unless $by_day{$when}; push @{$by_day{$when}}, $_; } say 'datetime ,mean ,stddev,min,max '; for (sort keys %by_day) { my @list = map $_->{$field}, grep { # this is weird, -9999.00 is apparently what they used # before they had undef? defined $_->{$field} && $_->{$field} != -9999.00 } @{$by_day{$_}}; my $avg = sprintf '%3.2f', mean \@list; my $min = sprintf '%3.2f', min @list; my $max = sprintf '%3.2f', max @list; my $stddev = sprintf '%3.2f', stddev \@list; say "$_,$avg,$stddev,$min,$max"; } |
Anyway, this was a fun project and a nice little valentines day surprise. Hope someone finds it useful