Chai Tea Mix

Author: fREW Schmidt

6 Apr

I’ve been using this for a couple years now, and I figure I’ll repost it so that it’s easy to find (for me.)

Sugar, White Granulated	1 Cup
Instant Non Fat Dry Milk	1 Cup
Non Dairy Creamer	1/2 Cup
Instant Tea	1/2 Cup
Cinnamon, Ground	1 Teaspoon
Ginger, Ground	1 Teaspoon
Salt, Table	1/2 Teaspoon
Nutmeg, Ground	1/2 Teaspoon
Allspice, Ground	1/4 Teaspoon
Cloves, Ground	1/4 Teaspoon
Cayenne Pepper	1/8 Teaspoon

And for convenience, here is the recipe x 4, which is how much I like to make:

Sugar, White Granulated	4 Cup
Instant Non Fat Dry Milk	4 Cup
Non Dairy Creamer	2 Cup
Instant Tea	2 Cup
Cinnamon, Ground	4 Teaspoon
Ginger, Ground	4 Teaspoon
Salt, Table	2 Teaspoon
Nutmeg, Ground	2 Teaspoon
Allspice, Ground	1 Teaspoon
Cloves, Ground	1 Teaspoon
Cayenne Pepper	1/2 Teaspoon

Enjoy!

0 Comments

Filed under: Uncategorized

DBIx::Class::Helper::Row::RelationshipDWIM: Awesome!

Author: fREW Schmidt

15 Mar

Thanks to some idle chatting in the #dbix-class channel on irc.perl.org I came up with DBIx::Class::Helper::Row::RelationshipDWIM. The gist of it is that you get to type

1	__PACKAGE__->has_many(addresses => '::Address', 'person_id' )

instead of

1	__PACKAGE__->has_many(addresses => 'MyApp::Schema::Result::Address', 'person_id' )

That yields a total sugar (with candy) of the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

package Lynx::SMS::Schema::Result::MessageParent;

use Lynx::SMS::Schema::Candy;

primary_column id => {
data_type => 'int',
is_auto_increment => 1,
};

column account_id => { data_type => 'int' };
column type_id => { data_type => 'int' };

column caller_id => {
data_type => 'int',
size => 11,
is_nullable => 1,
};

column message => {
data_type => 'nvarchar',
size => 1000,
};

column when_created => {
data_type => 'datetime',
set_on_create => 1,
};

column voice_id => {
data_type => 'int',
is_nullable => 1,
};

belongs_to account => '::Account', 'account_id';
belongs_to voice => '::Voice', 'voice_id';
belongs_to type => '::Type', 'type_id';
has_many children => '::MessageChild', 'message_parent_id';

1;

Pretty nice.

0 Comments

Filed under: Uncategorized

New Stuff in DBIx::Class::Candy

Author: fREW Schmidt

9 Mar

I’m extremely proud to announce a fairly major release of DBIx::Class::Candy, 0.002000. Not only are the tests much more complete as well as the underlying code much more comprehensible, but the usage of the Candy can now be even sweeter.

To get the full features of DBIx::Class::Candy you’ll want to first create the following base class:

(Of course you can call this sugar if you hate my naming scheme or rainbows if you love it.)

1
2
3
4
5
6
7
8
9

package MyApp::Schema::Candy;

use parent 'DBIx::Class::Candy';

sub base () { 'MyApp::Schema::Result' }
sub perl_version () { 12 }
sub autotable () { 1 }

1;

Now a basic id, name table would look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

package MyApp::Schema::Result::Permission;

use MyApp::Schema::Candy;

primary_column id => {
data_type => 'int',
is_auto_increment => 1,
};

unique_column name => {
data_type => 'varchar',
size => 30,
};

1;

id got set to the pk, name got a unique constraint, the table was named permissions, perl 5.12 features were imported, the base class was set to MyApp::Schema::Result. How awesome is that! Not that you can do the same thing as above without a subclass if you like still:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

package MyApp::Schema::Result::Permission;

use DBIx::Class::Candy
-base => 'MyApp::Schema::Result',
-perl5 => v12,
-autotable => v1;

primary_column id => {
data_type => 'int',
is_auto_increment => 1,
};

unique_column name => {
data_type => 'varchar',
size => 30,
};

1;

I should give credit where credit is due. Getty had lots of ideas for improvements, but the first one I implemented (due to how easy it was and how much I liked it) was primary_column. mst had the idea of automatically generating the table name and using a subclass of candy to avoid boilerplate. Enjoy!

0 Comments

Filed under: Uncategorized

Git 1.7.5.1 from git on ubuntu

Author: fREW Schmidt

1 Mar

I really like git. It has an excellent suite of tools bundled with it from the start and it gets lots of updates and active development. Today I was looking at the latest git version (1.7.4) because I was installing it on a new machine and, as usual with new versions of things, I perused the
release notes. What really caught my eye was this:

1 2	* "git log -G<pattern>" limits the output to commits whose change has added or deleted lines that match the given pattern.

I don’t know about you guys, but I fake that feature 2 or 3 times a month by just doing git log -p | grep foo -C50. It’s not nearly as nice as it catches other things, breaks color, etc. Anyway, I decided that instead of waiting for my already non-standard ubuntu repo to catch up, I’d just build it.

First, I checked out git with my installed git:

1	git clone git://git.kernel.org/pub/scm/git/git.git

That will take a while, so while that’s going on install

1
2
3
4

build-essential
autoconf
asciidoc
libcurl4-openssl-dev

Note that asciidoc tries to pull in a ton of TeX junk. Don’t let it, you don’t need that at all.

Once that’s done do something like the following:

1
2
3
4

make configure
./configure --prefix=/opt
make -j5 all doc
sudo make install install-doc install-html

I used the prefix because I’d rather not install on top of my existing stuff; you might want to install to home. Anyway, after doing that ensure that $prefix/bin is in your path and enjoy a brand new git!

4 Comments

Filed under: Uncategorized

Screen Scrape for Love with Web::Scraper

Author: fREW Schmidt

17 Feb

My fiancée and I have not yet picked out a date for our wedding, but we do know that we want it outdoors. We have scoped out a number of locations that can handle indoor and outdoor weddings just in case there is bad weather, but we’d prefer to have perfect weather.

After some searching I found NOAA’s NSSL, which has ridiculous amounts of data. Instead of most websites, which give you the average high temperature and average low temperature for a given day of the year from the past three years, this gives hourly measurements for basically anything back to 1910. Of course some stations are newer and whatnot, but it’s a lot of data.

Their website only lets you get one day of data at a time, so I wrote a screen scraper using the excellent Web::Scraper. Here’s most of it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

#!/usr/bin/env perl

use Modern::Perl;
use JSON;
use URI;
use Web::Scraper;

# http://www.unidata.ucar.edu/cgi-bin/gempak/manual/apxA_index
my %data_to_grab = (
SMPH => 'wind-speed',
TMPF => 'temperature',
RELH => 'humidity',
);

my $data_str = join ';', sort keys %data_to_grab;

my $weather = scraper {
# there isn't a class, so we find the table with width 90
process "table[width=90] tr", "datas[]" => scraper {
process "td:nth-child(2)", 'when' => 'TEXT';
my $i = 2;
for (sort keys %data_to_grab) {
$i++;
process "td:nth-child($i)", $data_to_grab{$_} => 'TEXT';
}
};
};

sub moar_data {
my ($y, $m, $d) = @_;
my $res = $weather->scrape( URI->new(sprintf 'http://data.nssl.noaa.gov/dataselect/nssl_result.php?datatype=sf&sdate=%4i-%02i-%02i&hour=00&sdate2=%4i-%02i-%02i&hour2=23&outputtype=list&param_val=%s&area=&area=@DFW', $y, $m, $d, $y, $m, $d, $data_str));

warn sprintf "%4i-%02i-%02i\n", $y, $m, $d;
sleep 3 + rand(2);
grep {
# undefined when means there wasn't actualy an observation
defined $_->{when} &&
# ignore headers
$_->{when} ne 'YYMMDD/HHMM'
} @{$res->{datas}}
}

my @end = (
map {
my $year = $_;
(map { moar_data($year, 9, $_) } (1..30)),
(map { moar_data($year, 10, $_) } (1..31)),
} ( 1990..2010 )
);

print to_json(\@end, { pretty => 1 });

The scraper object grabs a bunch of the data from TD’s in the table, skipping the first TD. I made the moar_data function which just takes year, month, day so that I could get more data. It outputs all the data as json, my prefered data format.

If you did the math at home, you realized this is a ridiculous amount of observations; something along the lines of 14 thousand observations. That means you can’t just look at it. So I also wrote a little tool to slice and dice the data. Check it out:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

#!/usr/bin/env perl

use Modern::Perl;
use JSON;
use List::Util qw(min max);
use Statistics::Basic qw(mean stddev);

my $field = $ARGV[0];

die "please choose a field to research" unless $field;
die "$field is not a valid field!" unless grep { $_ eq $field }
qw(wind-speed temperature humidity);

# expected format:
# [
# {
# wind-speed => 123,
# temperature => 123,
# when => 'YYMMDD/HHMM',
# },
# ...
# ]

my $data = from_json(do {
local $/ = undef;
open my $fh, 'weather.json';
<$fh>
});

# final format:
# MMDD/HHMM => [{...}],
my %by_day;

for (@$data) {
my $when = $_->{when};
$when =~ s/^\d\d//; # remove the year part
$by_day{$when} = [] unless $by_day{$when};
push @{$by_day{$when}}, $_;
}

say 'datetime ,mean ,stddev,min,max ';
for (sort keys %by_day) {
my @list = map $_->{$field}, grep {
# this is weird, -9999.00 is apparently what they used
# before they had undef?
defined $_->{$field} &&
$_->{$field} != -9999.00
} @{$by_day{$_}};
my $avg = sprintf '%3.2f', mean \@list;
my $min = sprintf '%3.2f', min @list;
my $max = sprintf '%3.2f', max @list;
my $stddev = sprintf '%3.2f', stddev \@list;
say "$_,$avg,$stddev,$min,$max";
}

Anyway, this was a fun project and a nice little valentines day surprise. Hope someone finds it useful

2 Comments

Filed under: Uncategorized

Catalyst Git Conversion

Author: fREW Schmidt

12 Feb

Hello All!

Some of you already know that I am working on converting the Catalyst repository to git. I am happy to announce that I am closing in on completion!

The current state of the git repo: https://github.com/frioux/Catalyst
The script to convert it: https://github.com/frioux/Git-Conversions/blob/master/cat-convert

The only things I know of that we must have before we finalize this conversion is:

Is it correct that the svn user rjk is Ronald J Kimball: rjk AT linguist DOT dartmouth DAWT edu ?
who is svn user didls?

Also, if you’d like to help ensure the sanity of the repo it would be great if you looked at it! Here are a few tools I use to try to get a feel for the quality of the final export:

1	gitk --all

Perusing the repo with gitk is good; another great thing to do is to View -> Edit View and click “Strictly sort by date”. This is helpful for finding duplicate commits. Note that there are some commits that look duplicate in this repo but actually aren’t; people decided it would be best to edit multiple branches in a single commit instead of just dealing with that at merge time.

1	git shortlog -s

This will make it clear if I misspelled your name.

Also looking at blames of files can be handy.

Hopefully this helps!

Update: hobbs found confirmation for rjk and identity for didls, so we are good to go assuming no one finds any issues.

(I hope to have a long blog post soon explaining some of the techniques I used to get this all working. Be prepared though, I’m not as smart as Haarg, so mostly it’s a manual process )

0 Comments

Filed under: Uncategorized

My Fork of ExtJS

Author: fREW Schmidt

8 Feb

Sencha has been pretty slow at fixing bugs for the company where I work. We not only pay for usage but also for forum support. I’ve decided to personally (that is, me, not my company) fork ExtJS and maintain a set of patches on top of it. Those patches will be licensed as GPLv3 (because they must, because ExtJS is licensed as GPLv3) and Sencha can take them and merge them into core whenever they want.

I’ve tried to deal with this with overrides, but that is a hassle as you must manually check to see if you missed something, where this is much more of a natural process, since I’ll just rebase my changes regularly. Also, some things are impossible to override. For example, see the first link above; without completely duplicating the definition, we can’t easily fix that issue.

Please fork it and help out!

0 Comments

Filed under: Uncategorized

New stuff in DBIx::Class::Helpers

Author: fREW Schmidt

31 Jan

I just released a new version of DBIx::Class::Helpers and it has two new components: DBIx::Class::Helper::ResultSet::ResultClassDWIM and DBIx::Class::Helper::Schema::GenerateSource.

Helper::ResultSet::ResultClassDWIM

This component solves an issue I’ve seen both by myself and with my coworkers; it’s too hard to remember/type the following:

1
2
3

my $rs = $schema->resultset('Foo')->search($q, {
result_class => 'DBIx::Class::ResultClass::HashRefInflator',
});

So I wrote this component which will let you generically write:

1
2
3

my $rs = $schema->resultset('Foo')->search($q, {
result_class => '::HashRefInflator',
});

or use the specially hardcoded:

1
2
3

my $rs = $schema->resultset('Foo')->search($q, {
result_class => '::HRI',
});

Handy right?

Helper::Schema::GenerateSource

This component is a little more unusual. The idea is to take care of some of the issues I mentioned here. It doesn’t solve everything due to some design issues in DBIx::Class, which I hope to take care of soon. Basically the idea is that instead of the boilerplate files that you get when you use DBIx::Class::Helper::Row::SubClass, you can instead just do the following in your schema:

1
2
3
4
5
6
7
8
9

package Foo::Schema;

__PACKAGE__->load_components('Helper::Schema::GenerateSource');

# ...

__PACKAGE->generate_source(User => 'MyCompany::Result::User');

1;

The main issue is that even though this correctly associates a new source to the schema, you cannot currently add relationships to the source. I’ll make another post when I fix that, but I doubt I’ll get it into the next release of DBIx::Class.

In other news I’m doing some pretty sweet stuff with the SQL generation code in DBIx::Class and I hope it will be ready for the next release. I’ll post more when that’s released.

2 Comments

Filed under: Uncategorized

New Stuff in Data::Dumper::Concise (Devel::Dwarn)

Author: fREW Schmidt

20 Jan

I just released a new Data::Dumper::Concise. There are new features!

In Devel::Dwarn we have two new features:

Ddie

This function dies on Dwarn, which has super handy for tests and stuff.

1
2
3

Ddie {
frew => 1,
};

DwarnF

This is like Log::Contextual’s Dlog methods. So you now can do the following:

1	DwarnF { "user: $_[0]\n session: $_[1]" } $user, $session;

DumperObject

Apparently people needed this. It’s part of Data::Dumper::Concise. Basically you can call DumperObject to get the underlying Data::Dumper object.

Hopefully you guys can use this!

0 Comments

Filed under: Uncategorized

Announcing Config::ZOMG

Author: fREW Schmidt

11 Jan

For a while now I’ve wanted to tear Config::JFDI up. Since I first used it it’s always been too heavy and had too many little weird things. Well, I did that last night and it ended up getting three times faster! I’ve released the fork as Config::ZOMG (I considered GTFO and STFU, but thought better of it.)

For the most part it’s the same as Config::JFDI of course, but basically what I did was remove the substitution and install_accessor features, removed isa checks, and switched from Any::Moose to Moo with inlined defaults. One other major thing I did was took out the superfluous API bits. I see no reason for $config->load, $config->get, and $config->config to do the exact same thing, so now you just get load.

If you are unfamiliar with all of this basically what this means is that you can create a file myapp.json and myapp_local.json and it will load both of those and merge them, with local winning. Also it will work with almost any other file format out there (yaml, perl, Config::General, etc) supported but Config::Any.

Here’s how to use it:

1
2
3
4
5
6
7

use Config::ZOMG;
my $config_hash = Config::ZOMG->open( path => './myapp' );

# or if you want to keep the object around (maybe to use the reload feature)

my $config = Config::ZOMG->new( path => './myapp' );
my $config_hash = $config->load;

Anyway, if you are interested make sure to check out the docs, since there are actually more features as is, but my usage is basically as above.

I have some plans for the future that I think are exciting, like instead of just a universal config file reader, a universal writer as well. I’ll keep you all posted.

Oh and if you are interested in a tiny, unscientific benchmark, check this out:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

helena [6555] ~ $ time perl -MConfig::JFDI -E'Config::JFDI->open(path => "./dbic")'
perl -MConfig::JFDI -E'Config::JFDI->open(path => "./dbic")' 0.53s user 0.05s system 99% cpu 0.583 total

helena [6556] ~ $ time perl -MConfig::JFDI -E'Config::JFDI->open(path => "./dbic")'
perl -MConfig::JFDI -E'Config::JFDI->open(path => "./dbic")' 0.52s user 0.05s system 99% cpu 0.580 total

helena [6556] ~ $ time perl -MConfig::JFDI -E'Config::JFDI->open(path => "./dbic")'
perl -MConfig::JFDI -E'Config::JFDI->open(path => "./dbic")' 0.54s user 0.03s system 97% cpu 0.577 total

helena [6556] ~ $ time perl -MConfig::JFDI -E'Config::JFDI->open(path => "./dbic")'
perl -MConfig::JFDI -E'Config::JFDI->open(path => "./dbic")' 0.51s user 0.06s system 98% cpu 0.575 total

helena [6556] ~ $ time perl -MConfig::ZOMG -E'Config::ZOMG->open(path => "./dbic");'
perl -MConfig::ZOMG -E'Config::ZOMG->open(path => "./dbic");' 0.16s user 0.02s system 98% cpu 0.187 total

helena [6557] ~ $ time perl -MConfig::ZOMG -E'Config::ZOMG->open(path => "./dbic");'
perl -MConfig::ZOMG -E'Config::ZOMG->open(path => "./dbic");' 0.18s user 0.01s system 95% cpu 0.201 total

helena [6557] ~ $ time perl -MConfig::ZOMG -E'Config::ZOMG->open(path => "./dbic");'
perl -MConfig::ZOMG -E'Config::ZOMG->open(path => "./dbic");' 0.16s user 0.02s system 96% cpu 0.183 total

helena [6557] ~ $ time perl -MConfig::ZOMG -E'Config::ZOMG->open(path => "./dbic");'
perl -MConfig::ZOMG -E'Config::ZOMG->open(path => "./dbic");' 0.15s user 0.03s system 97% cpu 0.180 total

and the file (dbic.json) looks like:

1
2
3
4
5
6
7
8
9
10

{
"profile":"console",
"log_sprintf": {
"caller_depth":2,
"caller_clan":"^Try::Tiny|^DBIx::Class|^Log::Sprintf",
"format": "%l%n%m%n",
},
"no_repeats":1,
"placeholder_surround":["\u001b[30;46m","\u001b[0m"]
}

2 Comments

Filed under: Uncategorized