Archive for March, 2012

The Rise and Fall of Event Loops (in one very small place of my code)

Author: fREW Schmidt

6 Mar

In the spirit of one of my other posts I’ve decided to chronicle my path with at least a couple event loops.

More than eighteen months ago I documented my decision to start using an event loop as it would handle things I may not have considered, the example mentioned specifically in that post being exceptions. Things went well! I used the code I documented in that post for a long time with no issues until recently. It turns out that the event loop I was using didn’t actually handle exceptions at all, thus completely nullifying my reason to use it.

So I looked elsewhere. I looked at the grandfather of event loops, POE. I like a lot of the components that have been written on top of POE, but POE itself is frustratingly low level. That’s a topic for another post though (yes I looked at Reflex.)

After my last post and speaking with Rocco Caputo, auther of our venerable POE, I came up with the following runner role:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

package Lynx::SMS::DoesRun;

use Moose::Role;
use POE;

# this merely uses our logger etc
with 'Lynx::SMS::HandlesDieForPOE';

requires 'single_run';

has period => (
is => 'ro',
required => 1,
);

has schema => (
is => 'ro',
);

sub run {
my $self = shift;

POE::Session->create(
inline_states => {
_start => sub {
$_[KERNEL]->sig( DIE => 'sig_DIE' );
$_[KERNEL]->yield('loop');
},
sig_DIE => \&die_handler,
loop => sub {
$_[KERNEL]->delay( loop => $self->period );
$self->single_run;
},
},
);

POE::Kernel->run;
}

no Moose::Role;

1;

This works fine. It’s (to me) a little ugly, but I imagine that I’d get used to it if I were to write much more POE. But then Rocco pointed out that maybe I’m just wasting my time with event loops for this use case. Ultimately using POE as a glorified Try::Tiny is stupid and really not even the goal. So finally I’ve ended up just a few steps beyond where I started:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

package Lynx::SMS::DoesRun;

use Moose::Role;
use Try::Tiny;
use Log::Contextual qw(:log :dlog);

requires 'single_run';

has period => (
is => 'ro',
required => 1,
);

has schema => (
is => 'ro',
);

sub run {
my $self = shift;

while (1) {
try {
$self->single_run;
} catch {
my $error = $_;
log_error { $error }
};
sleep($self->period)
}
}

no Moose::Role;

1;

The observant reader will notice that despite me mentioning the above use case, which is really the only important one for me given that our actual server will run all of our services in separate processes, there is still the benefit of Event Loops mentioned in the first post for development purposes (starting all services in a single program.) I have indeed converted that to POE, but that probably doesn’t matter. I run my unified service script maybe once or twice a year at this point. Here it is if anyone is interested:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

package Lynx::SMS::Runner;

use Moose;
use POE;

with 'Lynx::SMS::HandlesDieForPOE';

has tasks => (
is => 'ro',
default => sub { [] },
);

sub run {
my $self = shift;

POE::Session->create(
inline_states => {
_start => sub {
$_[KERNEL]->sig( DIE => 'sig_DIE' );
$self->create_children_sessions,
},
sig_DIE => \&die_handler,
},
);

POE::Kernel->run;
}

sub create_children_sessions {
my $self = shift;
my $x = 0;
my @tasks = @{$self->tasks};
for my $task (@tasks) {
POE::Session->create(
inline_states => {
_start => sub {
$_[KERNEL]->delay(loop => ($x++ / @tasks ));
},
loop => sub {
$_[KERNEL]->delay( loop => $task->period );
$task->single_run;
},
},
);
}
}

no Moose;

__PACKAGE__->meta->make_immutable;

1;

I look forward to using POE for actual heavy-lifting in another one of our projects, and will post about the experience when I get there.

0 Comments

Filed under: perl

Perl Event Loop

Author: fREW Schmidt

4 Mar

I have some extremely basic code using AnyEvent but I recently found out that I was doing it wrong. That is, the entire reason I am using an event loop is to catch errors, log them, and keep going. That’s one of the great benefits that Catalyst gives me; I override one thing and I get universal error logging. The problem is that AnyEvent specifically does not handle this use case.

I have a working solution, but as I am planning on rewriting our services in evented code this prohibition makes me really worried. The problem is that you can’t just know your code won’t die. Exceptions happen and as a developer of a language that’s not Java or C# I don’t know where they come from. My current solution is ok, but I don’t think it’s really viable long term. Here’s my current code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#!/usr/bin/env perl

use strict;
use warnings;

use AnyEvent;
use Try::Tiny;

sub event {
print "looped\n";
die "lol" if rand() < .5;
}

sub NEVER_DIE {
my $code = shift;
return sub {
try \&$code, catch { warn $_ } # <-- this should be logging, you get the idea
}
}

my $cv = AE::cv;
my $w = AE::timer 0, 1, NEVER_DIE(\&event);
$cv->recv;

This works for simple cases, but if I chose to go down this route in the long term I’d have to wrap every single code ref in NEVER_DIE, which is pretty lame.

I looked at POE as it may support my use case better but as far as I can tell it’s support is WORSE. Here’s what I came up with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

#!/usr/bin/perl

use strict;
use warnings;

use POE;
use Try::Tiny;

sub handler_start {
my ($kernel, $heap, $session) = @_[KERNEL, HEAP, SESSION];
$kernel->yield('event');
}

sub NEVER_DIE {
my $code = shift;
return sub {
try \&$code, catch { warn $_ } # <-- this should be logging, you get the idea
}
}

sub event {
print "looped\n";
die "lol" if rand() < .5;
$_[KERNEL]->delay_add('event', 1);
}

POE::Session->create(
inline_states => {
_start => \&handler_start,
event => NEVER_DIE(\&event),
_stop => sub{},
}
);

POE::Kernel->run();
exit;

So I still have to use NEVER_DIE, so that’s a lose, and worse, if event dies before the call to delay_add we end anyway. Sure, I could put delay_add at the beginning of event, but that brings me to another thing that really bothers me about the “POE Way” (my own terminology, I may just not be getting it), for my AnyEvent code I can add a bunch of things and they don’t have to know about each other. The loop handles calling the events. With POE it seems like I have to manually tell it “call this, now call this.” That seems to defeat the entire purpose! What am I missing here?

If anyone knows an event loop I should consider (MUST RUN WELL ON WINDOWS) or maybe some setting in POE and some kind of POE timer thing, or some way of safely overriding how AE calls it’s events, please, comment and let me know.

3 Comments

Filed under: perl