fREWdiculous!
3 Sep
How do you set instance variables from a constructor method?
The fundamental issue here is that often validation is bypassed at construction time, for whatever reason. So one’s accessor may look something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
Clearly this method is just doing to much. To solve this we make special set methods that are entirely to be used during construction. So in Perl this might look like the following:
1 2 3 4 | sub _set_x { my ($self, $x) = @_; $self->{x} = $x; } |
Interestingly, with Moose we happily side-step this issue, as the default constructor doesn’t go through the accessors and already sets the raw values.
Ok, so I think I may start trying to apply this stuff to JavaScript instead of Perl. I almost feel like the fact that I have Moose in Perl is cheating. I know that there is Joose in JavaScript, but I’ve yet to use that in production, and I find that I have a harder time making well factored code in JavaScript than Perl. Part of that is that the underlying libraries I use in JS (ExtJS 3) are not really well factored either, but I still struggle with overall structure.
24 Jun
A coworker sent this to our internal mailing list, partially to goad me into responding to the stupid comments. I don’t have an ars account, so I’ll just hope that trackbacks work.
Here are the arguments that someone actually put effort into making:
a) Magic ‘this’. This is this, except when this is that. JavaScript pushes you to use anonymous functions all over the place, except they always end up losing the proper context for the ‘this’ variable, so you end up having goofy code like “var _this = this” all over the place and then using that inside your callbacks or other functions. Some days I swear that the number of functions I manage to write that don’t use a renamed ‘this’ are actually smaller than the number that do.
If you think it’s magic you clearly don’t understand it. That’s not really an argument. I agree that sometimes it’s confusing, but sometimes programming is hard. Sorry?
b) 1 + “1″ – 1 = 10. Also, “1″ + 0 = “10″. Yes, this actually has caused bugs for our applications, where data that’s expected to be a number was loaded from a JSON file as a string due to a bug in another application, and the result was not good. All our loading code had to be updated to add a ton of type conversions all over the place. When I need something to be a number I really freaking absolutely want it to be a number, not a string or an object or null or anything else. Lua, which is very similar to JavaScript in most respects, fixed this problem by just not being retarded enough to use the same operator for addition and string concatenation.
I agree, this sucks, but I hit it rarely enough that it doesn’t discount the language.
c) Global by default variables. So even if you take the argument that dynamic typing is just “easier” because you don’t have to think about variable declarations, JavaScript throws that argument out the window by making you put ‘var’ in front of new identifiers all over the place. And then it silently screws you if you forget to.
Again, I disagree with this, but maybe because we use Perl and Perl works the same way (with the advantage of use strict, which some JavaScript implementations recently added but doesn’t really handle that issue as far as I can tell.)
d) Prototypes instead of classes. There are very few large-scale real-world JavaScript applications in existence that don’t plug in their own class system to work around the inherent uselessness of prototypes in large application architecture. Those same apps make minimal use of prototypes to extend the base JavaScript types, and only because JS was so poorly designed that even the two interesting built-in types it comes with are lacking half the features you’d expect them to have.
Wow I can’t believe how much I disagree with this. Prototypes give you more flexibility, so sure, they have caveats, but I totally think it’s worth it. Like a lot of things, you need to exercise discipline when using them, but I can think of numerous times in our real world, medium sized app (14K lines of JavaScript) where it’s helped to have.
e) Inability to create pass-by-value types. This is a frequent problem in just about every language aside from C++/D, actually. For those using JavaScript to write WebGL apps, take a look at all the linear algebra libraries for JavaScript. In 3D apps, you almost use vectors more often than you do scalars. Imagine if every integer in your app was passed by reference, so that “a = 1; b = a; b++” made both a and b equal to 2. Every little three component vector is a complete full object. They are passed by reference (the source of almost a half of the bugs in our WebGL game so far, in fact). They exist in great quantity, are heap-allocated, and are garbage-collected, which puts an intense amount of pressure on the GC which can and does result in GC pauses in even simple WebGL games, unless the developer jump through ridiculously complicated hoops to avoid creating new vectors in all the places where it’s logical to create new vectors. You can’t have operator overloading, so you have very large and ugly expressions to do basic operations. Accessing individual components is slow. The objects aren’t natively packed and hence are incredibly slow to push into a vertex buffer, unless you implement them as a Float32Array instances, which confuses the crap out of the optimizers of both V8 and SpiderMonkey currently. Did I mention they’re passed by reference?
I can’t relate with this. I just have never needed to pass anything other than a scalar by value, and those are pass by value. I can see that it’s a real issue for games and whatnot though. Also realize this argues against the poster’s language of choice.
f) No built-in include or require functionality. Seriously, still. Third-party libraries exist but almost all of them have some kind of bug or another, not least of which is a confusing caching problem in at least Chrome making doing actual development a pain in the butt.
Using a third party library should be expected. JavaScript isn’t Python, it’s not made to be “batteries included.” If there are bugs in a library you use you should fix those bugs.
g) Dynamic typing. Yes, I’m willing to start that argument. You start noticing it the most the second you stop writing little Web apps or Web pages and start writing large apps where you actually have data that persists for longer than a single mouse click or request/response cycle: add the wrong kind of object to an array to process later and get a crash later from a missing method or member in a completely different bit of code than where the actual mistake was. Fun times. Yes, Java makes static typing seem evil. No, Java/C#/C++ are not the one and only way to do static typing. Type inference, implicit interface binding, etc. give you all the “easy to deal with and not a lot of keystrokes” advantages of dynamic typing without all the bugs. The second most popular Web language — ActionScript 3 — is statically typed, in fact, despite otherwise being identical to JS/ECMAScript. As an aside, I get more crashes from the Python apps on my Fedora desktop than I do from the C/C++ apps (actually, none of the C/C++ apps on my desktop crash, now that I think about it). Missing member exceptions == so much easier to develop and maintain apps, right?
I don’t think I’ve had a type issue in our “large [app] where [we] actually have data that persists for longer than a single mouse click” for longer than a typo.
h) Speed. Yes, there has been some ridiculously immense amounts of effort by a large number of super bad-ass developers put into the language runtimes to make JS almost half as fast as a low-grade C compiler that a single college Junior could write in a few months. And LuaJIT is in the same boat as JS in terms of fundamental language limitations but manages to do better than every JavaScript implementation anyway. People who don’t understand what all the JS optimizations in V8 or such actually _do_ like to claim the JS can do amazing things speed-wise, but the reality is that all those optimizations are basically just “try very very hard to analyze the code to figure out types for variables and then compile it like a slightly retarded statically-typed language’s compiler would do it.” Oh, and there’s tracing, but then tracing also works on statically typed languages (and works better due to a lack of need for type guards in the generated machine code). Not a single one of those whizbang optimizations was invented by or for JS, in fact; most were taken from research JVMs (Java is evil!) or classical OOP languages (prototypes are awesome!).
Nice! So the poster says: “JS is fast, but not as fast as C and the things that made it fast were taken from other VM’s so it doesn’t really count.” Ok Mr. C#, I’m sure you’re correct since C# is entirely original research
Also, is C# as fast as C?
i) No IntelliSense even possible. Want to see what methods exist on that variable you’ve got there on line 187 of foo.js in your text editor? Too bad. Go trace through the code until you figure out where it was initialized, then go trace through the code to find out what its prototype has on it. And then hope there’s no code dynamically changing the prototype behind your back. In fact, just run it in a browser and set breakpoints, because finding out anything useful about the value any other way is basically impossible for any codebase larger than the toy_web_app.html sites that JavaScript apologists use to glorify the ease and simplicity of JavaScript. Some code editors try _really_ hard to do better, and almost kinda sorta succeed for the really simple cases, sometimes, once.
Yeah, that’s a real thing I guess, but I’m not the target user of IntelliSense anyway.
j) No advantage. JavaScript isn’t even special compared to other dynamically typed language. It is not capable of doing anything interesting at all all that can’t also be done by Lua, Python, Ruby, etc. None of the JS implementations are any faster than LuaJIT or PyPy or various other advanced JIT-ing implementations of other dynamic languages. JS has no plus sides to it compared to other commonly available languages. Oh, except it runs natively in a Web browser without a plugin. Which is the only reason in the world why it’s so popular. In fact, it’s the only reason it event _exists_. If someone 10 years ago had just thought, “heck, let’s drop an existing well-designed and well-established language into our browser and get the other guys to do the same instead of making everyone use this goofy little hackjob that NetScape came up with,” the Web would look a lot different (better) today. Just imagine the future if Chrome dropped Python into Chrome as a supported language. Or actually, imagine this: Google drops C/C++ into Chrome as a supported language (http://code.google.com/p/nativeclient/).
The fact that it’s the most widely deployed VM in the world isn’t an advantage? The fact that there are three major VM’s leading to healthy competition and improvements isn’t an advantage? The fact that it’s in all browsers right now (unlike some crazy future where browsers need to implement C or (good lord!) C++ safely) isn’t a feature?
Maybe I’m just a tool for thinking this guy just has confirmation bias. I’ve done the same thing of course in arguing against C#, so maybe I’m just as bad, but I can hardly take these arguments as reasons not to use JavaScript.
29 Jan
I write a lot of ExtJS grids at work. I have written JavaScript classes for our Ext grids that generate as much as possible automatically, but the actual column definitions of the grids are almost always unique. The project I am on now is nearing our first real deploy, and we’re late, so things have been really, really busy.
It wasn’t until recently that I realized just how much time I spent working on grids and their related records (representation of the rows of a grid.) Although even if I’d known just how much I do this at the beginning of the project, I certainly didn’t know DBIx::Class as much as I do now, in addition to the other 4 non-core modules that I’ll mention.
Because I’ve been working all these super long hours (12-14 a day), often after a frustrating day I’ll try to code something more relaxing and rewarding at home or at work but not during work. So today I decided to finally take the plunge and do what I’ve been pondering for a few months and write some code to generate Ext scaffolding for me. So I’m going to walk you through the script that I wrote (and will probably work on more as time goes by.)
This isn’t a full module, not even in my repo yet, so it’s all just in one file. Here’s the boilerplate intro:
1 2 3 4 5 6 7 8 9 10 11 12 | #!perl use strict; use warnings; use feature ':5.10'; use Syntax::Keyword::Gather; use String::CamelCase (); use Lingua::EN::Inflect (); use List::Util; use Statistics::Basic 'mean'; |
That’s pretty basic. We use a bunch of modules that I’ve used in at least 1 other project before and was happy with.
1 2 3 4 5 6 7 8 | use FindBin; use lib "$FindBin::Bin/../local/lib", "$FindBin::Bin/../lib"; use ACD::Schema; use Config::JFDI; my $config = Config::JFDI->new(name => 'acd', path => "$FindBin::Bin/../acd"); my $config_hash = $config->get; my $connect_info = $config_hash->{Model}{DB}{connect_info}; my $schema = ACD::Schema->connect($connect_info); |
That’s the code to parse any kind of Catalyst config file and the grab a new schema based on it. It’s a ton of biolerplate but I live with it.
So next up is a basic inflection function that gives us all the different forms of a word we might need. It starts with either “single_foo” or “SingleBar” and gives us six variations based on that:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | sub inflect { my $word = shift; my $return = { singular => {}, plural => {}}; if ( defined $word->{camel}) { $word = $word->{camel}; $return->{singular}{camel} = $word; $return->{singular}{noncamel} = String::CamelCase::decamelize($word); } else { $word = $word->{noncamel}; $return->{singular}{camel} = String::CamelCase::camelize($word), $return->{singular}{noncamel} = $word; } $return->{singular}{human} = join( q{ }, map ucfirst $_, split /_/, $return->{singular}{noncamel} ); $return->{plural}{noncamel} = join( q{_}, split / /, Lingua::EN::Inflect::PL( join q{ }, split /_/, $return->{singular}{noncamel} ) ); $return->{plural}{camel} = String::CamelCase::camelize( $return->{plural}{noncamel} ); $return->{plural}{human} = join( q{ }, map ucfirst $_, split /_/, $return->{plural}{noncamel} ); return $return } |
So that uses String::CamelCase and Lingua::EN::Inflect combined with join and split mostly.
We’re *almost* ready to generate a record. But first we need to define a mapping from the data type in the database to the data type that Ext uses:
1 2 3 4 5 6 7 8 | my $types_xform = { int => 'int', float => 'float', varchar => 'string', bit => 'boolean', datetime => 'date', money => 'float', }; |
Ok, now let’s look at the code to generate an Ext.record:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | sub generate_record { my $schema = shift; my $source_name = shift; my $source = $schema->source($source_name); return qq{Ext.ns('ACDRI.record'); ACDRI.record.$source_name = Ext.data.Record.create([\n} . join(qq{,\n}, sort { $a cmp $b } gather { for (map [$_, $source->column_info($_)], $source->columns) { my ($column, $info) = @{$_}; take " {name: '$column', type: '$types_xform->{$info->{data_type}}'}"; } }) . "\n]);"; } |
So what’s going on here is that we get the source from the schema. The source could be considered something like a table definition, although it can also point at a view or whatever too. Then we start generating the string representing our record, and then we use a join/gather combo to get the column data the way we want it.
We could certainly just use a more complex map instead of the for+gather that we have, but I personally feel than any map where you *must* use the block form is cumbersome. So we join together all of the strings that gather took, and then append the end of the definition, and voilà, we have a record!
Here’s example output on our Customer source:
1 2 3 4 5 6 7 8 9 10 11 | Ext.ns('ACDRI.record'); ACDRI.record.Customer = Ext.data.Record.create([ {name: 'color_code', type: 'string'}, {name: 'comments', type: 'string'}, {name: 'id', type: 'string'}, {name: 'last_edited_date', type: 'date'}, {name: 'name', type: 'string'}, {name: 'price_approval_required', type: 'boolean'}, {name: 'sales_representative_id', type: 'int'} ]); |
If you’re still following along you should be fine with the next, much more impressive functionality.
I’d like to have the script do as much as possible to speed up my work, so let’s see how far we can take this. First, I made a mapping from type to renderer, so that things that are datetimes use our custom datetime renderer, same with booleans:
1 2 3 4 | my $renderer_from_type = { datetime => 'ACDRI.fn.Renderers.dateTime', bit => 'ACDRI.fn.Renderers.bool', }; |
Next, I wrote a function that would try it’s best to guess how wide a column should be based on it’s header and the average width of the strings inside of it:
1 2 3 4 5 6 7 |
Now finally we have the grid function:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | sub generate_grid { my $schema = shift; my $source_name = shift; my $source_names = inflect({ camel => $source_name }); my $source = $schema->source($source_name); return qq`/*global Ext */ /*global ACDRI */ /*global MTSI */ Ext.ns('ACDRI.ui.grid'); /** * \@class ACDRI.ui.grid.$source_names->{plural}{camel} * \@extends MTSI.ui.Grid */ ACDRI.ui.grid.$source_names->{plural}{camel} = Ext.extend(MTSI.ui.Grid, { title: '$source_names->{plural}{human}', record: ACDRI.record.$source_names->{singular}{camel}, updateConfig: { xtype: 'ACDRI.ui.form.$source_names->{singular}{camel}' }, addConfig: { xtype: 'ACDRI.ui.form.$source_names->{singular}{camel}', }, initComponent: function () { //this.sortInfo = { // field: 'part_id', // direction: 'asc' //}; var config = { url: '$source_names->{plural}{noncamel}', itemName: '$source_names->{singular}{human}', columns: [` . join (qq{, }, gather { for (map [$_, $source->column_info($_)], $source->columns) { my ($column, $info) = @{$_}; my $colnames = inflect({ noncamel => $column }); my $renderer = $renderer_from_type->{$info->{data_type}} ? "\n renderer: $renderer_from_type->{$info->{data_type}}," : ''; my $width; given ($info->{data_type}) { when ('varchar') { $width = int 7*avg_width( $source->resultset, $column, $colnames->{singular}{human} ); } when ('datetime') { $width = List::Util::max( int 7 * length $colnames->{singular}{human}, 57 ); } default { $width = int 7 * length $colnames->{singular}{human}; } } take qq[{ header: '$colnames->{singular}{human}', dataIndex: '$column', sortable: true, hidden: false,$renderer width: $width }]; } }) . qq`] }; Ext.apply(this, Ext.apply(this.initialConfig, config)); ACDRI.ui.grid.$source_names->{plural}{camel}.superclass.initComponent.apply(this, arguments); } }); Ext.reg('ACDRI.ui.grid.$source_names->{plural}{camel}', ACDRI.ui.grid.$source_names->{plural}{camel}); `; } |
We see nearly the same structure here that we did with the record, except much more intricate. Notice the code to generate the width uses the magic constant 7. As I played with this I found that 7 seemed to work for the font that Ext uses by default (Arial?) Optimally I would actually use some kind of metrics package to ask it for the width of all of the strings that I generated in the function above and average that, instead of averaging character lengths. But this seems to work really, really well, so the ROI is pretty good.
Also note the code to pick and insert the renderer. It’s not complex or anything, but it yields very convenient results. Here’s the output of running this one on the same source we used for the record:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | /*global Ext */ /*global ACDRI */ /*global MTSI */ Ext.ns('ACDRI.ui.grid'); /** * @class ACDRI.ui.grid.Customers * @extends MTSI.ui.Grid */ ACDRI.ui.grid.Customers = Ext.extend(MTSI.ui.Grid, { title: 'Customers', record: ACDRI.record.Customer, updateConfig: { xtype: 'ACDRI.ui.form.Customer' }, addConfig: { xtype: 'ACDRI.ui.form.Customer', }, initComponent: function () { //this.sortInfo = { // field: 'part_id', // direction: 'asc' //}; var config = { url: 'customers', itemName: 'Customer', columns: [{ header: 'Comments', dataIndex: 'comments', sortable: true, hidden: false, width: 226 }, { header: 'Color Code', dataIndex: 'color_code', sortable: true, hidden: false, width: 70 }, { header: 'Id', dataIndex: 'id', sortable: true, hidden: false, width: 35 }, { header: 'Last Edited Date', dataIndex: 'last_edited_date', sortable: true, hidden: false, renderer: ACDRI.fn.Renderers.dateTime, width: 112 }, { header: 'Name', dataIndex: 'name', sortable: true, hidden: false, width: 136 }, { header: 'Price Approval Required', dataIndex: 'price_approval_required', sortable: true, hidden: false, renderer: ACDRI.fn.Renderers.bool, width: 161 }, { header: 'Sales Representative Id', dataIndex: 'sales_representative_id', sortable: true, hidden: false, width: 161 }] }; Ext.apply(this, Ext.apply(this.initialConfig, config)); ACDRI.ui.grid.Customers.superclass.initComponent.apply(this, arguments); } }); Ext.reg('ACDRI.ui.grid.Customers', ACDRI.ui.grid.Customers); |
Note that there are probably some things that we could do a little better. For instance, we should probably check for text fields and not include them in the grid. We could also be clever and look for anything ending with _id and have it hidden by default. But in general, the above is very nice results. At the very least you are probably going to need to reorder the columns (although you could give some kind of grid_order_id to the column definition in the Schema::Result code, but I don’t think that’s worth it.)
So I hope someone else enjoyed reading this as much as I did writing it. After I’ve exhausted all of the little Ext things I can use this for (form, etc) I will probably look into messing with DBICx::AutoDoc to add in Dia outputs.
25 Jan
(Sorry if you heard this already
)
At $work we do as much “view” type code as we can in JavaScript with the ExtJS framework. I have personally found it to be a great framework to work with, although often it is lacking in the non-UI department. One thing that at first I really liked about Ext was their Template and XTemplate classes. But as time went on I got more and more annoyed with those modules.
I’ve always thought that Template-Toolkit was a really nice templating library to work with. I hate templating html because of all the weird little gotchas having to do with CSS and whatnot, but doing almost all that kind of work in JavaScript I have started missing Template-Toolkit. At some point I heard about Template::Tiny and this past Saturday I thought, “that’s like, 160 LOC…I could probably port that to javascript!”
So I did! It’s not really done yet, but the current code is at github. I need to finish porting the test suite from Perl 5 to JavaScript so that I can ensure correctness (I am certain I screwed up some stuff.)
Alias mentioned in his post that I use XRegExp (600 LOC) to help out with the Regular Expression support in JavaScript for this module. I actually wasn’t going to, since the only thing I needed it for was the /x flag, or to be more clear telling the parser to ignore whitespace, but I want to keep Template.Tiny in sync with the Perl version, and I really don’t want to strip out all the whitespace by hand. If someone takes issue with the dep they can fork away
So once it is fully ported I fully intend to use it entirely from now on instead of the Ext templating. But take note, Ext templates and Template.Tinies (what?) really solve different problems and have different goals. The following is a list of Pros and Cons of each:
Pros:
Cons:
Example 1:
1 2 3 4 5 6 7 8 9 10 11 12 | var data = { age: 21 }; // XTPL var tpl = new Ext.XTemplate( "<tpl if='age > 18'>Can Vote!</tpl>"); var str = tpl.apply(data); // TT var tmp = new Template.Tiny(); var str = tmp.process( "[% IF old_enough %]Can Vote![% END %]", { old_enough: (age > 18 )} ); |
Example 2:
1 2 3 4 5 6 7 8 9 10 11 | var data = { person: {first_name: 'fREW', last_name: 'Schmidt', title: 'Mr.' }}; // XTPL var id = new Ext.XTemplate( "<tpl for='person'>{title} {first_name} {last_name}</tpl>"); var str = tpl.apply(data); // TT var tmp = new Template.Tiny(); var str = tmp.process( "[% person.title %] [% person.first_name %] [% person.last_name %]", data ); |
Example 3:
1 2 3 4 5 6 7 8 9 10 11 12 | var data = { werld: 'fail'}; // XTPL var tpl = new Ext.XTemplate( "Hello {world}"); var str = tpl.apply(data); // error message // TT var tmp = new Template.Tiny(); var str = tmp.process( "Hello [% world %]", data ); // silent failure |
Example 4:
1 2 3 4 5 6 7 8 9 10 11 12 | var data = { foo: 1 }; // XTPL var id = new Ext.XTemplate( "{[someComplexRenderer(values.foo)]"); var str = tpl.apply({ foo: 1}); // TT var tmp = new Template.Tiny(); var str = tmp.process( "[% foo %]", { foo: someComplexRenderer(data.foo) } ); |
Example 5:
1 2 3 4 5 6 7 8 9 10 11 12 13 | var data = { arr: ['foo','bar','baz']}; // XTPL var tpl = new Ext.XTemplate('<tpl for="arr">({#}. {.})</tpl>'); var str = tpl.apply(data); // "(1. foo)(2. bar)(3. baz)"; // TT var tmp = new Template.Tiny(); var idx = 0; var str = tmp.process( "[% FOREACH x IN arr %]([% x.i %]. [% x.var %][% END %]", { arr: data.arr.map(function(x) { return { i: idx++, var: x } } } ); |
Example 6:
1 2 3 4 5 6 7 8 9 10 11 12 | var data = { longtext: "this won't fit!"}; // XTPL var tpl = new Ext.XTemplate('{longtext:ellipsis(5)}'); var str = tpl.apply(data); // "this ..."; // TT var tmp = new Template.Tiny(); var str = tmp.process( "[% longtext %]", { longtext: Ext.util.Format( data.longtext, 5) } ); |
Pros:
Cons:
Example 7:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | var data = { jack_slocum: 1 }; // TT var tmp = new Template.Tiny(); var str = tmp.process( "[% IF jack_slocum %]We don't appreciate else-ifs[% ELSE %] Woot![% END %]", data ); // XTPL var tpl = new Ext.XTemplate( '<tpl if="jack_slocum">We don\'t appreciate else-ifs</tpl><tpl if="!jack_slocum">Woot!</tpl>' ); var str = tpl.apply(data); |
I am planning on making a wrapper for TT for our stuff that will allow an anonymous function that will do data transformation like above. But as you can see above XTemplate really has more to offer, it just annoys the heck out of me on a regular basis
18 Aug
The project I am working on at work is going to be deployed soon, so today I worked on some of those things that need to be taken care of before the deploy. One of those things was changing our gigantic list of javascript files into a single file with minimal hassle. I actually tried to implement it myself, but that was silly. A simple search on CPAN for catalyst javascript yields two promising results.
The first is Catalyst::View::JavaScript. While this is an excellent package which does caching and automatically minifies depending on whether the server is in debug mode or not, it expects javascript to be in memory already. Maybe that’s something that’s common and I don’t know about it, but we almost never write javascript on the server.
So onto the next result: Catalyst::View::JavaScript. It does not do caching, and it always minifies. But heck! That’s not too bad. Before installing it, note my RT that it does not have correct dependencies. Hopefully that will be fixed soon. I also sent another RT. Since I technically write more lines of javascript than perl, I need to be able to turn off minification. So I patched the module to only minify when the server is not in debug mode. Hopefully that will be accepted as well.
So after doing that I ensured that apache had gzip turned on to reduce the amount of bandwidth required by the clients to download our javascript. Then I got curious. How much of a difference do these things make? So I measured it:
| no gzip | gzip | |
| no minify | 807K | 212K |
| minify | 712K | 202K |
Wow! Minification really doesn’t help that much, whereas gzip’ing is huge.
Now, just to be clear, this isn’t a great solution for an externally facing site, because the clients don’t currently cache the minified javascript, because it gets served by Catalyst and it’s probably not worth my time to set that up all the headers to fix that for a site that runs mostly on a LAN. Once the customer does start allowing external access we’ll want to set some last-modified headers and whatnot.
I also did a few more cool tweaks using debug mode. For example I have the debug version use ext-all-debug.js, which has extra stuff for error messages and whatnot, and ext-all.js, which is preminified and has error checking removed for performance reasons (for the tests above I used ext-all.js the whole time.)
Furthermore I have the debug switch my server side error messages from actual exceptions to a simple message asking the user to get in touch with the devs and tell us what happened and what time it was when it happened.
All in all I felt really good about the code I wrote today. It will help me a lot as time goes on and it makes the dev server way faster (1 small download is way faster that 60~ really small ones). Too bad the customer will probably not notice a significant amount of it
Anyway, hopefully these tips will help give you some ideas for your webapps.
23 Jul
So some of you may have heard about RESTful interfaces. What I am going to describe here is my vague interpretation of REST through a web developers glasses, with respect to Catalyst and ExtJS. But first some background.
I am working on a relatively new project at work (6 months as opposed to 10+ years) and I’ve been striving to use the best tools for the job through and through. I was initially going to try for Rails, but fortunately my boss curbed that interest by saying no, (subtly!) So instead of using Rails I looked at Catalyst briefly and I already knew CGIApp from years before. Catalyst seemed like a no-go as there had previously been issues with it and windows at our work, so CGIApp seemed like the best route to take. It was still light years better than a giant bag of non-persistent CGI scripts with extremely little code reuse.
Next up is the front-end design. Say what you will about requiring users to use Javascript; if it’s an internal website, I see no reason not to do a pure Javascript site as long as you can at least try to pull off 508/ARIA. And if a website is pure Javascript, with just enough html for a single div and a header, it is probably using ExtJS. There may be other ways to pull it off, but Ext really is pretty nice to use. I very rarely have browser compatibility issues with Ext, whereas with prototype it was almost bare metal. So with Ext, instead of previously, where we had a giant bag of standalone scripts, we have been doing an inheritance based (and prototypical, duh) OO interface. It’s been excellent for UI design and code reuse. I think that some of the things we have done with Ext rival UI design I could do (as a mere programmer who’s read a couple books about UI design) with any other framework, and that includes flash and non-web GUI’s. It’s very elegant due to Javascript’s design (events, OO out of the box, etc).
Now that we have theory out of the way, lets talk more concrete issues. For Ext you have a Grid class. You can think of a Grid as basically somewhere between a table and a spreadsheet, plus lots of other sexy goodies. To set up an Ext Grid you need a store and a column model. Really the meat of the Grid is in the column model, so I made a subclass of the grid that generates the store automatically based on an action and a controller. This is really helpful because basically all of your dispatching is done in one place. That means if I change my server side dispatching I only have to change one thing in the javascript (and I have done this.)
I also add a bunch of CRUD-y buttons to the top toolbar of the grid. And of course, because they are generated automatically I generate the urls for Creating, Reading, Updating, and Deleting these records automatically. I define controller and action (currently this compiles to /controller/action, it used to be /Controller_controller/action…whatever) and I define server side actions for create_action, single_action, update_action, and delete_action. It works fairly well, except when I’m dumb and call action something like single_report. Then I end up with reasonable things like, delete_single_report…and silly things like single_single_report. Yeah that happened.
Recently I was working on a really basic Grid. The user won’t even have windows for the rows as usual, and will instead edit the Grid directly. One of my coworkers has been working on something like this lately anyway, so I asked him how he did it and went at it. It involved a REST based Ext store, a per-row editor for the grid, and the server has to support REST somehow. So far so good.
I implemented my Ext class by copying my original Ext subclass. I didn’t subclass it because this is slated to replace it. I added a temporary dispatching feature to it so that instead of dispatching to /controller/action it will dispatch to /controllerREST/action. That way I will slowly migrate things from my ghetto dispatching to REST and then when I want to switch everything at once I just move the controllers and change the dispatching in my one subclass. Easy peasy.
For the Catalyst side (and I specifically waited till I finished the Catalyst switch for this) we use Catalyst::Controller::REST. At first this is a small hassle to set it. It’s (seemingly) an all or nothing setup, so instead of making single actions RESTful (possible but not as nice) you just make RESTful controllers. The Action class takes care of the basics of RESTful dispatching. That is, PUT’s get routed to /controller/action_PUT. The controller actually does some really sexy stuff. It will look at the headers sent from the browser (Accept) to see what format the message sent across the wire was, automatically decode it, and put it in $c->request->data. You do whatever you want based on that and then you put data in (manually or using a helper function) $c->stash->{rest}, and it again uses the Accept header to decide how to format the data.
Now you may have missed it, but this makes it stupid easy to have your server support JSON, YAML, XML, and a bunch of other serializers baked in for free! Well, as long as you can set the Accept header. (I set it with a global override in Ext.) But more importantly for me it makes it quite clear what my interface is, and a simple way of automating the usage of that interface. Instead of adding single or update to some url, I just do a PUT for creation, or a DELETE for deletion. It’s simplified our server side and client side code significantly!
Also note: using REST you can simplify a lot of other things, like instead of everything being a text blob, you can have integers, booleans, and other complex data structures. This means that I can have non sucky checkboxes. This is a big win.
Now that I’ve finished writing and editing this blog post it is a whopping 1059 words long, so I should probably stop there. But I may blog more about this REST stuff in the future. What about you? Do you use a RESTful interface? Do you hate REST? Why?
16 Apr
The end! Ok not quite. So this was the last day of the conference. It was shorter than the other days and most of us had to checkout anyway. Still exciting!
First off we got an awesome demo of the Designer. It looks like it will be extremely useful for exploring the framework and playing with layouts. You can edit multiple components at once, as if it were an IDE. You can even load data into grids on the designer. The bad news is that it looks like the Designer will be a service or an Air app that you buy. I understand the motivation behind that, but it’s still frustrating.
The next talk I went to was the Application Deployment talk. Very good information, but little of it was new to me. First off, Yahoo has written a lot of best practices for website performance. Take a look at those. YSlow is based on those and can tell you if you are following some of those best practices. YSlow 2.0 is in development and will be made of unicorn’s and rainbows. Typically you should skip your overall score and just look at the specific items. Gzip is good to use. Putting versions on your filenames can really help with performance (more on this later.) CSS should be in the header and as much JS as possible should be in the body. This will allow you to load basic JS and render the page before all of the JS is loaded. Minify your JS. ETags probably won’t actually help you unless you are huge.
Firebug network monitor is your friend. Fiddler (.NET app) can help for some esoteric things like file uploads or flash interaction that won’t show up in firebug. JSLint is an amazing tool that can help you write better code. Some of the people there wrote a Yahoo Widget that will automatically run JSLint on any files that have been changed. Very cool!
You should use parseInt(foo, 10) instead of just parseInt(foo). Why? Because parseInt defaults to octal.
JS Builder is a .NET app written by the ext people that will compress your javascript
files nicely. It can also be run from the CLI for batch usage.
CSS Sprite generation can significantly reduce your server pings. Also it can reduce images a lot due to boilerplate png stuff in icons (30%!) Use this to automate the process.
You need a favicon because browsers will always check for it. Be careful about giving it a big expire time because you cannot always just rename it as older browsers only check for favicon.ico.
And that was basically it! I didn’t go to the next session because we were way over time and I had to check out. In general I really enjoyed the conference and think I learned a lot from it. The only major problem I had was the Ritz stuff I mentioned before. Other things they did were really cool. For example: to ask questions in the panel you posted them online and you could vote them up and then they would answer the ones with the highest votes. This was good because people who talk really slow had less of a chance to slow us down etc. I think they should have given the field a max length, as too many words are just confusing anyway.
I give the conference 4.5/5 stars. Great in almost every way
16 Apr
Ok, the next session I went to on Day 2 was the session on Refactoring. Refactoring is one of the few high quality buzzwords that I hear regularly, so I was excited to hear what the talk would go over. It was very much Ext specific, but the final changes to the component that we “Extified” were amazing.
First off, what does it mean to Extify a component? The comp needs to fit into the Component Model, which is mostly a lifecycle issue. Typically this will be extending an existing component or creating a new one by inheriting from Component. Refactoring also involves clean code, consistent code, and hardest of all, documented code. And part of that which is fairly Ext specific is that the configuration should be elegant; or the config needs to be predictable and not have too many options.
Unfortunately this session was like drinking from a firehose, so I couldn’t take notes more than that. For the most part though it involved simple changes like moving view changes out of the component entirely and into CSS. Documentation should be done with ext-doc, which I look forward to setting up at work. Config option names need to be thought through carefully so that they match well with the rest of the framework. Other nice things that were done is to allow a store to be defined as a store, a store config, an array, or a store id. Keep your eyes peeled for the notes from this presentation. Should be excellent.
The next session I went to was another usability session. I am of the opinion that it’s hard to have too many of these. This session was a lot more hands-on, specific ideas than the other session.
His first recommendation was to show that background events are occurring. First go look at Forever21. Make your browser window 1024×768. Go to a product and add it to your cart. The part of the page that shows that the item has been added to the cart is not where you clicked, and probably isn’t even in view. Did anything happen? Is the page just going slow? Nope. You added it every time you clicked.
Now, on the other hand, check out this Ext sample. Note how when you use the menus messages pop up fairly unobtrusively. Also note that the time the messages are displayed can be increased and they can be set up to go away when the user clicks them. Much nicer!
In that same vein, try to avoid modal browser feedback as it freezes not just your whole app, but often the whole browser. Also avoid feedback about trivial events that the user does not care about. If at all possible, make actions easy to undo. For the actions that cannot be undone, warn your user.
Speed matters for all applications. There are some easy ways to make ext applications very snappy. Use xtypes possible as they allow for lazy instantiation on rendering. When waiting for the server, a basic load mask should be used for .5s or more. If it takes 5s or more, use a spinner or some other kind of progress indicator. If it takes more than 10s a real progress bar should be used. Note that real progress bars can still be faked by precalculating how long certain sets of data take.
The help tool for the title of a panel can be used for easy context sensitive help. In general the user shouldn’t need help, but it doesn’t hurt to make it available.
Also, your app should stay out of the user’s way. UI inconsistencies can confuse the user and slow them down. Try your best to be consistent across your application. I chimed in here to mention that we actually put UI standards in code by making classes to inherit from. That way you actually have to do more work to make a part of the application look different.
Labels and messages need to be well thought out. Don’t ever let a user see lorem ipsum or even just poorly worded text. Don’t be vague in your text (“invalid input” etc.) Maintain your tone (informal vs. formal style.)
Be careful about button placement. Ok/Cancel should always be in the same order. Ok should actually normally be a verb, like “Create” or “Destroy.” Also the cancel action should be demarcated somehow. A red icon works, or some people even go so far as to make the positive action a button and the negative action a link.
As you grow your application beware of making too much clutter. The example given in the talk was Google vs. iGoogle. Switching between the two is easy and you aren’t forced to use iGoogle at all.
You should favor clarity and predictability over cleverness and coolness. For example, the speaker mentioned making a wizard where a regular form made more sense. The form is easier for the user and less work to implement.
In general layouts should flow left-to-right, top-to-bottom. So more general things should come first in that ordering. And furthermore try not to make user interfaces extremely busy. A 40% data density is optimum. Again, try to keep your UI balanced.
The use of ellipsis (…) in menus to point out which commands bring up new dialogs is a good convention. Shortcut keys for menu commands should be underlined, and the shortcuts should be mentioned in tooltips as well. 10 or more items in a menu are too many. Use submenus etc. Often used items should be closer to the top of the menu. Often used commands should have icons, but too many icons can look cluttered.
Use grouping in grids to reduce clutter. Use cell renderers to highlight important information. In editable grids, make it clear which fields are not editable. As we all know, too many rows and columns is bad, even though every user everywhere wants it.
In forms you shouldn’t use check boxes for radio buttons and vice versa just because you prefer the appearance. That is very confusing for users.
Also with treepanels it’s a good idea to preserve tree state to save the users from having to open the sub trees all the time.
Next up was “Ask the Ext Team.” I learned a lot from this panel. First off, creating manager classes to decouple classes can help with complex interaction. Or you could use the new Ext 3 bubbling feature. Another really cool idea they mentioned was making your top level ns inherit Observable and make it a singleton, and then allow things to communicate through that. I *love* that idea.
LongPollingProvider can allow for Comet (faked push.)
Ext is corporately located in Tampa, but there are devs in DC, Australia, and other places.
Good places to learn about Ext are the wiki and the forums.
The Ext team recommends extending classes only when you are actually making your own functionality as opposed to just configuring a class. (I humbly disagree
) You do need to buy a new license to use Ext 3.
Component.mon is a replacement for on that does some memory management for you. The listeners config doesn’t use it, but it doesn’t have to.
Also interesting to note: the presenters actually used Ext for the slides. It was very cool and made very good sense for their use case.
HBox and VBox are really cool and will help with layouts immensely.
After that we all went out in the courtyard to talk. It was a lot fun to talk to some of the other devs. I even met a few that I ended up riding to the airport and eating with the next day
And that’s it for Day 2. Stay tuned for the exciting conclusion: Day 3!
15 Apr
Enjoy day 2:
First off was the Ext 3 Release. They gave some interesting history (Ext 1.0 was released exactly 2 years ago today!) And then mentioned a few features of Ext 3. Mainly it was about Ext.Direct and how it is a solution for communication to/from the server that is apparently a need in the community. I hope to use it myself; but we’ll have to see based on the spec. More on that later. As for the designer the plan is not only to make it visual but also reusable, which is pretty exciting for me.
Honestly I was a little disappointed with the release. A lot of time was spent on GWT, which is kinda cool, but only half of the people there could use it, and even less do. I was hoping for a Chuck style release with a button and hype. Oh well
The next thing that I went to was Dissecting Ext’s Signature Sample. The app was very cool. I really liked learning about the implementation details. Often I have found the quality of the examples lacking (see here, in particular this horrendous line: [var firstGridDropTargetEl = firstGrid.getView().el.dom.childNodes[0].childNodes[1];
]), and therefore not good enough to model an application after. Well, this example app was just rife with great examples of how to do things.
First off, the spoke about namespaces. That was pretty much the same as what we already all know.
Second, how about implementing your App object as a singleton? A good idea and makes perfect sense if I’d ever thought of it myself. I also happened to find out about the itemId/getComponent stuff here. Apparently it was around in Ext 2.2, but it wasn’t documented. The idea is that you can set an itemId property on an item, and then on the container you can call getComponent(‘itemId’) to find that item. It’s kinda like non-global ids. Too bad it was never documented… Now that we have Ext 3.0 there is an even better way though!
Before you had to do this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | Ext.ux.Foo = Ext.extend(Ext.Foo, { initComponent: function() { var this.foo = new Blah({...}); var config = { ... items: this.foo }; Ext.apply(this, Ext.apply(this.initialConfig, config)); Ext.ux.Foo.superclass.initComponent.apply(this,arguments); }, someFn: function() { this.foo.bar } }); |
Now you can do this:
1 2 3 4 5 6 7 8 9 10 11 12 13 | Ext.ux.Foo = Ext.extend(Ext.Foo, { initComponent: function() { var config = { ... items: {xtype: 'blah', ref: 'foo'} }; Ext.apply(this, Ext.apply(this.initialConfig, config)); Ext.ux.Foo.superclass.initComponent.apply(this,arguments); }, someFn: function() { this.foo.bar } }); |
That is pretty excellent right there!
They also mentioned some Ext style stuff. This should be done when you make your own classes. First you list your properties, then overridden methods, and then you end with new methods that are specific to the class. They also mentioned that all strings used in the class should be properties to allow for simple i18n. Nice to know. We also had some good talks about regular old (aka bizarre) javascript OO. Any complex variables defined in the properties (classes, maybe arrays too) are basically class variables. So they only get instantiated on load time and they are shared (for better or worse) by all objects of that type. This has caused issues for me in the past, but I can see where it would be good for performance reasons.
The Ext.DataView class could be excellent for the TemplatePanel that I have created. I have to look into it some more, but something to look at nonetheless.
Plugins basically are a method for doing Mixins with javascript. I won’t go into why mixins are a good thing. Just look it up. One way or another, this will really help make our classes of higher quality. As for ext 3, we can now use ptype (like xtype) for plugins, and Ext.preg (like Ext.reg) for registering plugins. Excellent!
Ext.Direct sounds like it could clear up a lot of our boilerplate code on the server side….maybe. Basically what it would do is expose choice methods from our model classes. So instead of making four line actions that find a specific model and return the json version of the model, it happens automatically with some configuration. Beside that stuff it also automatically batches queries. So if you load two stores on one page they should automatically be batches into one request and one response. Depending on how hard it ends up being on the server side it could be totally awesome.
Next we had another QA panel. I learned that Ext tends to be getting a foothold anywhere that it is used. Static ids are a bad idea (not news.) Use xtypes for lazy instantiation when you can. Creating usable components is a good idea. HTML is bad; use JSON and Templates instead. Singletons are good. To be an Ext superstar spend 4+ hours per day on the forums. All the Ext superstars are stoked about Ext.Direct. As established too many times already, people don’t test with Ext UI stuff. And lastly, most Ext apps are not public facing.
Next was User Experience Design with Ext JS. This talk was a big deal. There were tons of people there and it was a very solid presentation. I couldn’t take perfect notes because the slideshow and what the presenter said didn’t match up for the first few slides. I’ll just document what I understood.
First off, user experience is based on psychology; what your users expect etc.
Design is based on decisions and constraints. One perfect example was that in the hotel one of the presentation rooms was way too wide, so most people couldn’t see the screen. That was a decision, whether a conscious one or not. Constraints are based on technology, costs, etc.
Design is not art. Or in other words, there is a science to it. That doesn’t mean it’s not creative. Part of his point here was that design is not graphic design. In fact, he said that you could basically ditch graphic design and still have good applications if you still paid attention to User Experience design.
One tip he gave for User Experience Design was that you shouldn’t pretend to be your user, because you just can’t. Instead, pretend to be your app and ask yourself how you should interpret what the user is doing. Furthermore, you can’t really learn from users without actually watching them. This isn’t news to me, but it’s still great advice. We use GoToMeeting for this.
Another point was that it’s good to think of the user experience with the “Halloween Principle.” That is, imagine your user is a mother of three on Halloween. They are in the middle of something fairly complex and then trick or treaters ring the bell. The user gives them candy, goes back to the app, and has no idea what she was doing. Can she remember where she was based on the visual clues in your app?
Another interesting insight that the presenter gave is that if your app has no bugs and the user uses it, they will be happy (and probably rate your app 7/10.) If a user finds a bug and you don’t fix it, they will be upset, but probably blame themselves (3/10.) But the important thing is that if there is a bug, and you fix it immediately and give them very personal service, they will be extremely satisfied and even go out of their way to advocate you. I can give you an example of this. Last week I ran into a few bugs with Ext 3.0. I posted about them on the forums and two were fixed in 5 minutes and one was fixed in less than a day. I am extremely happy about that service and I really will tell people about that.
(This is going to be a long post…)
Next up is the idea of flow. Basically a user will be most effective when the difficulty of the things they are trying to do match up with their skill level. So if they are extremely skilled and doing very simple things, they will get bored and be unproductive. On the other hand it’s fairly obvious that if you have a very inexperienced user they won’t be able to do extremely complex tasks.
Also interesting: a lot of small features matter much more to users than one huge one. This one isn’t that hard to understand. Just think about small things in software you use that drives you crazy. And how often do you use that one amazing feature that they added?
WTFs per minute are a good metric for user interfaces.
Don’t fall in love with your work because you need to be willing to throw it away.
Don’t “bend” (aka train) your users. Change the software so that it is more flexible. Part of that is that basically you need to be able to get to different parts of you software from numerous different angles. Think hotkeys, menubars, right click menus, etc.
Balance items on the screen so everything isn’t just on one side or the other.
Match your colors (use kuler for example.)
Align things. This will help your users when they look at different parts of your app.
Make things look 3-D. Your app should look like the user can touch it and the focused things are in the front and non focused are in the back, etc. See here for demos of the app the presenter made. Note: that is all ExtJS.
Ok, I have lots more to post, but it’s midnight and I have a conference to be at at 8:30 tomorrow. I’ll post the rest of my notes tomorrow. The sessions tomorrow end much sooner, so it shouldn’t be to much.
3 Feb
So today the Ext JS guys posted about their conference in April. We use Ext JS at work and I have become pretty competent in using the framework for UI design. A lot of the things that they are adding in Ext JS 3.0 will make the toolkit more compelling. The most significant of all will be the accessibility changes and the addition of Ext Core as a standalone library.
Anyway, with that in mind I asked my boss if I could go and it was decided that I am going! I am very excited! This is the first conference I have gone to and I think it will be a good one. The talks that I am looking forward to the most are the Keynote with Douglas Crockford, Ext 3.x – What’s new & how to use it, Introducing Ext Core, Building Accessible Apps for Ext JS and Ext GWT, Refactoring, Application Deployment, Building Usable Interfaces, Optimizing Performance, and Critical Developer Tools.
That may seem like a lot to be excited about to you, but I am quite stoked. By my reckoning that’s %33 Ext talks and %66 general javascript (or more general) programming talks, which is quite acceptable to me.
I also think it’s pretty Cool that they give you a discount based on how long you have had a login for the forum and how many posts you have made to the forum. I got a 25% discount based 200 posts and five months of participation. That really is an interesting way to foster community.
When I do end up going to the conference I will do my best to distill the talks into written word