I write a lot of ExtJS grids at work. I have written JavaScript classes for our Ext grids that generate as much as possible automatically, but the actual column definitions of the grids are almost always unique. The project I am on now is nearing our first real deploy, and we’re late, so things have been really, really busy.

It wasn’t until recently that I realized just how much time I spent working on grids and their related records (representation of the rows of a grid.) Although even if I’d known just how much I do this at the beginning of the project, I certainly didn’t know DBIx::Class as much as I do now, in addition to the other 4 non-core modules that I’ll mention.

Because I’ve been working all these super long hours (12-14 a day), often after a frustrating day I’ll try to code something more relaxing and rewarding at home or at work but not during work. So today I decided to finally take the plunge and do what I’ve been pondering for a few months and write some code to generate Ext scaffolding for me. So I’m going to walk you through the script that I wrote (and will probably work on more as time goes by.)

This isn’t a full module, not even in my repo yet, so it’s all just in one file. Here’s the boilerplate intro:

That’s pretty basic. We use a bunch of modules that I’ve used in at least 1 other project before and was happy with.

That’s the code to parse any kind of Catalyst config file and the grab a new schema based on it. It’s a ton of biolerplate but I live with it.

So next up is a basic inflection function that gives us all the different forms of a word we might need. It starts with either “single_foo” or “SingleBar” and gives us six variations based on that:

So that uses String::CamelCase and Lingua::EN::Inflect combined with join and split mostly.

We’re *almost* ready to generate a record. But first we need to define a mapping from the data type in the database to the data type that Ext uses:

Ok, now let’s look at the code to generate an Ext.record:

So what’s going on here is that we get the source from the schema. The source could be considered something like a table definition, although it can also point at a view or whatever too. Then we start generating the string representing our record, and then we use a join/gather combo to get the column data the way we want it.

We could certainly just use a more complex map instead of the for+gather that we have, but I personally feel than any map where you *must* use the block form is cumbersome. So we join together all of the strings that gather took, and then append the end of the definition, and voilà, we have a record!

Here’s example output on our Customer source:

If you’re still following along you should be fine with the next, much more impressive functionality.

I’d like to have the script do as much as possible to speed up my work, so let’s see how far we can take this. First, I made a mapping from type to renderer, so that things that are datetimes use our custom datetime renderer, same with booleans:

Next, I wrote a function that would try it’s best to guess how wide a column should be based on it’s header and the average width of the strings inside of it:

Now finally we have the grid function:

We see nearly the same structure here that we did with the record, except much more intricate. Notice the code to generate the width uses the magic constant 7. As I played with this I found that 7 seemed to work for the font that Ext uses by default (Arial?) Optimally I would actually use some kind of metrics package to ask it for the width of all of the strings that I generated in the function above and average that, instead of averaging character lengths. But this seems to work really, really well, so the ROI is pretty good.

Also note the code to pick and insert the renderer. It’s not complex or anything, but it yields very convenient results. Here’s the output of running this one on the same source we used for the record:

Note that there are probably some things that we could do a little better. For instance, we should probably check for text fields and not include them in the grid. We could also be clever and look for anything ending with _id and have it hidden by default. But in general, the above is very nice results. At the very least you are probably going to need to reorder the columns (although you could give some kind of grid_order_id to the column definition in the Schema::Result code, but I don’t think that’s worth it.)

So I hope someone else enjoyed reading this as much as I did writing it. After I’ve exhausted all of the little Ext things I can use this for (form, etc) I will probably look into messing with DBICx::AutoDoc to add in Dia outputs.

(Sorry if you heard this already )

At $work we do as much “view” type code as we can in JavaScript with the ExtJS framework. I have personally found it to be a great framework to work with, although often it is lacking in the non-UI department. One thing that at first I really liked about Ext was their Template and XTemplate classes. But as time went on I got more and more annoyed with those modules.

I’ve always thought that Template-Toolkit was a really nice templating library to work with. I hate templating html because of all the weird little gotchas having to do with CSS and whatnot, but doing almost all that kind of work in JavaScript I have started missing Template-Toolkit. At some point I heard about Template::Tiny and this past Saturday I thought, “that’s like, 160 LOC…I could probably port that to javascript!”

So I did! It’s not really done yet, but the current code is at github. I need to finish porting the test suite from Perl 5 to JavaScript so that I can ensure correctness (I am certain I screwed up some stuff.)

Alias mentioned in his post that I use XRegExp (600 LOC) to help out with the Regular Expression support in JavaScript for this module. I actually wasn’t going to, since the only thing I needed it for was the /x flag, or to be more clear telling the parser to ignore whitespace, but I want to keep Template.Tiny in sync with the Perl version, and I really don’t want to strip out all the whitespace by hand. If someone takes issue with the dep they can fork away

So once it is fully ported I fully intend to use it entirely from now on instead of the Ext templating. But take note, Ext templates and Template.Tinies (what?) really solve different problems and have different goals. The following is a list of Pros and Cons of each:

Ext.(X)Template

Pros:

• Can be compiled to JavaScript, for speed
• Allows complex expressions in “if” blocks (see Example 1)
• Interesting “topicalizing” feature (Example 2)
• Crashes when you leave out a variable (no mystery as to why a field is blank) (Example 3)
• Can execute arbitrary JavaScript code in a template for complex stuff (Example 4)
• Neat automatic “current item” style variables when you are iterating over an array (Example 5)
• Basic Math Support
• Ability to call functions associated with Template object
• Very cool builtin Renderer support (Example 6)

Cons:

• No else if. If you want something like that you must do if !expr. Lame.
• Crashes when you leave out a variable (mostly that’s just annoying)
• This is almost entirely subjective, but having xml as a templating thing is kinda gross.
• Not really open source, so I can’t use it in personal projects and have people use my code in a corporate setting

Example 1:

Example 2:

Example 3:

Example 4:

Example 5:

Example 6:

Template.Tiny

Pros:

• Has IF/ELSE (Example 7)
• Doesn’t crash on undefined fields
• Nicely Licenced (Perl License)

Cons:

• No complex expressions, math, or external javascript support
• No topicalizing
• Doesn’t crash on undefined fields (could be nice for debugging)
• Probably slower (haven’t checked that yet)

Example 7:

I am planning on making a wrapper for TT for our stuff that will allow an anonymous function that will do data transformation like above. But as you can see above XTemplate really has more to offer, it just annoys the heck out of me on a regular basis

I’m probably preaching to the choir here, but it must be said: code reuse is most excellent!

Today I got a somewhat complex feature working for our customer, and almost all of it was features I’d already written, and due to the organization of our system I could easily reuse most of the code.

Our customer fixes airplane parts. When they fix a part they need to document every single thing they did to the part. We have each operation (more or less) that can be done already defined so that they can at least save those keystrokes (they are actually operation templates.) But there are a lot of operations and a typical work order will be around 50 operations, so choosing the same operations over and over is a waste of time. So we have a feature that lets them look at other work orders that were fixing the same type of part and copy operations (and materials) from that.

It was really easy to use the existing view for operations and materials because the front end is entirely comprised of JS classes. I even used an instance of the work scope grid to list all of the work scopes that are for the given part. The nice thing was that so far I’ve written no new server side code yet. And for the classes I didn’t use inheritance; I used a role style object modification by doing what Moose people would see as an after method on new (called a plugin in ExtJS). With the plugins I could simply change the store to ask for a given part-type’s work orders, hide extra columns, and add listeners to update the operations and materials when a user clicked a row.

Don’t think this is all just JS praise; Perl and Catalyst were help too. But really the benefit here was the use of any web framework in general. Because I’m using a framework I can easily find server side actions that do what I need (which the grids were already tied to in their base classes, but still.) In our other projects I’d be hard pressed to give you a list of all of our “actions,” whereas with CGIApp I can easily make a list of runmodes myself, and with Catalyst the server will make a list for me.

Excellent!

The end! Ok not quite. So this was the last day of the conference. It was shorter than the other days and most of us had to checkout anyway. Still exciting!

First off we got an awesome demo of the Designer. It looks like it will be extremely useful for exploring the framework and playing with layouts. You can edit multiple components at once, as if it were an IDE. You can even load data into grids on the designer. The bad news is that it looks like the Designer will be a service or an Air app that you buy. I understand the motivation behind that, but it’s still frustrating.

The next talk I went to was the Application Deployment talk. Very good information, but little of it was new to me. First off, Yahoo has written a lot of best practices for website performance. Take a look at those. YSlow is based on those and can tell you if you are following some of those best practices. YSlow 2.0 is in development and will be made of unicorn’s and rainbows. Typically you should skip your overall score and just look at the specific items. Gzip is good to use. Putting versions on your filenames can really help with performance (more on this later.) CSS should be in the header and as much JS as possible should be in the body. This will allow you to load basic JS and render the page before all of the JS is loaded. Minify your JS. ETags probably won’t actually help you unless you are huge.

Firebug network monitor is your friend. Fiddler (.NET app) can help for some esoteric things like file uploads or flash interaction that won’t show up in firebug. JSLint is an amazing tool that can help you write better code. Some of the people there wrote a Yahoo Widget that will automatically run JSLint on any files that have been changed. Very cool!

You should use parseInt(foo, 10) instead of just parseInt(foo). Why? Because parseInt defaults to octal.

JS Builder is a .NET app written by the ext people that will compress your javascript
files nicely. It can also be run from the CLI for batch usage.

CSS Sprite generation can significantly reduce your server pings. Also it can reduce images a lot due to boilerplate png stuff in icons (30%!) Use this to automate the process.

You need a favicon because browsers will always check for it. Be careful about giving it a big expire time because you cannot always just rename it as older browsers only check for favicon.ico.

And that was basically it! I didn’t go to the next session because we were way over time and I had to check out. In general I really enjoyed the conference and think I learned a lot from it. The only major problem I had was the Ritz stuff I mentioned before. Other things they did were really cool. For example: to ask questions in the panel you posted them online and you could vote them up and then they would answer the ones with the highest votes. This was good because people who talk really slow had less of a chance to slow us down etc. I think they should have given the field a max length, as too many words are just confusing anyway.

I give the conference 4.5/5 stars. Great in almost every way

Ok, the next session I went to on Day 2 was the session on Refactoring. Refactoring is one of the few high quality buzzwords that I hear regularly, so I was excited to hear what the talk would go over. It was very much Ext specific, but the final changes to the component that we “Extified” were amazing.

First off, what does it mean to Extify a component? The comp needs to fit into the Component Model, which is mostly a lifecycle issue. Typically this will be extending an existing component or creating a new one by inheriting from Component. Refactoring also involves clean code, consistent code, and hardest of all, documented code. And part of that which is fairly Ext specific is that the configuration should be elegant; or the config needs to be predictable and not have too many options.

Unfortunately this session was like drinking from a firehose, so I couldn’t take notes more than that. For the most part though it involved simple changes like moving view changes out of the component entirely and into CSS. Documentation should be done with ext-doc, which I look forward to setting up at work. Config option names need to be thought through carefully so that they match well with the rest of the framework. Other nice things that were done is to allow a store to be defined as a store, a store config, an array, or a store id. Keep your eyes peeled for the notes from this presentation. Should be excellent.

The next session I went to was another usability session. I am of the opinion that it’s hard to have too many of these. This session was a lot more hands-on, specific ideas than the other session.

His first recommendation was to show that background events are occurring. First go look at Forever21. Make your browser window 1024×768. Go to a product and add it to your cart. The part of the page that shows that the item has been added to the cart is not where you clicked, and probably isn’t even in view. Did anything happen? Is the page just going slow? Nope. You added it every time you clicked.

Now, on the other hand, check out this Ext sample. Note how when you use the menus messages pop up fairly unobtrusively. Also note that the time the messages are displayed can be increased and they can be set up to go away when the user clicks them. Much nicer!

In that same vein, try to avoid modal browser feedback as it freezes not just your whole app, but often the whole browser. Also avoid feedback about trivial events that the user does not care about. If at all possible, make actions easy to undo. For the actions that cannot be undone, warn your user.

Speed matters for all applications. There are some easy ways to make ext applications very snappy. Use xtypes possible as they allow for lazy instantiation on rendering. When waiting for the server, a basic load mask should be used for .5s or more. If it takes 5s or more, use a spinner or some other kind of progress indicator. If it takes more than 10s a real progress bar should be used. Note that real progress bars can still be faked by precalculating how long certain sets of data take.

The help tool for the title of a panel can be used for easy context sensitive help. In general the user shouldn’t need help, but it doesn’t hurt to make it available.

Also, your app should stay out of the user’s way. UI inconsistencies can confuse the user and slow them down. Try your best to be consistent across your application. I chimed in here to mention that we actually put UI standards in code by making classes to inherit from. That way you actually have to do more work to make a part of the application look different.

Labels and messages need to be well thought out. Don’t ever let a user see lorem ipsum or even just poorly worded text. Don’t be vague in your text (“invalid input” etc.) Maintain your tone (informal vs. formal style.)

Be careful about button placement. Ok/Cancel should always be in the same order. Ok should actually normally be a verb, like “Create” or “Destroy.” Also the cancel action should be demarcated somehow. A red icon works, or some people even go so far as to make the positive action a button and the negative action a link.

As you grow your application beware of making too much clutter. The example given in the talk was Google vs. iGoogle. Switching between the two is easy and you aren’t forced to use iGoogle at all.

You should favor clarity and predictability over cleverness and coolness. For example, the speaker mentioned making a wizard where a regular form made more sense. The form is easier for the user and less work to implement.

In general layouts should flow left-to-right, top-to-bottom. So more general things should come first in that ordering. And furthermore try not to make user interfaces extremely busy. A 40% data density is optimum. Again, try to keep your UI balanced.

The use of ellipsis (…) in menus to point out which commands bring up new dialogs is a good convention. Shortcut keys for menu commands should be underlined, and the shortcuts should be mentioned in tooltips as well. 10 or more items in a menu are too many. Use submenus etc. Often used items should be closer to the top of the menu. Often used commands should have icons, but too many icons can look cluttered.

Use grouping in grids to reduce clutter. Use cell renderers to highlight important information. In editable grids, make it clear which fields are not editable. As we all know, too many rows and columns is bad, even though every user everywhere wants it.

In forms you shouldn’t use check boxes for radio buttons and vice versa just because you prefer the appearance. That is very confusing for users.

Also with treepanels it’s a good idea to preserve tree state to save the users from having to open the sub trees all the time.

Next up was “Ask the Ext Team.” I learned a lot from this panel. First off, creating manager classes to decouple classes can help with complex interaction. Or you could use the new Ext 3 bubbling feature. Another really cool idea they mentioned was making your top level ns inherit Observable and make it a singleton, and then allow things to communicate through that. I *love* that idea.

LongPollingProvider can allow for Comet (faked push.)

Ext is corporately located in Tampa, but there are devs in DC, Australia, and other places.

Good places to learn about Ext are the wiki and the forums.

The Ext team recommends extending classes only when you are actually making your own functionality as opposed to just configuring a class. (I humbly disagree ) You do need to buy a new license to use Ext 3.

Component.mon is a replacement for on that does some memory management for you. The listeners config doesn’t use it, but it doesn’t have to.

Also interesting to note: the presenters actually used Ext for the slides. It was very cool and made very good sense for their use case.

HBox and VBox are really cool and will help with layouts immensely.

After that we all went out in the courtyard to talk. It was a lot fun to talk to some of the other devs. I even met a few that I ended up riding to the airport and eating with the next day

And that’s it for Day 2. Stay tuned for the exciting conclusion: Day 3!

Enjoy day 2:

First off was the Ext 3 Release. They gave some interesting history (Ext 1.0 was released exactly 2 years ago today!) And then mentioned a few features of Ext 3. Mainly it was about Ext.Direct and how it is a solution for communication to/from the server that is apparently a need in the community. I hope to use it myself; but we’ll have to see based on the spec. More on that later. As for the designer the plan is not only to make it visual but also reusable, which is pretty exciting for me.

Honestly I was a little disappointed with the release. A lot of time was spent on GWT, which is kinda cool, but only half of the people there could use it, and even less do. I was hoping for a Chuck style release with a button and hype. Oh well

The next thing that I went to was Dissecting Ext’s Signature Sample. The app was very cool. I really liked learning about the implementation details. Often I have found the quality of the examples lacking (see here, in particular this horrendous line: [var firstGridDropTargetEl = firstGrid.getView().el.dom.childNodes[0].childNodes[1];
]), and therefore not good enough to model an application after. Well, this example app was just rife with great examples of how to do things.

First off, the spoke about namespaces. That was pretty much the same as what we already all know.

Second, how about implementing your App object as a singleton? A good idea and makes perfect sense if I’d ever thought of it myself. I also happened to find out about the itemId/getComponent stuff here. Apparently it was around in Ext 2.2, but it wasn’t documented. The idea is that you can set an itemId property on an item, and then on the container you can call getComponent(‘itemId’) to find that item. It’s kinda like non-global ids. Too bad it was never documented… Now that we have Ext 3.0 there is an even better way though!

Before you had to do this:

Now you can do this:

That is pretty excellent right there!

They also mentioned some Ext style stuff. This should be done when you make your own classes. First you list your properties, then overridden methods, and then you end with new methods that are specific to the class. They also mentioned that all strings used in the class should be properties to allow for simple i18n. Nice to know. We also had some good talks about regular old (aka bizarre) javascript OO. Any complex variables defined in the properties (classes, maybe arrays too) are basically class variables. So they only get instantiated on load time and they are shared (for better or worse) by all objects of that type. This has caused issues for me in the past, but I can see where it would be good for performance reasons.

The Ext.DataView class could be excellent for the TemplatePanel that I have created. I have to look into it some more, but something to look at nonetheless.

Plugins basically are a method for doing Mixins with javascript. I won’t go into why mixins are a good thing. Just look it up. One way or another, this will really help make our classes of higher quality. As for ext 3, we can now use ptype (like xtype) for plugins, and Ext.preg (like Ext.reg) for registering plugins. Excellent!

Ext.Direct sounds like it could clear up a lot of our boilerplate code on the server side….maybe. Basically what it would do is expose choice methods from our model classes. So instead of making four line actions that find a specific model and return the json version of the model, it happens automatically with some configuration. Beside that stuff it also automatically batches queries. So if you load two stores on one page they should automatically be batches into one request and one response. Depending on how hard it ends up being on the server side it could be totally awesome.

Next we had another QA panel. I learned that Ext tends to be getting a foothold anywhere that it is used. Static ids are a bad idea (not news.) Use xtypes for lazy instantiation when you can. Creating usable components is a good idea. HTML is bad; use JSON and Templates instead. Singletons are good. To be an Ext superstar spend 4+ hours per day on the forums. All the Ext superstars are stoked about Ext.Direct. As established too many times already, people don’t test with Ext UI stuff. And lastly, most Ext apps are not public facing.

Next was User Experience Design with Ext JS. This talk was a big deal. There were tons of people there and it was a very solid presentation. I couldn’t take perfect notes because the slideshow and what the presenter said didn’t match up for the first few slides. I’ll just document what I understood.

First off, user experience is based on psychology; what your users expect etc.

Design is based on decisions and constraints. One perfect example was that in the hotel one of the presentation rooms was way too wide, so most people couldn’t see the screen. That was a decision, whether a conscious one or not. Constraints are based on technology, costs, etc.

Design is not art. Or in other words, there is a science to it. That doesn’t mean it’s not creative. Part of his point here was that design is not graphic design. In fact, he said that you could basically ditch graphic design and still have good applications if you still paid attention to User Experience design.

One tip he gave for User Experience Design was that you shouldn’t pretend to be your user, because you just can’t. Instead, pretend to be your app and ask yourself how you should interpret what the user is doing. Furthermore, you can’t really learn from users without actually watching them. This isn’t news to me, but it’s still great advice. We use GoToMeeting for this.

Another point was that it’s good to think of the user experience with the “Halloween Principle.” That is, imagine your user is a mother of three on Halloween. They are in the middle of something fairly complex and then trick or treaters ring the bell. The user gives them candy, goes back to the app, and has no idea what she was doing. Can she remember where she was based on the visual clues in your app?

Another interesting insight that the presenter gave is that if your app has no bugs and the user uses it, they will be happy (and probably rate your app 7/10.) If a user finds a bug and you don’t fix it, they will be upset, but probably blame themselves (3/10.) But the important thing is that if there is a bug, and you fix it immediately and give them very personal service, they will be extremely satisfied and even go out of their way to advocate you. I can give you an example of this. Last week I ran into a few bugs with Ext 3.0. I posted about them on the forums and two were fixed in 5 minutes and one was fixed in less than a day. I am extremely happy about that service and I really will tell people about that.

(This is going to be a long post…)

Next up is the idea of flow. Basically a user will be most effective when the difficulty of the things they are trying to do match up with their skill level. So if they are extremely skilled and doing very simple things, they will get bored and be unproductive. On the other hand it’s fairly obvious that if you have a very inexperienced user they won’t be able to do extremely complex tasks.

Also interesting: a lot of small features matter much more to users than one huge one. This one isn’t that hard to understand. Just think about small things in software you use that drives you crazy. And how often do you use that one amazing feature that they added?

WTFs per minute are a good metric for user interfaces.

Don’t fall in love with your work because you need to be willing to throw it away.

Don’t “bend” (aka train) your users. Change the software so that it is more flexible. Part of that is that basically you need to be able to get to different parts of you software from numerous different angles. Think hotkeys, menubars, right click menus, etc.

Balance items on the screen so everything isn’t just on one side or the other.

Match your colors (use kuler for example.)

Align things. This will help your users when they look at different parts of your app.

Make things look 3-D. Your app should look like the user can touch it and the focused things are in the front and non focused are in the back, etc. See here for demos of the app the presenter made. Note: that is all ExtJS.

Ok, I have lots more to post, but it’s midnight and I have a conference to be at at 8:30 tomorrow. I’ll post the rest of my notes tomorrow. The sessions tomorrow end much sooner, so it shouldn’t be to much.

I am in the beginning of writing a web application with ExtJS. ExtJS is a javascript ui framework that’s extremely object oriented. I read once that it’s a good idea to predefine your user interface objects as (effectively) classes. One of the reasons for this is that it uses far less memory in our browsers. That’s a pretty good reason. Another reason is that you end up with smaller bits of code to work with at a time, thus allowing you to focus better on the task at hand.

Well just now my boss asked me to add an image beneath a treeview. That basically involves making a panel that has a treeview and an image. Well, instead of having to go through the code and basically do it, all I had to do was change my ACDRI.ui.LeftPanel from a special treeview to a Panel that has an ACDRI.ui.NavigationPanel (what it used to be) and an image. The original code was actually entirely unchanged except for the name. Furthermore, if we decide to go back it will require exactly one line of code to change. Excellent!