The project I am working on at work is going to be deployed soon, so today I worked on some of those things that need to be taken care of before the deploy. One of those things was changing our gigantic list of javascript files into a single file with minimal hassle. I actually tried to implement it myself, but that was silly. A simple search on CPAN for catalyst javascript yields two promising results.

The first is Catalyst::View::JavaScript. While this is an excellent package which does caching and automatically minifies depending on whether the server is in debug mode or not, it expects javascript to be in memory already. Maybe that’s something that’s common and I don’t know about it, but we almost never write javascript on the server.

So onto the next result: Catalyst::View::JavaScript. It does not do caching, and it always minifies. But heck! That’s not too bad. Before installing it, note my RT that it does not have correct dependencies. Hopefully that will be fixed soon. I also sent another RT. Since I technically write more lines of javascript than perl, I need to be able to turn off minification. So I patched the module to only minify when the server is not in debug mode. Hopefully that will be accepted as well.

So after doing that I ensured that apache had gzip turned on to reduce the amount of bandwidth required by the clients to download our javascript. Then I got curious. How much of a difference do these things make? So I measured it:

Wow! Minification really doesn’t help that much, whereas gzip’ing is huge.

Now, just to be clear, this isn’t a great solution for an externally facing site, because the clients don’t currently cache the minified javascript, because it gets served by Catalyst and it’s probably not worth my time to set that up all the headers to fix that for a site that runs mostly on a LAN. Once the customer does start allowing external access we’ll want to set some last-modified headers and whatnot.

I also did a few more cool tweaks using debug mode. For example I have the debug version use ext-all-debug.js, which has extra stuff for error messages and whatnot, and ext-all.js, which is preminified and has error checking removed for performance reasons (for the tests above I used ext-all.js the whole time.)

Furthermore I have the debug switch my server side error messages from actual exceptions to a simple message asking the user to get in touch with the devs and tell us what happened and what time it was when it happened.

All in all I felt really good about the code I wrote today. It will help me a lot as time goes on and it makes the dev server way faster (1 small download is way faster that 60~ really small ones). Too bad the customer will probably not notice a significant amount of it Anyway, hopefully these tips will help give you some ideas for your webapps.

You may have wondered why I had the slight delay in posts this week. I had a good reason: we switched one of our major products from IIS to Apache! In general it was a fairly painless process. The details are documented in my previous post, Migrating from IIS to Apache. There was one hitch though…

We have an autocomplete field that needs to be pretty snappy. For IIS we just installed ActivePerl and named the file autocomplete.plex and it was good. Well, mod_perl isn’t quite so easy; we had a couple major snags.

For some reason we had a lot of issues trying to get mod_perl working with ActivePerl 5.8. I initially wanted to leave it at 5.8 because of various modules that were already installed and couldn’t easily be upgraded. I eventually decided to bite the bullet and install 5.10 and update the modules.

After installing Apache (see previous post) and ActivePerl 5.10 (plus lots of modules) we had to install mod_perl, which was surprisingly easy:

After that I had to add to the configuration for Apache:

Since we were only converting that one script we limited mod_perl to that single directory.

The last issue that we had was that you cannot use CGI.pm with mod_perl; fortunately for us in this file all we were using it for was to print a header, and mod_perl does that for you by default, so we just commented out those lines. At some point I’ll need to do some more research and learn the mod_perl way to get params and print headers.

In general it was pretty nice. We now have a much more stable configuration and mod_perl is extremely fast. The autocompleter went around ten times faster (60~ms total afterwards) after switching. I didn’t get a persistent DBI connection set up, but I did put all of the use directives in the BEGIN block.

If anyone has pointers on persistent DBI connections, a way to only set mod_perl for a single file extension, or whatever the mod_perl replacement is for CGI.pm, let me know!

At my job we use a combination of IIS, SQL Server, and Perl. In general it works pretty well. But there is one major problem: if we ever do a warn in perl, instead of printing the message to the log, it crashes the server. That’s a big deal since multiple people are using the server and fixing the issue means VNCing in and recycling the app pool. This doesn’t always happen, but it happens a lot; enough to make me consider setting up Apache on my personal computer so that I can get some serious logging. Anyway, I don’t know if we have a typical setup or not, but this is what I had to do to get it all going.

1. Install ActiveState perl into C:/usr (not C:/Perl.) You can get the latest version here. That’s more or less it for installing perl. Note: Latest at the time of writing is Perl 5.10.0.1004
2. Install Apache. You can get the latest version here. I suggest getting the binary msi. Get the one with OpenSSL if you want to set up https (not covered here.) Note: Latest at the time of writing is Apache 2.2.10 (OpenSSL 0.9.8i).
3. Install Perl Modules. I am not sure of what all modules our software requires that doesn’t come with perl out of the box, but I know for sure that we need DateTime. So to install that open a console and type ppm install DateTime. You can use the gui instead if you’d like, but it tends to just get in my way because it’s so slow. The way you will know that you are missing a module is if you get an error in the log like this:

   1 2 3

   [Tue Nov 18 17:33:05 2008] [ERROR] [client 127.0.0.1] Premature END of script headers: foo.plx, referer: http://127.0.0.1/ [Tue Nov 18 17:33:05 2008] [ERROR] [client 127.0.0.1] Can't locate DateTime.pm in @INC (@INC contains: C:/usr/site/lib C:/usr/lib .) at foo.plx line 39., referer: http://127.0.0.1/ [Tue Nov 18 17:33:05 2008] [ERROR] [client 127.0.0.1] BEGIN failed--compilation aborted at foo.plx LINE 39., referer: http://127.0.0.1/

4. Migrate source code. For me this just meant checking out one folder from subversion into C:/Inetpub (recommended so that things will continue to work with IIS and apache,) and copying a directory of static html into the same directory. I ended up with two main directories like this: C:/Inetpub/main and C:/Inetput/static. (Names changed to protect the innocent )
5. Configure Apache. This, besides the last step, is probably the hardest step. First open httpd.conf, probably C:/Program Files/Apache Software Foundation/Apache2.2/conf/httpd.conf… but you can find it in the start menu). On IIS our static directory is the root and then the main directory is a subdirectory of the static directory. To set this up, first find the

   1	DocumentRoot "..."

   directive in the httpd.conf and change it to

   1	DocumentRoot "C:/Inetpub/static"

   Next you’ll want to make sure that your Directories are configured. Find the existing Directory section and just change the directory to whatever you just did, and then add another one for each other directory in Inetpub. This is what I ended up with:

   1 2 3 4 5 6 7 8 9 10 11 12

   <Directory "C:/Inetpub/static">     Options Indexes FollowSymLinks ExecCGI     AllowOverride None     Order allow,deny     Allow from all </Directory> <Directory "C:/Inetpub/main">     Options Indexes FollowSymLinks ExecCGI     AllowOverride None     Order allow,deny     Allow from all </Directory>

   And then because our static directory was root and the main directory was a subdirectory of the static directory, add a line like the following to the alias_module section:

   1	Alias /user /Inetpub/main

   Also, since we are a perl shop, we have to allow execution of various types of perl programs, so find the mime_module section of the code and make the AddHandler part look like this:

   1	AddHandler cgi-script .cgi .plx .plex

   And then last of all, the main page of our root directory on IIS is Default.html, so instead of renaming it to index.html, find the secion of the code for the Directory and add a DirectoryIndex part so it is like this:

   1	DirectoryIndex Default.html

   I ended up setting it for both main and static. Here’s my entire httpd.conf if you just wanna see the final product:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85

   ServerRoot "C:/Program Files/Apache Software Foundation/Apache2.2" Listen 80 LoadModule actions_module modules/mod_actions.so LoadModule alias_module modules/mod_alias.so LoadModule asis_module modules/mod_asis.so LoadModule auth_basic_module modules/mod_auth_basic.so LoadModule authn_default_module modules/mod_authn_default.so LoadModule authn_file_module modules/mod_authn_file.so LoadModule authz_default_module modules/mod_authz_default.so LoadModule authz_groupfile_module modules/mod_authz_groupfile.so LoadModule authz_host_module modules/mod_authz_host.so LoadModule authz_user_module modules/mod_authz_user.so LoadModule autoindex_module modules/mod_autoindex.so LoadModule cgi_module modules/mod_cgi.so LoadModule dir_module modules/mod_dir.so LoadModule env_module modules/mod_env.so LoadModule include_module modules/mod_include.so LoadModule isapi_module modules/mod_isapi.so LoadModule log_config_module modules/mod_log_config.so LoadModule mime_module modules/mod_mime.so LoadModule negotiation_module modules/mod_negotiation.so LoadModule setenvif_module modules/mod_setenvif.so <IfModule !mpm_netware_module> <IfModule !mpm_winnt_module> User daemon Group daemon </IfModule> </IfModule> ServerAdmin frewmbot@gmail.com DocumentRoot "C:/Inetpub/static" <Directory />     Options FollowSymLinks     AllowOverride None     Order deny,allow     Deny from all </Directory> <Directory "C:/Inetpub/static">     Options Indexes FollowSymLinks ExecCGI     AllowOverride None     Order allow,deny     Allow from all     DirectoryIndex Default.html </Directory> <Directory "C:/Inetpub/main">     Options Indexes FollowSymLinks ExecCGI     AllowOverride None     Order allow,deny     Allow from all     DirectoryIndex main.plx </Directory> <IfModule dir_module>     DirectoryIndex index.html </IfModule> <FilesMatch "^.ht">     Order allow,deny     Deny from all     Satisfy All </FilesMatch> ErrorLog "logs/error.log" LogLevel warn <IfModule log_config_module>     LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" combined     LogFormat "%h %l %u %t "%r" %>s %b" common     <IfModule logio_module>       LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" %I %O" combinedio     </IfModule>     CustomLog "logs/access.log" common </IfModule> <IfModule alias_module>     Alias /user /Inetpub/main     ScriptAlias /cgi-bin/ "C:/Program Files/Apache Software Foundation/Apache2.2/cgi-bin/" </IfModule> <Directory "C:/Program Files/Apache Software Foundation/Apache2.2/cgi-bin">     AllowOverride None     Options None     Order allow,deny     Allow from all </Directory> DefaultType text/plain <IfModule mime_module>     TypesConfig conf/mime.types     AddType application/x-compress .Z     AddType application/x-gzip .gz .tgz     AddHandler cgi-script .cgi .plx .plex </IfModule>

6. Ensure all of your perl files start with #!/usr/bin/perl. If you don’t do this Apache will give you an error, 500 Internal Server on the output, and then something like this in the log:

   1 2

   [Tue Dec 09 20:59:04 2008] [error] [client 127.0.0.1] (OS 3)The system cannot find the path specified.  : couldn't create child process: 720003: employee_training_report.plx [Tue Dec 09 20:59:04 2008] [error] [client 127.0.0.1] (OS 3)The system cannot find the path specified.  : couldn't spawn child process: C:/Inetpub/epms/customer/employee_training_report.plx

   As stated, this just means that the bangline is wrong and needs to be set to #!/usr/bin/perl. Note: the log is probably in C:/Program Files/Apache Software Foundation/Apache2.2/logs/error.log, but again, you can find that in the start menu.

7. Fix all headers. Usually with IIS we output headers like this:

   1 2	print "HTTP/1.0 200 OK\n"; print header;

   header is a function of the CGI module. For IIS you print out the first part to force the server into NPH mode. I recommend, for ease of migration, making your own module that your scripts can use that will print the header correctly whether it’s Apache or IIS. Here’s ours:

   1 2 3

   sub header {     return (($ENV{PERLXS})?"HTTP/1.0 200 OK\r\n":"").CGI->header(@_);<br /> }

   And note that anything that gets passed to the header method automatically takes any arguments that you passed and gives them to CGI. This allows for a simple method of regular expression based search and replace to fix things to use your new method. (for vim something like this will work: :%s/v^(s*prints+).*header((.*));/1Module::header2;/g )

   I do this part as I see it as a problem, as my boss didn’t want me to search and replace the whole codebase, so the error you are going to look for is a 500 from the browser and then something like this in the log:

   1	[Tue Nov 18 17:38:38 2008] [error] [client 127.0.0.1] malformed header from script. Bad header=HTTP/1.0 200 OK: foo.plx

And that’s basically it! Any tips you might have to add are welcome!