Entirely Too Motivated

So this week I've been very motivated to work on my more nerdy hobbies. Here's an update on my latest progress

• Made use of Hudson to set up a continuous build. WOW is this software great. I'm amazed at how good free software can get!


• Attempted to get a Scala job up on hudson, so I'll know early on when scala fails on windows. :Sigh: I'm not sure what's going on here, but it looks like no one at EPFL really wants to maintain Scala for windows. While I understand their sentiments, I now have a mini-project at work that I cannot develop for because they've broken the eclipse plugin. I've managed to compile with an older scala, but this means I've lost my dev environment. I've heard someone say, if you stay on the bleeding-edge, expect to bleed. I guess I was hoping scala was further along in maturity then this. Oh well, back to contributing bug-fixes/enhancements in my spare time and secretly hoping we can use this for future development.
  
• I managed to get Archiva up and running on ubuntu. For some reason (probably my stupidity), I had a difficult time figuring out where the common/lib file was. (Archiva requires derby, mail and activation in common/lib). For those of you like myself, common/lib is under /usr/share/tomcat5.5 NOT /var/lib/tomcat5.5. Anyway, now that I have archiva + hudson + trac up and running, it's time for a really sweet development environment. I'm working on trying to wire at least hudson into our project at work. I'll probably be posting what's useful and what to avoid later.

Also I'd like to take this time for a mini-rant.
<rant>
I'm really confused here. I got on the #eclipse-dev list serve to ask some questions about the eclipse PDE project. I received answers on where to look to solve my issues. I got on the #scala list serve to ask some general architecture questions, and was told to look at some haskell apps because Scala is impure (Note: this is my wording). Ok, the haskel App engine thing looked pretty cool, but I couldn't get past the syntax...(I'd rather read LOLCode all day). How many key contributors to Scala feel it's a "less-good" Haskell? I know a lot of libraries are inspired by Haskell, but not a lot of libraries appear to be inspired from other JVM languages. This confuses me, as I thought Scala was targeted towards Java developers. There aren't a lot of Helper routines to make using Scala with existing java code/libraries easier.

Example: I'm integrating with a bunch of Java-Threaded code. It uses Runnable and callable. I created a JavaThreadConversion object that looks like this:

object JavaThreadConversion
{
  def functionToRunnable(f : => Unit) = new Runnable() {
     override def run() = f
  }
  import java.util.conncurrent.Callable
  def functionToCallable[A](f : => A) = new Callable() {
     override def call() = f
  }
}

That little bit of code save me a lot of typing when interfacing with my old java libraries.

Why don't you rewrite them in scala Not enough time/money/priority. Not gonna happen any time soon.

Oh well. Hopefully as more Java developers see the power of scala, they'll begin to contribute more utilities that can make transitioning off java easier. Until then, I'll see what of mine really is useful and can pass the "functional purity" filter.

Amusing note: At point I wondered why there weren't any Scala + OSGi helpers as I hear a lot of people asking how to do XYZ scala task using OSGi. I'd like to see how much that make a FP purist cringe... OSGi seems to be the bane of FP. Even worse would be the mixing of a Scala core with a Groovy plugin system...
</rant>

Anyway, don't mean to offend anyone. Just frustrated with certain things, including my slow-learning-process for various FP concepts (like monads)

Statically Duck-Typed (Take 2)

So I've been reading the "Programming in Scala" book in my spare time. There's a lot of neat patterns to learn from the Scala language. Anyway, I was reading about curried functions and the loan pattern, and decided to write an updated version of my "doThenClose" helper method.

To accomplish this, we want to change the "doThenClose" method (which was on a "RichCloseable" object). Here's what we need to do:

• Rename to "withResource"


• Make doThenClose global


• Make it take in a "resource" or "closeable"


• Modify the type parameters so we don't lose the original type of the closeable. (This way we can "loan" it to the function that uses it


• Modify the type of functions we take to those that operate on the type based in and return something

Here's our original "doThenClose" method

/**
   * Defines new scala-awesome features we want.
   */
  class PimpedCloseable(closeable: MyCloseable) {
     /**
      * Executes a function, then closes the resource.
      */
     def doThenClose[A](f : => A) : Option[A]= {
      try {
        return Some(f)
      } catch {
      case _ => System.err.println("Trouble executing", f);
      } finally {
        CloseUtils.close(closeable);
      }
      return None;
    }
  }

And here's the new version:

  def withResource[A <: Closeable, B](closeable : A)( f : A => B) = {
    try {
      f(closeable)
    } finally {
      CloseUtils close closeable
    }
  }

First of, let's describe the type parameters. "A <: Closeable" means "anything meeting the closeable interface" and "B" is the return type of the withResource method (which we infer from the argument that will use our resource). Also remember that Closeable is a TYPE not a class (i.e. type Closeable = { def close() : Unit } ). This is where the "static duck-typing" comes in.

Now, the method (withResource) defines what appears to be two argument lists. What this is really doing is creating two functions, the first generates the second. This is the function currying. The result of this, is that when calling the function, you use two argument lists as well. Because scala performs some syntactic sugar for 1-argument functions, we can use {} instead of () to surround either of the arguments. What this means is we can write the following test application:

object BlogTest {
  def main(args : Array[String]) : Unit = {
    import CloseUtils._
    import java.io._
    withResource(new FileInputStream("src/com/suereth/CloseUtils.scala")) {
      file =>
       val reader = new BufferedReader(new InputStreamReader(file))
       var line = reader.readLine()
       while ( line != null ) {
         System.out.println("" + line.trim.size + ":" + line)
         line = reader.readLine
       }
    }
  }
}

Notice the second argument list to "withResource" is { file => ...}. This is a function that is "loaned" the resource created. The function can do whatever it wants to the resource, however when it exists (via return or throw), the resource *will* be closed. Although I'm not a fan of the "resource =>" syntax, it's still a pretty amazing feature *and* totally user-defined.

Trac Updates

A few updates on setting up your own Trac...

First I finally got around to install plugins to Trac. I've got graphviz, web admin, and a new theme. Here's a few things that caught me up.

First of all, I used easy_install (which I guess techinically isn't required. You need to

sudo apt-get install python-setuptools

After which you can easily type:

sudo easy_install http://suversion-repo/for/trac/plugin

I found a good theme manager here: trac-hacks.org along with several themes that just improve the usage of trac.

After installing a plugin you need to enable it like this in the trac.ini file for your project:

[components]
webadmin.* = enabled
graphviz.* = enabled

(just keep adding the plugin name, .* = enabled)

Each plugin may require other kinds of configuration.

I'm also including an email I received from my friend John Sweeney. John lists how to install Trac on slackware (yes, John could be a masochist). Also, John's shows the apache configuration to enable basic logins and set up passwords. THANKS AGAIN JOHN! Here's the email:

Josh,

I tried to post this in a comment to your trac/subversion/apache post (http://suereth.blogspot.com/2008/05/setting-up-ubuntu-development-server.html), but i couldn't get the formatting right, so here goes:

After (mostly) following the steps in the post, I was able to set up my Trac site as a subdomain, including Apache authorization for webadmin. This is how I did it on Slackware 12.0 (you've been warned):

1. Set up your apache passwords file:
   
   cd /path/trac/
   htpasswd -c trac.htpasswd john
   


2. Set up webadmin (it's built into Trac-0.11, which i installed) by adding the following to /path/trac/siteName/conf/trac.ini:
   
   [components]
   webadmin.* = enabled
   
   
   Note: Ubuntu 8.04 installed trac 0.10, so you need to install the web admin egg first


3. Configure apache (could be in httpd.conf or extras/httpd-vhosts.conf):
   
   NameVirtualHost *:80
   
   <VirtualHost *:80>
   ServerAdmin me@mysite.com
   DocumentRoot /path/trac/siteName/
   ServerName trac.mysite.com
   
   
   WSGIDaemonProcess siteName user=apache group=apache threads=25
   # "apache" above may be "httpd" or something else depending on your config
   WSGIScriptAlias / /path/trac/siteName/apache/main.wsgi
   
   <Directory />
   WSGIProcessGroup siteName
   WSGIApplicationGroup %{GLOBAL}
   SetEnv trac.env_path /path/trac/siteName
   Order deny,allow
   Allow from all
   </Directory>
   
   <LocationMatch "/login">
   AuthType Basic
   AuthName "Trac"
   AuthUserFile /path/trac/trac.htpasswd
   Require valid-user
   </LocationMatch>
   </VirtualHost>
   


4. Add your user (from apache passwd file) to the admin list:
   
   trac-admin /path/trac/siteName permission add john TRAC_ADMIN
   trac-admin /path/trac/siteName permission list


5. Restart apache
   
   
   /etc/rc.d/rc.httpd restart # again, running Slackware...


6. Open your Trac site in your favorite browser and rejoice.

Enjoy.

Ubuntu Dev Server - Hudson

I've decided to switch from using Continuum for Continuous Integration to Hudson. This is based on a suggestion from a friend. We'll see how things turn out.

So first, I'm going to list here (because I continually have to look it up) how to install tomcat5.5 and apache2 on Ubuntu server and wire the mod_jk forwarding up.

First, install apache, tomcat and mod_jk

sudo apt-get install apache2 tomcat5.5 libapache2-mod-jk

Next create a workers.properties file (mine is in /etc/apache2/workers.properties). Here's what mine looks like:

workers.java_home=/usr/lib/jvm/java-6-sun
worker.list=worker1
worker.default.port=8009
worker.default.host=localhost
worker.default.type=ajp13
worker.default.lbfactor=1

Next, make sure there's a sym-link in /etc/apache2/mods-enabled to /etc/apache2/mods-available/jk.load.

Now that we know mod-jk is loaded on apache startup, it's time to actually set up apache to forward appropriately. I edited my apache2.conf file (instead of available-hosts/default). Why? No real reason. In fact this should probably be moved into available-hosts/default, but for now I'll just show you the relevant apache configuration to write in the worker you specified above:

#Place this somewhere in the configuration after the mod_jk is loaded (or the jk.load file is included)
# Tomcat Configuration
JkWorkersFile /etc/apache2/workers.properties
JkMount /hudson/* worker1

Note that I'm only mounting the hudson directory. This means that I'm only forwarding to tomcat for the tomcat application. This is because I'm using apache to host my subversion repository AND trac AND hudson (and archiva in the future).

Finally, we have to turn off (or configure) java security for tomcat 5.5 in ubuntu. I'm turning it off because I'm lazy and I don't plan to place my dev box outside my internal network. The basic just to turn it off is to open your /etc/init.d/tomcat5.5 file and look for the following line: TOMCAT5_SECURITY=yes. Change the "yes" to "no" and you should be ready to go. For more details check the link here.


Installing Hudson

First of, download the hudson war into tomcat's webapps directory -

cd /var/lib/tomcat5.5/webapps
sudo wget http://hudson.gotdns.com/latest/hudson.war

Next, set up a directory for hudson. I made one in tomcat's directories:

cd /var/lib/tomcat5.5
sudo mkdir hudson
sudo chown tomcat5.5 hudson

Now, you need to add the following line somewhere in the beginning of /etc/init.d/tomcat5.5 -

JAVA_OPTS="${JAVA_OPTS} -DHUDSON_HOME=/var/lib/tomcat5.5/hudson"

Now, restart tomcat and apache; Hudson should be working at http://youraddress/hudson/

Adding Scala Actors to your Spring application

So this morning I came up with a solution to something that has been bugging me about Scala actors recently. This fix solves two issues:

• Type-safe interfaces for actors
• wiring up actors (decoupling them)

The basic idea follows this pattern. Say, for instance, I have the rather unhelpful "EmailService" interface defined in my spring application:

public interface EmailService {
   public void sendEmail(String address, String subject, String message);
   public boolean isValid(String address);
}

Let's say (initially) I want to migrate all my service calls to scala actors. The goal here is to start pipelining my business logic and because I read a white paper about Erlang/OTP and wanted to "get some" into my J2EE application. Well, since I already have a java interface, and I already have a spring configuration file (or many), then I should be able to swap out implementations of the EmailService. This is the whole reason for dependency injection right?

Here's our first cut at a implementation using scala actors:

import scala.actors.Actor._

class ScalaActorEmailService extends EmailService {
     //Here we define the actor API (based on the interface)
     case class IsValid(address : String)
     case class SendEmail(address : String, subject : String, message : String)

     //Here we define the actor that actually performs service calls
     private val worker = actor {
       loop {
           react {
               case IsValid(address) => reply(false) //TODO Implement
               case SendEmail(address, subject, message) => Console.println("TODO -Implement send");
           }
       }
     }


     //Now we implement the API
     override def isValid(address : String) : Boolean = {
          val response = Some(worker !? IsValid(address))
          response.getOrElse(false) match {
              case b : Boolean => return b
              case _ => return false
          }
     }
     override def sendEmail(address : String, subject : String, message : String) : Boolean = {
        worker ! SendEmail(address, subject, message)
     }
}

Notice how all API calls are actually being sent as messages over actors. If we can ensure immutable arguments to our services... (which may be hard after the fact), you can see how we can add rich concurrent behavior in a piecemeal fashion. Another good part about this technique is that you can NOT pass messages to an actor in the case of "simple" operations.

Let's pretend we profiled and found that the isValid API call was slowing our application down significantly due to the return value. We want to inline in it to see what kind of performance gain we might make. Here's what our service would change into.

import scala.actors.Actor._

class ScalaActorEmailService extends EmailService {
     //Here we define the actor API (based on the interface)     
     case class SendEmail(address : String, subject : String, message : String)

     //Here we define the actor that actually performs service calls
     private val worker = actor {
       loop {
           react {
               case SendEmail(address, subject, message) => Console.println("TODO -Implement send");
           }
       }
     }


     //We inline this method for performance
     override def isValid(address : String) : Boolean = {
          //TODO - Implement
          return false
     }
     //This method calls out to the actor to make sure we don't block on send
     override def sendEmail(address : String, subject : String, message : String) : Boolean = {
        worker ! SendEmail(address, subject, message)
     }
}

Now we're starting to have some power in how we can architect our system. Not only that, we're dealing with type safe interfaces, so we don't have to search through documentation to figure out what messages can be sent to what actors. We can also remove/add actor calls as necessary for our app.

Another thing I'd like to say, and this is totally my preference. When in Java land and initially learning spring, a *lot* of examples show dependency injection using setters. I actually am starting to think this is a bad idea, as it makes mis-constructing an object (or mis-configuring it within the spring container) *much* easier. It seems to be a safer (and perhaps more functional?) way of doing things to inject everything in the constructor. In this manner we can inject type-safe interfaces for other actor-based-services into our actor-based service! Instantly we start talking pipelines and scalability based on hardware (i.e. # of CPUs).

The last thing to do, of course would be to attempt to unify choice of languages into scala by creating a trait instead of an interface that can be used.

trait EmailService {
   //Perfectly abstract method
   def sendEmail(address : String, subject : String, message : String) : Unit
   def isValid(String address) : Boolean = {
     //We could implement the "basic" version here and override it later if desired
     false
   }
}