How should traits in Scala be used?

This blog is a survey of the way I've seen traits used in various scala projects (including my own). The purpose of this article is to explore the new-found (at least to those coming from java) power of traits and determine what works well and what the cost/benefits are of each. The taxonomy used in this Blog is non-standard. I believe if I read more of the academic papers, I'd know the real names attributed to these usages, but for now I believe the names chosen will suffice to describe the concepts. (If anyone knows appropriate references to articles/papers please comment with links!)


Traits as Interfaces
This is the most-natural usage of traits for Java developers. This is where you define a trait that has no method definitions (only declarations). The traits then act merely as interface definitions, and allow developers to code to the interface of a class, rather than any specific implementation of it. I like to use these when defining entry points for third-party implementers to use when entering the "domain" of a given library. It allows my library to interact with the users classes without having to know anything about their implementations. I also like to use this for high-level constructs I pass out of my library, as it keeps the user of the library from knowing too much about the specifics of the implementation.

An example of this comes from my earliy post about writing a Scala DSL for Database Migrations. Here's the trait that was passed to all migration scripts in the DBMigrationEngine:

trait MigrationContext {
  
  def createTable[A](name : String)(creatorScript : TableCreator => A) : A
  def executeScript(queries : Function[DbVendor,String]) : Unit
  
}

I'm not sure what all to say about these, except that they provide nice clean interfaces between larger modules of your system. They can also be compilation barriers (assuming they don't change frequently and you manually specify them in return types... I know sometimes I get lazy here with scala's nice type inference).

Traits as Abstract Classes
This usage comes naturally to those of us coming at Scala from C++. A trait can be treated as an abstract virtual class with all virtual methods. This means we can proivde a very rich interface for a class, while only requiring subclasses to implement one or two "pure virtual" (or undefined) methods. The Scala collections libraries have good examples of traits as Abstract classes. Let's look at the Seq class in Scala.


The Seq class in scala implements all sorts of nice collections functions, flatMap, map, foreach, isEmpty, etc. However to create your own sequence, you only need to override the following methods:

  def length: Int
  def elements: Iterator[A]

This is perhaps the style of traits that I find myself developing most for my various projects. It has all the familiarity of a Java abstract class, but all the power of real multiple-inheritance.


Traits as Bolt-Ons
This is where trait self types really find their use. The goal of a bolt-on trait is to add functionality (or "bolt on") to an existing class. Bolt-on traits are very similar to decoraters in that they "decorate" existing classes with functionality. As you can create mixin traits when instantiating objects, it has almost as much flexibility in construction as Decorators, only methods don't need to pass down to delegates. Bolt-on traits serve more purposes than just decoration though (as the above example doesn't quite fit the decorator pattern).

In my own projects, I've created two very simple bolt-on traits that I think show case the idea. These two traits are "InitializingSpringActorBean" and "DisposableSpringActorBean". The basic of just of these traits are that they are to be mixed in *only* with Actor subclasses. They implement the "InitialzingBean" and "DisposableBean" interfaces of Spring in the context of actors. Here's the source code for the InitializingBean:

/** 
 * This is a trait which will start an actor after a bean's properties have been set.
 */
trait InitializingSpringActorBean extends InitializingBean { 
  self : Actor =>  
  override def afterPropertiesSet() {
    start
  }
}

Notice the usage of the self type. This ensures that the trait can only be mixed into an actor class, and (IMHO) is what makes it a "bolt-on" trait. This particular trait implements an interface from spring that enables the Spring BeanContext (or IoC Container) to call "start" on an Actor at the correct point in the Spring lifecycle. Typical usage of this would be:

/**
 * Some kind of bean the emulates business logic.
 */
@Service("CoffeeBean")
class CoffeeBean extends Actor with InitializingSpringActorBean {
  ... Injected properties...
  override def act() {
        ...
  }
  //No need to call start as spring knows how/when to start us now!
}

The thing to always remember about traits, is that when editing one that defines methods, you must recompile all the *subclasses* of the trait as well. This is because the mangling Scala uses to implement traits duplicates the methods in each of its concrete subclasses. So far, most of my bolt-on classes have been rather simple, and do not need to be constantly tweaked/recompiled, but this is a valid concern for a large project. As in C++, I would reserve bolt-on traits for cross-cutting concerns or specific applications. (The most commonly used "bolt-on" from my C++ days was boost::non_copyable, which luckily for me, hardly ever changed! However the issues are the same in Scala as they were in C++ [assuming all code resides in header files])

Traits as Class Ecosystems
This is perhaps one of the neatest (and dangerous) usage of traits I've seen so far in Scala. This is where a trait contains one or more definitions of classes that operate together. These classes are all used to solve a particular problem. If trait includes some high level methods to call into it ecosystem, it really starts to resemble a facade pattern. However, there are many other uses of this, such as the Virtual Class pattern (shown off in the eclipse plugin). This use of traits (once again in my opinion) has the potential for pure awesome and the greatest potential for pure evil. Here is actual Ecosystem trait code with the code removes to help protect the innocent (or guilty?)

trait Files {
   class FileDependencies{
        class Tracker extends OpenHashMap[File, Set[File]]{...}
        ...
   }
   object FileDependencies {...}
   case class File ... {...}
}
object Files extends Files

You can see that the above ecosystem consists of a FileDependencies class, a FileDependencies Object and a File class. The ecosystem trait also provides convenience methods for using the above classes (not shown above). In the above example, the trait actually provides some implicit conversions for "files" form another ecosystem (namely the java standard library) to be automatically converted into this ecosystem. Finally (and important to me) is there is a static Object provided from which one can gain access to the ecosystem *without* mixing in the trait (i.e. import Files._ works somewhat similarly to extends Files). Ecosystem traits also take form in two interesting ways in the Scala Compiler and the Scala Eclipse Plugin.

I'm going to hold off discussing the Scala compiler because it deserves its own blog entry. The Scala Compiler uses what is known as the "Cake" pattern. Martin Odersky (creator of Scala) outlines the Cake Pattern in his description of how the scala compiler was creating in his paper "Scalable Component Abstractions." The paper is quite worth the read, and you should all google for it and read it RIGHT NOW. However, the pattern isn't without critics. I've heard it mentioned a few places as the "Bakery of Doom" used in the compiler. Why do some developers love this pattern and others hate it? Well, you'll have to wait for my next entry, as I'd like to really spend some time delving into this.

For now, let's look into an easier (and smaller) code base for trait ecosystem examples: The Scala Eclipse Plugin.


The Eclipse Plugin and Virtual Class pattern.
The Scala Eclipse Plugin makes heavy use of "Virtual Classes". The original author of the plugin (Sean McDirmid) outlines a simple skeleton of the pattern/idiom used when creating Virtual classes in scala here. Virtual Classes originated (AFAIK) in the programming language BETA, and is described here. BETA is an interesting language (for those interested in languages), but the pattern itself holds interest to us in Scala, and is used a great deal in the Eclipse Plugin.

The main benefit of the Virtual Class "pattern" is that you can layer not only your application, but your "types" or classes". At the lower levels, the class only contains methods needed by lower levels, and as you move up the inheritance chain, the Class "fills out" with all the necessary method. This helps create a minimal interface for the lower levels, while providing a robust interface for the higher levels.

The virtual class pattern is great when you need to slowly build a type in a variety of layers, and slightly change behavior in each layer. However, it does have some downsides. One of which is that unit testing partial types isn't a walk in the park. You actually need to create a "final" layer for the virtual class and unit test that. You end up doing this for every layer in the pattern. As creating a new layer means implementing the entire ecosystem, you're no longer really unit tests, as you are integration testing. I believe as long as you *do* testing, it shouldn't affect the overall stability of your project.

The real issues show up in terms of sprawl. I believe an inherit weakness in using virtual classes is the desire to shrink your layers into minimal sets of functionality (a good thing) and then re-use layers in many many different places (perhaps not so good a thing). The virtual class layers really need well-defined boundaries in the system and preferably low-level (or facade-like) entry points to keep them loosely coupled. If you decide to create virtual-classes for things like lists and strings, it would be very hard to loosely couple this code. (Reminds me of private inheritance from C++) The eclipse plugin really showcases the issues involved in designing with virtual classes (as well as showcasing the benefits). The plugin actually implemented a much faster (but not 100% correct) algorithm to improve recompilation speeds for itself. This is because of the massive amount of trait-inheritance sprawl in the plugin.

To illustrate my point, I've tried to make a few Ven diagrams of the "ecosystem traits" in the eclipse plugin. The names of each "bubble" are real trait names. I've left out showing the internal ecosystem classes, as the diagram gets very cluttered very quickly. In terms of a Ven Diagram, you can assume that any trait (or bubble) encapsulating another bubble mean "inheritance", as inheritance of "ecosystem" traits involves merging the two, well, ecosystems of objects. Here's the diagram just for the lampion.core package:



Notice how most of the ecosystem traits extend each other before reaching the RangeTrees trait. This is a side effect of using the Virtual Class pattern. Each trait partially defines a type with the outer ecosystem traits refining the ones from the inner ecosystem, as well as adding new types. Things get slightly more confusing when we bring in one more scala eclipse plugin package (and this is the furthest I'll go for clarity).

The next diagram not only has the above ven diagram features, but I've also added thick black lines to denote "is-the-same-as" and dashed lines to denote package boundaries. As scala promotes multiple inheritance this idea of one ecosystem encapsulating another gets interesting, as it's possible to merge the same ecosystem from different parent ecosystems. You'll see interesting code spattered around the eclipse plugin to handle this. Here's my representation of the lampion.core and lampion.compiler plugin:




As you can see besides a horrible color selection, things in the plugin are really getting interesting. The Parsers trait inherits every ecosystem class from lampion.core (except for Plugin). The consequences are any change to any class (even internal to an ecosystem) involve recompiling parsers (and therefore typers and tokenizers... not to mention their subclasses). Once again, realize that the eclipse plugin defines its own dependency analysis to recompilation to attempt to bring its cost down to one that is acceptable for development. This seems to me a like a "bad smell" to on of the pieces in the plugin, and I'm pointing at the over-use of virtual-classes (and traits in general) in the plugin.

My biggest complaint (as stated before) with the design (as it exists in the eclipse plugin) is there are no boundaries for the Virtual Class pattern. As you can see, if I were to ven diagram the eclipse plugin, almost all classes would be encompassed be a few high-level classes, with minor amounts of utility code/standalone classes here and there. Not only is the barrier to learning the pattern high (without appropriate documentation), the maintenance (and recompilation) costs are also high. The virtual-class pattern is useful in the right contexts (and a portion of the eclipse plugin might even have that context), but it seems like more of a golden-hammer approach as it stands, and tends to dissuade me from using virtual classes in plugin enhancements/bug fixes that I try to contribute.

I also believe one reason the eclipse plugin gets far more criticism than scalac (besides lack of design documentation) is the lack of automated testing. Although unit testing (as mentioned above) is rather difficult, integration testing is not, and there should be a very strong integration test suite accompanying the plugin. This would not only prove the virtual class pattern is viable for its current usage, it could also help new developers see how the layers work, and what is expected of each. I would love to see a project that uses the virtual class pattern with a strong set of integration tests to prove my theory correct (that integration testing can make up for the pain of unit testing virtual classes).

Conclusions

If you have any examples of traits being used (either efficiently or horribly) please make some noise! I'd love to get a better sampling to color the descriptions in this blog.

To the Java developer, Scala provides new-found power, and requires a modified approach to design. Coming from C++ to Scala, I've been able to apply quite a bit of knowledge on software architecture, however there are still some things that Functional Programming brings to the table that I need to wrap my head around. I'd like to challenge those that have real-world experience in designing large-scale scala systems to start sharing some knowledge of helpful practices/tips and architectures.