The Lagniappe Programming Environment

This is the last post here. I recently found out that a paper we wrote on Lagniappe has been accepted into the 2010 MODELS Conference in October in Oslo, Norway. I have continued to work on Lagniappe after I graduated in August, 2008, and getting this paper in marked the conclusion of the project.

A side note, Blogger discontinued FTP-based blogs. Unfortunately, I built this simple little page on top of that functionality, making it hard to continue the page here as well. (I am currently editing this post by hand.)

A lot of what I have learned from Lagniappe has influenced my current and future research directions, so all-in-all I believe we did something good here. The current implementation works, and it is stable and fast. It can still be downloaded from the google code repository found up in the left side-bar.

Au revoir!

I have my two initial benchmark applications coded up. They still need some testing, and I still need to finalize the workload generation, but having the apps down on the machine is nice.

I haven't touched the multi-machine code in a week. It just got to me after a while. The serialization aspect was killing me. I've decided to use the boost serialization library. At first I didn't think there was that much that I needed serialization for, but I eventually figured out that was not definitely not the case. Using the boost library will allow me to keep drastic changes between the trunk development of Lagniappe and the multi-machine branch, which will hopefully mean that the multi-machine support will be easier to merge back in when it's all said and done.

Labels: benchmarks, multi-machine

While the OSDI deadline has come and gone, I am now making the final push towards defending and turning in my dissertation by the middle of August. Needless to say, this is the most important deadline I've ever run up against and it's not okay to pull the paper for this one.

I am working on two major tasks regarding the main system. The first is making Lagniappe function on multiple machines at the same time. If Lagniappe runs on multiple machines, the true heterogeneity of multi-* becomes an issue (as now we have a drastically different communication channel in the mix). The second task is combining operators based on their state preferences. I feel that we can truly answer the pipelined vs. parallel question based on the state access semantics of the operators in an application.

The third major issue is creating a suite of benchmark applications. The benchmarks has been a thorn in my side for a while, but I am confident we can make it all come together.

Labels: dissertation, multi-machine, operator combination

Post OSDI I'll update the site on all of the new stuff that has been built into Lagniappe in the last month. It's really coming together, but there is still so much to do before the deadline.

Labels: OSDI

So I've checked in code (in both the library and compiler) that automatically generates adaptation triggers based on the specifics of the application and system. I did a quick test and it does, in fact, generate different triggers for different hardware platforms, so that was a nice sanity check.

I've also now officially put all of the code under the GPL.

I'm going to spend the next few days on getting some of the paper up-to-date, and then I will start to tackle the smart mapping issue. With those two features on the books, the system will be pretty powerful. We will automatically generate policies to efficiently adapt resources to handle changes in workload based on your specific application and hardware platform. I think that is pretty powerful, but yes, I'm biased.

Labels: adaptation, mapping, OSDI, writing

Things are starting to gel in my head and I'm starting a long section of implementation. Many features to get up and running and tested. Testing is something that I'm actually going to have to put some thought in. It is easy to test particular features in isolation, but the real trick in this work is watching the system deal with large-scale changes in incoming traffic. Generating the traffic that will properly trigger this activity and model real-life traces will be challenging.

The machines are here and installed in the machine room. They should be completely functional soon.

I will try to discuss the implementation as I go along.

Labels: new hardware, OSDI, traces