Runa is hiring developers

We, Runa (runa.com), are looking for great developers to join our small team. We’re an early stage, pre-series-A startup (presently funded with strategic investments from two large corporations) playing in the e-commerce space. We’re creating a new product in the small-to-medium online-retailing segment, and if we’re successful, it will be a very large disruption.

Techie keywords: clojure, hadoop, hbase, rabbitmq, erlang, ruby, rails, javascript, amazon EC2, unit-testing, functional-testing, selenium, agile, lean, XP

If you’re interested, email me at amit@runa.com

If you want to know more, read on!

What we do

Runa aims to provide small-to-not-so-large online retailers with tools/services that companies like amazon.com use/provide. These smaller guys can’t afford to do anything on that scale, but by using our SaaS services, they can make more money while providing customers with greater value.

The first service we’re building is what we call Dynamic Sale Price.

It’s a simple concept – it allows the online-retailer to offer a sale price for each product on his site, personalized to the individual consumer who is browsing it. By using this service, merchants are able to –

  • increase conversion (get them to buy!) and
  • offer consumers a special price which maximizes the merchant’s profit

This is different from “dumb-discounting” where something is marked-down, and everyone sees the same price. This service is more like airline or hotel pricing which varies from day to day, but much more dynamic and real-time. Further, it is based on broad statistical factors AND individual consumer behavior. After all, if you lower prices enough, consumers will buy. Instead, we dynamically lower prices to a point where statistically, that consumer is most likely to buy.

How we do it

Runa does this by performing statistical analysis and pattern recognition of what consumers are doing on the merchant sites. This includes browsing products on various pages, adding and removing items from carts, and purchasing or abandoning the carts. We track consumers as they browse, and collect vast quantities of this click-stream data. By mining this data and applying algorithms to determine a price point per consumer based on their behavior, we’re able to maximize both conversion (getting the consumer to buy) AND merchant profit.

We also offer the merchant comprehensive reports based on analysis of the mountains of data we collect. Since the data tracks consumer activity down to the individual product SKU level (for each individual consumer), we can provide very rich analytics. This is a tool that merchants need today, but don’t have the resources to build for themselves.

The business model

For reference, it is useful to understand the affiliate marketing space. Small-to-medium merchants (our target audience) pay affiliates up to 40% of a sale price. Yes, 40%. The average is in the 20% range.

We charge our merchants around 10% of sales the Runa delivers. Our merchants are happy to pay it, because it is a performance-based pay, lower than what they pay affiliates, and there is zero up-front cost to the service. In fact, the above mentioned analytics reports are free.

We’re targeting e-commerce PLATFORMS (as opposed to individual merchants); in this way, we’re able to scale up merchant-acquisition. We have 10 early-customer merchants right now, with about 100 more planned to go live in the next 2-3 months. By the end of next year, we’re targeting about 1,000 merchants and 10,000 merchants the following year. Our channel deployment model makes these goals achievable.

At something like a 5% to 10% service charge, and a typical merchant having between 500K to 1M in sales per year, this is a VERY profitable business model. That is, of course, if we’re successful… but we’re seeing very positive signs so far.

And we haven’t even talked about all the other things on our product-roadmap!

Technology

Most of our front-end stuff (like the merchant-dashboard, reports, campaign management) is built with Ruby on Rails. Our merchant integration requires browser-side Javascript magic. All our analytics (batch-processing) and real-time pricing services are written in Clojure. We use RabbitMQ for all our messaging needs. We store data in HBase. We’re deployed on Amazon’s EC2.

We need to be extremely scalable and fast. This is for two reasons –

  • The prices are offered to consumers in real-time, as they browse. There can be no delay.
  • The load on our service grows quickly – each time we sign on a merchant, we’re hit with the traffic from all their customers

It is a very challenging problem, and a lot of fun to solve.

Here are a few blog postings about what we’ve been up to –

We’ve also open-sourced a few of our projects –

  • swarmiji – A distributed computing system to write and run Clojure code in parallel, across CPUs
  • capjure – Clojure persistence for HBase

Culture at Runa

We’re a small team, very passionate about what we do. We’re focused on delivering a ground-breaking, disruptive service that will allow merchants to really change the way they sell online. We work start-up hours, but we’re flexible and laid-back about it. We know that a healthy personal life is important for a good professional life. We work with each other to support it.

We use an agile process with a lot of influences from the Lean and Kanban world. We use Mingle to run our development process. Everything, OK mostly everything 🙂 is covered by automated tests, so we can change things as needed.

We’re all Apple in the office – developers get a MacPro with a nice 30” screen, and a nice 17” MacBook Pro. We deploy on Ubuntu servers. Aeron chairs are cliché, yes; but, very comfy.

The environment is chilled out… you can wear shorts and sandals to work… Very flat organization, very non-bureaucratic… nice open spaces (no cubes!). Lunch is brought in on most days! Beer and snacks are always in the fridge.

We’re walking distance to the San Antonio Caltrain station (biking distance from the Mountain View Caltrain/VTA lightrail station).

What’s in it for you

  • Competitive salaries, and lots of stock-options
  • Cutting edge technology stack
  • Fantastic business opportunity, and early-stage (= great time to join!)
  • Developer #5 – means plenty of influence on foundational architecture and design
  • Smart, fun people to work with
  • Very comfortable, nice office environment

OK!

So, if you’re interested, email me at amit@runa.com

Startup logbook – v0.1 – Clojure in production

Late last night, around 3 AM to be exact, we made one of our usual releases to production (unfortunately, we’re still in semi-stealth mode, so I’m not talking a lot about our product yet – I will in a few weeks). There was nothing particularly remarkable about the release from a business point of view. It had a couple of enhancements to functionality, and a couple of bug-fixes.

What was rather interesting, at least to the geek in me, was that it was something of a technology milestone for us. It was the first production release of a new architecture that I’d been working on over the past 2-3 weeks. Our service contains several pieces, and until this release we had a rather traditional architecture – we were using ruby on rails for all our back-end logic (eg. real-time analytics and pricing calculations) as well as the user-facing website. And we were using mysql for our storage requirements.

This release has seen our production system undergo some changes. Going forward, the rails portion will continue to do what it was originally designed for – supporting a user-facing web-UI. The run-time service is now a combination of rails and a cluster of clojure processes.

When data needs to be collected (for further analytics down the line), the rails application simply drops JSON messages on a queue (we’re using the excellent erlang-based RabbitMQ), and one of a cluster of clojure processes picks it up, processes it, and stores it in an HBase store. Since each message can result in several actions that need to be performed (and these are mostly independent), clojure’s safe concurrency helps a lot. And since its a lisp, the code is just so much shorter than equivalent ruby could ever be.

Currently, all business rules, analytics, and pricing calculations are still being handled by the ruby/rails code. Over the next few releases we’re looking to move away from this – to instead let the clojure processes do most of the heavy lifting.

We’re hoping we can continue to do this in a highly incremental fashion, as the risk of trying to get this perfect the first time is too high. We absolutely need to get the feedback that only production can give us – so we’re more sure that we’re building the thing right.

The last few days have been the most fun I’ve had in any job so far. Besides learning clojure, and hadoop/ hbase pretty much at the same time (and getting paid for doing that!), it has also been a great opportunity to do this as incrementally as possible. I strongly believe in set-based engineering methods, and this is the approach I took with this as well – currently, we haven’t turned off the ruby/rails/mysql system – it is doing essentially the same thing that the new clojure/hbase system is doing. We’re looking to build the rest of the system out (incrementally), ensure it works (and repeat until it does) – before turning off the (nearly legacy) ruby system.

I’ll keep posting as I progress on this front. Overall, we’re very excited at the possibilities that using clojure represents – and hey, if it turns out to be a mistake – we’ll throw it out instead.

Software development agility – bottom-up design and domain-specific languages (DSLs)

It’s been nearly three months that I’ve been working at Runa. And in that time I’ve worked with the team to get a lean and agile process in place, and while we’re not there yet, things are getting more and more focused every day.

I’ve also been working to significantly re-factor the code that existed, as we get ready to get the first beta release out the door. In this post, I’d like to talk about one aspect of our design philosophy which I’m glad to say has been working out really well – bottom-up design (and an associated DSL).

Before we decided to change the system, it had been designed in the usual top-down manner. Runa operates in the online retail space – we provide web-based services to online merchants, and while we have a grand vision for where we want our platform to go, our first offering is a promotions service. It allows merchants to run campaigns based on a variety of criteria. Merchants are a fussy lot, and they like to control every aspect of campaigns – they want to be able to tweak all kinds of things about it.

In true agile fashion, our initial release allows them to select only a few parameters. Our system, however, needs to be extensible, so that as we learn more about what our merchants need, we can implement these things and give it to them. Quickly. And all of this needs to be done in a changing environment with lots of back-and-forth between the business team and the dev team.

So here’s what we came up with – it is an example of what we call a campaign template –

campaign_template :recapture_campaign do
  title is 'Recapture'
  subtitle is 'Reduce cart abandonment rate'
  description is 'This campaign presents offers to shoppers as they abandon their shopping cart.'

  #adding criteria here
  accept required time_period_criteria with defaults start_date('1/1/08'), end_date('12/31/10')
  accept required product_criteria with no defaults

  hardcode visit_count_criteria with number('1')

  #more criteria
  reject optional url_referrer_criteria with no defaults

  inside context :view_badge do
    never applicable
  end

  inside context :abandon_cart do
    allow only customer_type_criteria with customer_type('visitor')
  end

  inside context :cart do
    allow only user_action_criteria with user_action('accepted_recapture')
  end

end

A campaign template behaves like a blue-print of an actual campaign. Sort of like the relationship between a class and an object. In a sense, this is a higher-order description of just such a relationship. The (computer) language now lets us speak in the domain of the business.

There are a couple of reasons why our core business logic is written like this –

a) It lets us communicate easily with the business. Whenever a question about a rule comes up, I bring up the associated template up on the screen, and make them read the ‘code’. Once they agree thats what they mean, it just works, because it is real code.

b) Since this is in essence a way to script the domain model, it has forced a certain design upon it. All the objects evolved in a bottom-up manner, and each does a very specific thing. It lends to a very highly de-coupled design where objects collaborate together to achieve the higher goal, but each is very independent of the other.

c) This notation makes several things easier. One, the actual business rules are described here, and they just work. The other thing is that we’re able to use this same representation for other things – for example, our merchant GUI is completely auto-generated off these template files. Menu items, navigation, saving, editing, error-reporting, everything is generated.

This allows very fast turn around time for implementing new concepts, or making changes to existing ones.

It’s an internal DSL written in Ruby, and does whatever it can without any extra parsing, as you can probably imagine. I will write about the specifics of how this is implemented in future posts. For the moment, I would like to stress, however, the importance of the bottom-up approach. Because our domain model is made up of many small objects (instead of a few larger ones), each representing a tiny aspect of the domain, we’re able to put them together and fashion more complex beasts out of them. And the combinations are limitless, bounded only by business rules. This is where the power of this approach lies.

The DSL itself is almost only a convenience!

Easy external DSLs for Java applications

Or JRuby for fun and profit

I’ve been developing software for some time now, and have recently found myself applying ideas from various esoteric areas of computer-science to every day tasks of building common-place applications (often these concepts are quite old, indeed some were thought of in 1958).

One powerful idea that I’ve been playing with recently is that of embedding a DSL (domain specific language) into your basic application framework – and writing most of the features of the application in that DSL.

This is simply an implementation of the concept of raising the level of abstraction. The point, of course, being that when writing code in a DSL implemented in such a fashion, one can express ideas in terms of high-level abstractions that represent actual concepts from the problem domain. In other words, it is like using a programming language that has primitives rooted in the domain.

A lot of people have been writing about this kind of software design – and most implement these ideas in a dynamic language of their choice. How does one go about doing the same in a language like Java? That is what this article is about. And I cheat in my answer. Consider the following design stack –

Creating DSLs in JRuby

I propose that you only implement basic and absolutely required pieces of functionality in Java – the things that rely on, say, external systems that expose a Java interface, or some EJB-type resource, or some other reason that requires the use of Java. The functionality you develop here is then exposed through an interface to the layers above. You can also add APIs for other support services you might need.

The layer above is a bunch of JRuby code that behaves as a facade to the Java API underneath. This leaves you with a Ruby API to that underlying Java (and other whatever, really!) stuff – and makes it possible to code against that functionality in pure Ruby. The JRuby interpreter runs as part of the deployable and simply executes all that Ruby code transparently. As far as your Ruby code is concerned, it doesn’t even care that some of the calls are internally implemented in Java. Sweet!

We can stop here. At this point, we are in a position to write the rest of our application in a nice dynamic language like Ruby. For some people, a nice fluent interface in Ruby suffices – and keeps all developers happy. This is depicted on the top-left part of the diagram.

However, we can go one step further, and implement a DSL in Ruby that raises the level of abstraction even more – as referred to earlier. So on top of the Ruby layer, you’d implement a set of classes that allow you to write code in simple domain-like terms, which then get translated into a form executable by the Ruby interpreter. This is shown in the top right part of the diagram. Ultimately, potentially any one (developers, QA or business analysts) could express their intent in something that looked very much like English.

So where to write what?

How much to put in your Java layer depends on the situation – some people (like me) prefer to write as little as possible in such a static language. Others like the static typing and the associated tool support, and prefer to put more code here. When merely shooting for a little bit of dynamism through the DSL engine in the layers above, most of the code could be written in Java, and a fluent API in the dynamic language could be enough. When shooting for rapid feature turn-around and a lot more flexibility, most of the code could be in the DSL or in the external dynamic language.

The answer to this question really depends on things like the requirements, team structure, skill-sets, and other such situational factors.

OK, so where’s the code?

My intention with this post was to stay at a high level – and talk of how one could structure an application to make it possible to embed a scripting engine into it, and to give an overview of the possibilities this creates. In subsequent posts, I will talk about how actual DSLs can be created, tested, and also how a team might be structured around it.

Gotham Ruby Conference 2007

I attended GoRuCo 2007 yesterday, and it was a fine conference – where I met a bunch of old friends I hadn’t seen in a while. Here’s what the agenda looked like – I enjoyed the talks on Adhearsion, JRuby, and BNLs. And the conference took place in Google’s offices in NYC, and that was neat.

It was cool how many people knew of ThoughtWorks, and were interested in talking about what it was like to work there. (Probably less than the number that were interested in Google in the same way, but hey…) We ended the conference by finishing up with the afterparty in the wee hours of the morning. It was fun! Amazingly, both Deepthi and I won swag at the event – I got this neat laptop bag, and Deepthi got Rails Recipes. Oh, and here are some pictures.

You might want to read some live-blogging from the event.

Ruby, managing global variables, and dynamic scope

Everyone knows that global variables are bad. However, they are quite unavoidable. Java’s System.out is an example.

Globals, sometimes offer a certain kind of flexibility. They offer a way to tweak the behavior of the entire (or a subset of) the system. For instance, one can redirect System.out to a different stream (into a file, say) and capture all the messages a program spits out.

This kind of stuff works nicely. Except when someone else goes and changes the same variable to something else in another part of the code, and you’ve no idea where.

What is needed, is something like optional dynamic scope, so that the globals can be used when needed, but can be managed better. After all, from the above example, what seems to be needed is a way for a piece of code to say – for my purposes, and for all code that runs when I’m called, I want the value of this global(s) to be _something_, and when I return, these globals should be reset.

This can be done manually, by saving the existing value of a global before setting it to something else, and then resetting it back when the code block completes running. Perl and Common Lisp have had a mechanism to do this type of stuff for a long time, built into the language.

Here’s a hacky (and probably naive) way to implement this in Ruby, to illustrate how this might work –


module Let
def let(bindings, &block)
old_bindings = capture_existing_bindings_for(bindings)
block.call
rehydrate_old_bindings_with(old_bindings)
end
def capture_existing_bindings_for(bindings)
old_bindings = { }
bindings.each do |k, v|
old_bindings[k] = eval "@"+k.to_s
create_binding k, v
end
return old_bindings
end
def create_binding(var_name, value)
instance_eval "@#{var_name}=value"
end
def rehydrate_old_bindings_with(old_bindings)
old_bindings.each do |k, v|
create_binding k, v
end
end
end

And the way you would use this, would look something like this –


require 'let_module'
include Let
@num_var = 1
@char_var = 'a'
class Car
attr_reader :name
def initialize(name)
@name = name
end
end
@obj_var = Car.new("hyundai")
def do_something
puts "num_var is " + @num_var.to_s + ", char_var is '" + @char_var.to_s + "', obj_var is " + @obj_var.name
puts "returning"
end
do_something
puts "changing num_var to 2, char_var to b, obj_var to 'kia'"
let :num_var => 2, :char_var => 'b', :obj_var => Car.new("kia") do
do_something
puts "changing num_var to 3, char_var to c, obj_var to 'toyota'"
let :num_var => 3, :char_var => 'c', :obj_var => Car.new("toyota") do
do_something
end
do_something
end
do_something

And when run, would produce this –

num_var is 1, char_var is 'a', obj_var is hyundai
returning
changing num_var to 2, char_var to b, obj_var to 'kia'
num_var is 2, char_var is 'b', obj_var is kia
returning
changing num_var to 3, char_var to c, obj_var to 'toyota'
num_var is 3, char_var is 'c', obj_var is toyota
returning
num_var is 2, char_var is 'b', obj_var is kia
returning
num_var is 1, char_var is 'a', obj_var is hyundai
returning

This could be one way to control unruly global variables in Ruby.

CruiseControl.rb – Continuous integration for ruby projects

cruisecontrol.rb

I downloaded and installed CruiseControl.rb today – from the ThoughtWorks website. Well, I suppose ‘install’ is not quite the right word as it comprised entirely of running a couple of commands, and took all of 30 seconds.

It works beautifully, right out of the box! The example on the website didn’t have any reference to the situation where you connect to your subversion server over ssh, so when I went ahead and tried it, it succeeded and then gave me the following message –

IMPORTANT!!! – It looks like you are connecting to your repository with an svn+ssh connection. For cruise to build this project, you need to have set up authentication caching for ssh, see this article
http://subversion.tigris.org/faq.html#ssh-auth-cache

What a pleasant surprise! I like software that was designed with users in mind. The information above for setting up ssh authentication works like a charm, btw.