s-expressions

Amit Rathore blogs about software development

Posts Tagged ‘lisp’

conjure – simple mocking and stubbing for Clojure unit-tests

Posted by Amit Rathore on January 24, 2010

Siva and I were pairing on a unit-test that involved writing something to HBase. When Siva said that mocking the call to the save-to-hbase function would make testing easier (a simple thing using JMock, he said), I decided to write a quick mocking utility for Clojure.

Then later, we realized that we wanted to go one step further. The row-id that was used as the key to the object in HBase was generated using system-time. That meant that even if we wanted to confirm that the object was indeed saved, we had no way of knowing what the row-id was. One solution to such a problem is to inject the row-id in (instead of being tightly coupled to the function that generated the row-id). Instead, I wrote a stubbing utility that makes this arbitrarily easy to do.

So here they are – mocking and stubbing – packaged up as the conjure project on github.

The set up

Imagine we had the following functions -

(defn xx [a b]
  10)

(defn yy [z]
  20)

(defn fn-under-test []
  (xx 1 2)
  (yy  "blah"))

(defn another-fn-under-test []
  (+ (xx nil nil) (yy nil)))

Also imagine that we had to test fn-under-test and another-fn-under-test, and we didn’t want to have to deal with the xx or yy functions. Maybe they’re horrible functions that open connections to computers running Windoze or something, I dunno.

Mocking

Here’s how we might mock them out -

(deftest test-basic-mocking
  (mocking [xx yy]
    (fn-under-test))
  (verify-call-times-for xx 1)
  (verify-call-times-for yy 1)
  (verify-first-call-args-for xx 1 2)
  (verify-first-call-args-for yy "blah"))

Pretty straightforward, eh? You just use the mocking macro, specifying all the functions that need to be mocked out. Then, within the scope of mocking, you call your functions that need to be tested. The calls to the specified functions will get mocked out (they won’t occur), and you can then use things like verify-call-times-for and verify-first-call-args-for to ensure things worked as expected.

Stubbing

As mentioned in the intro to this post, sometimes your tests need to specify values to be returned by the functions being mocked out. That’s where stubbing comes in. Here’s how it works -

(deftest test-basic-stubbing
  (is (= (another-fn-under-test) 30))
  (stubbing [xx 1 yy 2]
    (is (= (another-fn-under-test) 3))))

So that’s it! Pretty simple. Note how within the scope of stubbing, xx returns 1 and yy returns 2. Now, for the implementation.

Implementation

The code is almost embarrassingly straight-forward. Take a look -

(ns org.rathore.amit.conjure.core
  (:use clojure.test))

(def call-times (atom {}))

(defn stub-fn [function-name return-value]
  (swap! call-times assoc function-name [])
  (fn [& args]
    (swap! call-times update-in [function-name] conj args)
    return-value))

(defn mock-fn [function-name]
  (stub-fn function-name nil))

(defn verify-call-times-for [fn-name number]
  (is (= number (count (@call-times fn-name)))))

(defn verify-first-call-args-for [fn-name & args]
  (is (= args (first (@call-times fn-name)))))

(defn verify-nth-call-args-for [n fn-name & args]
  (is (= args (nth (@call-times fn-name) (dec n)))))

(defn clear-calls []
  (reset! call-times {}))

(defmacro mocking [fn-names & body]
  (let [mocks (map #(list 'mock-fn %) fn-names)]
    `(binding [~@(interleave fn-names mocks)]
       ~@body)))

(defmacro stubbing [stub-forms & body]
  (let [stub-pairs (partition 2 stub-forms)
        fn-names (map first stub-pairs)
        stubs (map #(list 'stub-fn (first %) (last %)) stub-pairs)]
    `(binding [~@(interleave fn-names stubs)]
       ~@body)))

It’s just an hour or so of work, so it’s probably rough, and certainly doesn’t support more complex features of other mocking/stubbing libraries. But I thought the simplicity was enjoyable.

Posted in Uncategorized | Tagged: , , , , , | 2 Comments »

frumiOS – a simple object-system for Clojure

Posted by Amit Rathore on December 10, 2009

I’ve nearly stopped blogging, because all my spare time goes into writing Clojure in Action. But I was a bit bored this weekend, and wrote this little library that can be used to write traditional looking Object-Oriented (TM) code in Clojure.

Why would you do that, when you can use a rifle-oriented programming style instead? Think of it like using the rifle as a club… On the other had, the implementation makes plenty use of closures and macros, so it is probably a rifle-oriented program :-)

The implementation is hosted on github, in a project called frumios. And if you reall want to see it now, click below.

(ns org.rathore.amit.frumios.core)

(declare new-object find-method) 

(defn new-class [class-name parent methods]
  (let [klass ((comp resolve symbol name) class-name)]
    (fn [command & args]
      (cond
	(= :parent command) parent
	(= :name command) klass
	(= :method-names command) (keys methods)
	(= :methods command) methods
	(= :new command) (new-object klass)
	(= :method command)
          (let [[method-name] args]
	    (find-method method-name methods parent))
	:else (throw (RuntimeException. (str "Unknown message: " command)))))))

(def OBJECT (new-class :o rg.rathore.amit.frumios.core/OBJECT nil {}))
(def this)

(defn new-object [klass]
  (let [state (ref {})]
    (fn thiz [command & args]
      (cond
        (= :class command) klass
        (= :set! command) (let [[k v] args]
			    (dosync (alter state assoc k v))
			    nil)
        (= :get command) (let [[key] args]
			   (state key))
        :else (let [method (klass :method command)]
		(if method
		  (binding [this thiz]
		    (apply method args))))))))

(defn find-method [method-name instance-methods parent-class]
  (let [method (instance-methods method-name)]
    (or method
	(if-not (= #'org.rathore.amit.frumios.core/OBJECT parent-class)
	  (find-method method-name (parent-class :methods) (parent-class :parent))))))

(defn parent-class-spec [sexprs]
  (let [extends-spec (filter #(= :extends (first %)) sexprs)
        extends (first extends-spec)]
    (if (empty? extends)
      'org.rathore.amit.frumios.core/OBJECT
      (do
	(if-not (= 1 (count extends-spec))
	  (throw (RuntimeException. "defclass only accepts a single extends clause")))
	(if-not (= 2 (count extends))
	  (throw (RuntimeException. "the extends clause only accepts a single parent class")))
	(last extends)))))

(defn method-spec [sexpr]
  (let [name (keyword (second sexpr))
	remaining (next sexpr)]
    {name (conj remaining 'fn)}))

(defn method-specs [sexprs]
  (let [method-spec? #(= 'method (first %))
	specs (filter method-spec? sexprs)]
    (apply merge (map method-spec sexprs))))

(defmacro defclass [class-name & specs]
  (let [parent-class-symbol (parent-class-spec specs)
        this-class-name (keyword class-name)
	fns (method-specs specs)]
    `(def ~class-name
        (new-class ~this-class-name (var ~parent-class-symbol) ~(or fns {})))))

But first, examples -

(ns frumios-spec)

(use 'org.rathore.amit.frumios.core)

(defclass animal
  (method sound []
    "grr")

  (method say-something []
    (str (this :sound) ", I say!"))

  (method move []
    "going!"))

(defclass cat
  (:extends animal)

  (method sound []
    "meow"))

There, that defines a simple class hierarchy. Let’s examine these classes -

frumios-spec> (cat :parent)
#'frumios-spec/animal

frumios-spec> (animal :parent)
#'org.rathore.amit.frumios.core/OBJECT

frumios-spec> (animal :method-names)
(:move :say-something :sound)

frumios-spec> (cat :method-names)
(:sound)

Now, let’s define a couple of instances -

(def a (animal :new))
(def c (cat :new))

What can we do with these instances? Let’s explore -

frumios-spec> (c :class)
#'frumios-spec/cat

frumios-spec> (c :set! :name "Mr. Muggles")
nil

frumios-spec> (c :get :name)
"Mr. Muggles"

That’s the basic stuff, how about calling methods?

frumios-spec> (a :move)
"going!"

frumios-spec> (a :sound)
"grr"

frumios-spec> (c :sound)
"meow"

Notice how cat overrides the sound method. OK, how about a method that calls another method? It calls for the this keyword. Here it is in action -

frumios-spec> (a :say-something)
"grr, I say!"

frumios-spec> (c :say-something)
"meow, I say!"

Notice how in the second call, (this :sound) resolved itself to the overridden sound method in the cat class. That’s subtype polymorphism, common to languages such as Java and Ruby. We could use it to implement something like the template pattern. We can do fairly arbitrary things with frumiOS -

(defclass person
  (method greet [visitor]
    (println "Hi" visitor ", I'm here!"))

  (method dob []
    (str "I was born on " (this :get :birth-date)))

  (method age []
    2)

  (method experience [years]
    (str years " years"))

  (method bio []
    (let [msg (str (this :dob) ", and have " (this :experience (this :age)) " of experience.")]
      (println msg))))

Let’s play with it -

frumios-spec> (def kyle (person :new))
#'frumios-spec/kyle

frumios-spec> (kyle :greet "rob")
Hi rob , I'm here!
nil

The bio method makes two calls using the this construct, one nested inside the other. It works as expected -

frumios-spec> (kyle :set! :birth-date "1977-01-01")
nil

frumios-spec> (kyle :bio)
I was born on 1977-01-01, and have 2 years of experience.
nil

So there it is. I’m sure it doesn’t do lots of stuff a real object-system does. But at 70 lines of Clojure code, you can’t expect a whole lot more. Silly as this is, I had fun writing it! Click here to see how the frumiOS is implemented.

Posted in Uncategorized | Tagged: , , , , | 2 Comments »

Clojure, the REPL and test-driven development

Posted by Amit Rathore on July 28, 2009

I’ve been using Clojure for nearly a year now, and something strange has been happening… I still think unit-tesitng is extremely important, but for some reason I don’t seem to be writing the same number of tests any more. I’m ashamed to say it, but there it is. And it gets stranger – this new lower test count doesn’t seem to matter.

It seems to me that my Clojure code works right the first time more often than my Ruby or Java code ever did. And I seem to find less defects in the Clojure code over time, too.

This is not just a fanboy speaking, though I am a huge fan of Clojure. I think that the reasons I’m observing this is due to a an important characteristic of the language. Instead of just talking about it, let me first walk you through an example.

This is something I had to do recently – we wanted to build a kind of reverse index for an HBase table. The row ids of this table are time-stamps. The idea was that this “reverse index” would allow us to answer the question of what the first time-stamp for a given day was. In other words, we needed to convert a list of time-stamps into a lookup of day vs. the first time-stamp of that day. Eg.

Input:


[“112323123” “1231231231” “123123123” “ 1231231123” ....]
 

Output:


{“2009-07-01” “123123123”
 “2009-07-02” “123131213”
 “2009-07-03” “123123122”}
 

(Note: I plucked the numbers out of the air, they aren’t accurate. But the idea is that the input is a long stream of timestamps, and possibly hundreds could correspond to each day.)

So I get started… thinking to myself – I know how to convert a timestamp to a day. From there, it’s easy to write a function that returns a hash containing the day vs. timestamp (Since I already had a function day-for-timestamp, it was easy) -


(defn day-vs-timestamp [time-stamp]
  {(day-for-time-stamp time-stamp) time-stamp})

So now, all I have to do is map the above function across the input. This gives me a list of hash-maps, each with one key-value pair. To ensure that I’m doing this in order of oldest first, I sort the input as well. Inside of a let form, all of this looks like –


(let [all-pairs (map day-vs-timestamp (sort input-list))]

Now, I have this list of hashes, each with one key (the day) and one corresponding value (the time-stamp itself). I want to combine these into one single hash-map which would be the final answer. But I have to deal with the issue of duplicate keys – when I find a duplicate key, I want to keep the first value associated with the key since it would be the oldest.

Clojure has a merge-with function which does just this – it accepts a function with 2 arguments (which are the two values in case a duplicate key is found) and the returning value is used in the merged hash-map.


(apply merge-with #(first [%1 %2]) all-pairs)

That’s basically it.

Combining everything -


(defn day-vs-timestamp [time-stamp]
  {(day-for-time-stamp time-stamp) time-stamp})

(defn lookup-table [input-timestamps]
  (let [all-pairs (map day-vs-timestamp (sort input-list))]
    (apply merge-with #(first [%1 %2]) all-pairs)))

When I write code like this – I often ask myself, what exactly should I test? I end up writing a few happy path tests that prove my code works. And then a couple of tests that test border cases and negative paths. And I sometimes do it test first.

But the REPL has spoilt me. What I used TDD for when coding with Ruby (and still do), I often do at the REPL. I build tiny functions that work – these are often single lines of code. Then I combine these into other functions, often no more than two lines of code each, sometimes three. And it all just works – leaving me wondering what to cover with tests.

The main reason I still write tests is for regression – if something breaks in the future, I catch it quickly. However, the other thing – the test *driven* design aspect of TDD – has been somewhat replaced by the REPL. And its very much more dynamic than a set of static tests. It really brings out the rapid, in rapid application development – especially when combined with Emacs and SLIME.

One main difference with Clojure vs. Ruby (say) is that Clojure is functional (I use very little of Clojure’s constructs for state). And in the functional world, I just don’t have to worry about state (obviously), and this tremendously simplifies code. I think in terms of map, filter, reduce, some, every, merge, etc. and the actual logic is in tiny functions used from within these other higher level constructs. The idea of first-class functions is also key – I can build up the business logic by writing small functions that do a tiny thing each – and combine them using higher-order functions.

This is one reason why we’re so productive with Clojure. We’ve moved to Clojure for 90% of our work. That said, we still use Ruby for parts of our code-base, and it’s still my favorite imperative language :)

Posted in Uncategorized | Tagged: , , , , , | 9 Comments »

Capjure: a simple HBase persistence layer – updated with documentation

Posted by Amit Rathore on May 25, 2009

This is just to let folks know I’ve (finally) written up some documentation on how to use capjure. Hope it helps!

Posted in webscale | Tagged: , , , , | 1 Comment »

Startup logbook – v0.2 – distributed Clojure system in production

Posted by Amit Rathore on May 2, 2009

This past weekend, we pushed another major release into production. We’ve been working on several things and have made a few pushes since the last time I wrote about this – but this release has a bunch of interesting Clojure related stuff.

Long-running processes

The main thing of note is that the majority of our back-end is now written in Clojure. You might recall that our online-merchant customers send us a lot of data, and we run a ton of analytics on that data. Our initial plans involved Ruby, but as we started using Clojure, it turned out that it is very well suited for this job as well (long running, message-driven processes that crunch numbers).

The raw data sits in HBase, and every night a “master” process starts up which kicks-off the processing of the previous day’s worth of data. The job of this master is only to coordinate the work (it doesn’t actually do any real work), it does this by breaking work into chunks and dispatching messages that each assign work to any worker process that picks it up. The master is single threaded for simplicity, but failure tolerant – it checkpoints everything in a local MySQL database, and if it crashes, it is automatically re-spawned and it recovers from where it left off.

clojure-in-production-v0.2.png

An elastic cloud of worker processes run in anticipation of the master handing out this work. The worker processes use the MySQL database to keep track of their progress as well. The rest is rather domain-specific. We use intermediate representations of the raw data, which is also stored in HBase, before finally storing the summarized version again in HBase.

Swarmiji

We use an in-house distributed-programming framework called Swarmiji to make such distributed programs very easy to write and run. Swarmiji implements a flavor of staged event-driven architecture (SEDA) to allow server processes that exhibit scalable, predictable throughput. This is especially true in the face of over-load, which we can certainly expect in our environment.

The reason I wrote this framework was that I wanted to create distributed, parallel programs which exploited large numbers of machines (like in a data-center) – without being limited by clojure’s in-JVM-threads-based model. So each worker process in Swarmiji gets deployed as a shared-nothing JVM process.

I will write up a post introducing Swarmiji in the next few weeks – once its a bit more battle-tested, and I’ve added a few more features (mainly around process management).

Posted in webscale | Tagged: , , , , , | 3 Comments »

The first meetup of the Bay Area Clojure User Group

Posted by Amit Rathore on February 8, 2009

We held the first meeting of the BACUG (ugh, that’s a tedious acronym) last week, on Thursday the 5th of February. I was expecting about 6-8 people to show up, but we had about 12 people, quite a nice turn out!

After introductions and such, Chris Turner presented his unit-testing library for clojure called Clojure Spec. We talked about testing in general, and because he uses a lot of macro-writing macros in his code, we talked about macros in general and the difficulty in debugging macros. It was a good discussion – though I did throw in (tongue-in-cheek) that we’re using Test Is at Cinch. Thanks Chris, it was a good presentation.

After that talk, I gave an introduction to how we’re presently using Clojure in our startup. Since we’re using HBase as our persistent data-store, and processing that data using a bunch of clojure processes, we talked about data-modeling for HBase-like systems (I will write about my implementation of that data-model in another post).

An interesting thread came up – led by Joe Mikhail who works at Google and obviously does a lot of Map/Reduce/BigTable stuff – around how at least in the beginning, multi-threaded clojure processes can be used in place of Hadoop when processing HBase data. And he did mention that using systems like Terracotta, one can scale up such solutions. We’re going to look into that next week.

The remainder of the meeting went by in a buzz of talking about different languages, technologies, and general geek topics. The one thing of interest here was the point that software transactional memory is no panacea (surprise!) to the whole concurrency thing. Here’s an ACM article (there are several actually, just keep turning the pages) that throws some light on the issue. Here’s another.

Overall, it was a very good meeting – we’re hoping that the next one would be attended by more people, and especially some folks that have used other Lisps in the past – we’re all curious about what such folks think about Clojure. Indeed, and we want to learn from their experience in using idiomatic Lisp.

I’m thinking that we’ll do another Clojure meetup in about 6 weeks or so… join up, and stay tuned!

P.S. – A shout of thanks to my employer Runa. for hosting this meeting.

Posted in Uncategorized | Tagged: , , , , | Leave a Comment »

Startup logbook – v0.1 – Clojure in production

Posted by Amit Rathore on January 28, 2009

Late last night, around 3 AM to be exact, we made one of our usual releases to production (unfortunately, we’re still in semi-stealth mode, so I’m not talking a lot about our product yet – I will in a few weeks). There was nothing particularly remarkable about the release from a business point of view. It had a couple of enhancements to functionality, and a couple of bug-fixes.

What was rather interesting, at least to the geek in me, was that it was something of a technology milestone for us. It was the first production release of a new architecture that I’d been working on over the past 2-3 weeks. Our service contains several pieces, and until this release we had a rather traditional architecture – we were using ruby on rails for all our back-end logic (eg. real-time analytics and pricing calculations) as well as the user-facing website. And we were using mysql for our storage requirements.

This release has seen our production system undergo some changes. Going forward, the rails portion will continue to do what it was originally designed for – supporting a user-facing web-UI. The run-time service is now a combination of rails and a cluster of clojure processes.

When data needs to be collected (for further analytics down the line), the rails application simply drops JSON messages on a queue (we’re using the excellent erlang-based RabbitMQ), and one of a cluster of clojure processes picks it up, processes it, and stores it in an HBase store. Since each message can result in several actions that need to be performed (and these are mostly independent), clojure’s safe concurrency helps a lot. And since its a lisp, the code is just so much shorter than equivalent ruby could ever be.

Currently, all business rules, analytics, and pricing calculations are still being handled by the ruby/rails code. Over the next few releases we’re looking to move away from this – to instead let the clojure processes do most of the heavy lifting.

We’re hoping we can continue to do this in a highly incremental fashion, as the risk of trying to get this perfect the first time is too high. We absolutely need to get the feedback that only production can give us – so we’re more sure that we’re building the thing right.

The last few days have been the most fun I’ve had in any job so far. Besides learning clojure, and hadoop/ hbase pretty much at the same time (and getting paid for doing that!), it has also been a great opportunity to do this as incrementally as possible. I strongly believe in set-based engineering methods, and this is the approach I took with this as well – currently, we haven’t turned off the ruby/rails/mysql system – it is doing essentially the same thing that the new clojure/hbase system is doing. We’re looking to build the rest of the system out (incrementally), ensure it works (and repeat until it does) – before turning off the (nearly legacy) ruby system.

I’ll keep posting as I progress on this front. Overall, we’re very excited at the possibilities that using clojure represents – and hey, if it turns out to be a mistake – we’ll throw it out instead.

Posted in Uncategorized | Tagged: , , , , , , | 5 Comments »

More on Lisp syntax, and language extensions

Posted by Amit Rathore on February 16, 2007

Following my recent post on the topic, I thought of one more thing that the syntax of Lisp allows you to do. Being homoiconic, and the fact that code manipulation is so simple (it’s all lists), layering on “language extensions” becomes possible. For example, if Betty Programmer realizes that OO is a great way to design and write code but that Lisp by itself doesn’t provide an OO facility (there are no “class” constructs, no inheritance etc.) – she doesn’t need to despair.

She can write code to add an OOP system to the language. Yes, this means Lisp really blurs the distinction between the language designer and the programmer. In other words, while it’s fairly obvious that Lisp is very well suited to writing DSLs, it is also possible to fundamentally extend the language as well – like adding an OO system, or pattern-matching, or logic-programming (ala Prolog).

Now, obviously, I’m not proficient enough yet to do anything of this sort. But, as I said before, it is my intention to learn :)

Lisp. A language where being meta is something worth thinking about.

Posted in Uncategorized | Tagged: , , , , | 3 Comments »

Lisp syntax, and when code is data

Posted by Amit Rathore on February 14, 2007

Like I said earlier, my friend Ravi introduced me to Lisp several years ago, but it has taken me many years to really want to learn it well enough. I’ll write about my reasons in another post. In any event, at the beginning of this year, I started to pick it up again, promising myself that I’d be serious. This time. So far so good.

I think I’ve started to grok one of the core ideas of Lisp. I had always read that the syntax of Lisp was one of its strengths. And I had always struggled with that idea, knowing it was important, yet was quite unable to really put my finger on it. I think I’m closer to it today.

If you had to create a programming language to write programs that wrote programs (as in, say DSLs) – what design choices would you make?

For one thing, you’d have to be able to generate and manipulate (walk parse-trees, compare and transform nodes etc.) code as though it were just another data-structure. Right? OK, so the code that was being generated would look like and behave like data.

You would then create an EVAL function that could run the generated code. Maybe your generated code would in turn produce generated code, so to keep things easy and simple, your language syntax would be the same as that of the generated language. In other words, you’d end up with a homoiconic programming language. Finally, you would bootstrap your language processor and arrive at your final metacircular evaluator.

To recap, this language would have syntax that looked and behaved like data and because of it could generate and manipulate that data, which itself could be code. What would this data structure look like? One obvious choice for this is a tree (because of parse-trees). If you think about it, XML is just like a tree. But it’s kludgey. What we want is something like XML but without all the cruft. For example -


<program>
	<function name=\"add_to_stock\">
		<param name=\"counter\" />
			<call_function name=\"increment\">
				<argument value=\"counter\"/>
			</call_function>
	</function>

	<function name=\"remove_from_stock\">
		<param name=\"item\"/>
			<call_function  name=\"decrement_from_stock_file\">
				<argument value=\"item\"/>
			</call_function>
	</funtion>
</program>

The syntax is truly disgusting, but useful – especially if you need to programmatically generate it. Let’s now try to make it easier for humans, too. I’m going to remove the ‘program’ tag, because all this stuff is code. I’m going to then change from XML tags to simple ‘(‘ and ‘)’ without the names – and make an assumption – the first word that appears is always a function call. Except for define – which I’ll use to denote a definition for a function. I’ll also lose the XML attribute names, assuming that words that follow the function name are always parameters (unless it’s a code block itself – which would get evaluated first). So, we’re left with -




(define (add_to_stock counter)
    (increment counter))

 (define (remove_from_stock item)
    (decrement_from_stock_file item))



Where does this leave us?

It’s the same exact XML syntax, but it’s just a bit modified and has a few rules thrown in. Importantly, it’s still as easy to generate as XML. It’s just a list of lists of words. As in, a unit of code in this format would always start and end with parenthesis, and they would enclose either a bunch of zero or more symbols, or other lists.

In fact, a language that was good at list processing and had an eval function would probably do a really good job with this stuff!

Posted in Uncategorized | Tagged: , , , , | 2 Comments »

Beyond objects – I

Posted by Amit Rathore on February 9, 2007

… It’s in words that the magic is–Abracadabra, Open Sesame, and the rest–but the magic words in one story aren’t magical in the next. The real magic is to understand which words work, and when, and for what; the trick is to learn the trick.
… And those words are made from the letters of our alphabet: a couple-dozen squiggles we can draw with the pen. This is the key! And the treasure, too, if we can only get our hands on it! It’s as if–as if the key to the treasure is the treasure!

John Barth, Chimera

It’s a copy of a quotation that appears in The Structure and Interpretation of Programs. It speaks to the fact that if you know the name of something, as in a way to refer to it, then you have control over it. Like functions or closures for instance. If you can refer to it by name, you can tell it to someone else. Or pass it to another function. You can remember the name, so you can speak it when you’re ready. When the context is right. Like when all the variables, objects, and functions have aligned.

The idea of metalinguistic abstraction is a fundamental one. It’s a bottom-up way of thinking about your problem-space, and figuring out primitives and operations on those primitives. Rather than trying to build a system that satisfies a particular set of constraints in the given problem-space, the idea is to build a way to express more complex concepts using those very primitives and operations. That way, changes to the system can be made by changing higher level constructs – and going further down the layered stack of abstractions as needed, depending on how fundamental the changes are. This way, changes (or any new set of constraints) can be handled in a more clean and elegant fashion, and the entire system benefits automatically by a change made at a lower level.

If this sounds like building what these days is called a domain-specific language (DSL), then sure, but it is not a new way of doing things. It is however, possible to do a lot of this stuff easier in “enterprise-type” scenarios, because of a larger acceptance of more dynamic languages. Like Ruby, for example. It isn’t particularly easy to write software this way – it involves a change in the way most of us were taught programming. Everything is not imperative any more, code doesn’t have to be “written” before it runs, lines between data and code begin to blur. On that last point, what is a closure? Is it data or is it a procedure? It’s sometimes one, sometimes the other, and occasionally both, and indeed, sometimes it is neither. Here’s an example -


def create_coord(x, y)
Proc.new { |selector|
if selector == :x
x
elsif selector == :y
y
end
}
end

c = create_coord 10, 20
puts "X is " + c.call(:x).to_s
puts "Y is " + c.call(:y).to_s

So, the question here is, what is c? Is it data? Or is it code, that does something? Here, c is an object that represents a cartesian coordinate, but without any explicit data elements like variables or attributes… so, what is it?

Note – This is Ruby code, but a different language, (like Lua, for example, that has syntactic sugar for things like keys on a hash table) could make it unnecessary for the call(:symbol) syntax and instead, just calling x and y would translate it to a call on the object with the given method name.

The point, however, is that there are more ways to think about abstraction than just OO, and to paraphrase Paul Graham, some of them can transcend objects. The bottom-up approach itself can be implemented using plain objects and OO thinking, but there are other ways which can make for some powerful expressablilty. The code above approaches functional programming, and I’m interested in scaling that model to build large systems from pure or near-pure functional constructs. I’m nowhere close to being proficient in writing software this way, but it is my intention to learn.

Posted in Uncategorized | Tagged: , , , , | 2 Comments »