Welcoming Rich Hickey to the Staples Innovation Lab

I’m pleased to announce that we’ve engaged Rich Hickey at the Staples Innovation Lab, as Special Technical Advisor. He will provide architectural and general technical oversight across all the products we’re building. These include things like dynamic and personalized pricing, product recommendation engines, email targeting and behavioral retargeting, a new search engine, shipping delivery optimization, etc. 

We’re already heavy users of Clojure, Datomic, ClojureScript, Simulant, and Pedestal. So this just makes a ton of sense for us… we’re also working with Cognitect on several initiatives, so having Rich’s oversight will just add to the awesomeness of the team.

Please join me in welcoming Rich to the lab! 

P. S. Get in touch with me if you’d like to explore opportunities working with the Clojure stack at the second largest E-Commerce operation in the world.

It’s always Day 2 at Staples

It’s been over two months since Runa was acquired and we became Staples Innovation Lab. I’ve been meaning to write about this transition, but things have just been so busy. I do think it’s important that I record this journey though, because in a few years time, people will look around and will wonder at the steep rise of Staples. 🙂

We started Runa with the same gleam in our eyes as most Silicon Valley startups – of changing the face of the industry we were in. We chose e-commerce, even though in 2008, it wasn’t exactly sexy. Those were the years of the consumer startups, and no one was even looking at enterprise focused deals. My own viewpoint though was, and still is, that the e-commerce story is still being written. I’m an Amazon.com shareholder, I’ve been using their service ever since I moved to this country over 10 years ago, and I’m a huge fanboy. My appreciation for Jeff Bezos is paralleled only by my admiration for Steve Jobs. Amazon.com has literally defined e-commerce… and is overwhelming every other contender in the space. 

So we started Runa with one overarching goal. To help e-commerce merchants survive the Amazon.com onslaught. The idea was that most retailers aren’t technology companies, and don’t have the capabilities to battle Amazon, one of the most advanced tech companies in the world. Further, these folks are unlikely to ever grow such capabilities in-house, for a whole variety of reasons. At Runa, we built out a set of SaaS products, each focused on a specific area. All these services were built on top of a big-data + machine-learning stack. Our output was a run-time platform that ran predictive models to address individual aspects of the shopping experience. Merchants simply plugged our APIs into their world, and we enabled their site.

For instance, we built PerfectOffer to help merchants stop giving away discounts, indiscriminately, to everyone. Our platform built sophisticated statistical models of the behavior of the shoppers, and then in real-time would determine which shopper should get what deal on what product. Or even if they should get an offer at all. Not only was the discount spend made far more efficient (which helped average gross margins), but it also had a highly non-linear positive impact on net margins. Another service we built was called PerfectShipping, which used past seller performance and shipment tracking data to figure out when shoppers could expect to receive their items, with a high degree of confidence, even if merchants used the cheapest carrier services. Online marketplaces and retailers alike used this to take on Amazon Prime’s 2-day free shipping, and the incremental sales numbers this service generated for our large merchants was incredible. Other services included PerfectEmail and PerfectBundles.

By the time Runa was acquired, we were able to count some very large retailers as our customers, including eBay, Groupon, and Target. In July this year, we added Staples. Now, to be clear, we had never thought Staples was a particularly relevant company… actually, I had never really given much thought to Staples at all. However, once they became a customer, I did find out a fair bit about them, and things seemed quite impressive. It turns out, Staples is the world’s second largest e-commerce player, after Amazon.com. Sure, it’s a distant second, but it does make over 10 billion dollars a year online. Remember, this is despite them not being a tech company. Not in any realistic sense of the word.

So when they talked to us about a strategic partnership, and they described where they wanted to go, I realized that this could be an incredible opportunity. My vision for Runa was always this set of AI programs that would drive all aspects of an e-commerce operation. We would use data to optimize just about everything, and all in real-time, all automated. This would drive down costs, and would then allow for lower margins, which would mean cheaper prices for shoppers. And all the while, it would have the amazing side-effect of improving the customer experience. Staples, in my mind, was the perfect place to put this into action. Kind of like a PerfectPlayground 🙂

Staples is an old company – they’re 27 years old, and they’re still a brick-n-mortar retailer at heart. This is changing, and there’s a lot to be done. The reality though is that it was a startup at one point, and that entrepreneurial spirit is still alive. The first phase of the company, in a sense, was the retailer phase, and they opened thousands of stores around the world. All are (or certainly were) amazing cash-generating machines, and even today, even as the the face of stores business is changing, they’ll continue to generate the cash needed to help the overall company as a whole.

I like to think of the offline business as Day 1 for Staples, and we’re now on Day 2. This is going to be a long day, since we’re never going to be done. The good news is that there’s tremendous upside as we execute on this path to becoming a technology company ourselves. There’s much work to be done, and much software to be written. There are plenty of business processes to change. There are plenty of people to win over, Wall Street analysts included. The glory is in the challenge, of course, and this is one heck of a challenge. How does one attempt to build what a company like Amazon.com has built over the past 15 years (at least the relevant e-commerce bits) in a span of just 2-3 years? This is a tech challenge, a people challenge, a process challenge, and a business challenge. It means we can’t do things in the typical, traditional manner. We’ve made very different technology choices (Lisp, anyone?) and are asking all the crazy people we can find, the doers, to join us. The enemy is formidable, but one thing’s for sure: it’s going to be one hell of a fight.

It’s always Day 2 at Staples.

Why Datomic?

Cross-posted from Zololabs.

Many of you know we’re using Datomic for all our storage needs for Zolodeck. It’s an extremely new database (not even version 1.0 yet), and is not open-source. So why would we want to base our startup on something like it, especially when we have to pay for it? I’ve been asked this question a number of  times, so I figured I’d blog about my reasons:

  • I’m an unabashed fan of Clojure and Rich Hickey
  • I’ve always believed that databases (and the insane number of optimization options) could be simpler
  • We get basically unlimited read scalability (by upping read throughput in Amazon DynamoDB)
  • Automatic built-in caching (no more code to use memcached (makes DB effectively local))
  • Datalog-as-query language (declarative logic programming (and no explicit joins))
  • Datalog is extensible through user-defined functions
  • Full-text search (via Lucene) is built right in
  • Query engine on client-side, so no danger from long-running or computation-heavy queries
  • Immutable data – audits all versions everything automatically
  • “As of” queries and “time-window” queries are possible
  • Minimal schema (think RDF triples (except Datomic tuples also include the notion of time)
  • Supports cardinality out of the box (has-many or has-one)
  • These reference relationships are bi-directional, so you can traverse the relationship graph in either direction
  • Transactions are first-class (can be queried or “subscribed to” (for db-event-driven designs))
  • Transactions can be annotated (with custom meta-data) 
  • Elastic 
  • Write scaling without sharding (hundreds of thousands of facts (tuples) per second)
  • Supports “speculative” transactions that don’t actually persist to datastore
  • Out of the box support for in-memory version (great for unit-testing)
  • All this, and not even v1.0
  • It’s a particularly good fit with Clojure (and with Storm)

This is a long list, but perhaps begins to explain why Datomic is such an amazing step forward. Ping me with questions if you have ’em! And as far as the last point goes, I’ve talked about our technology choices and how they fit in with each other at the Strange Loop conference last year. Here’s a video of that talk.

Startup marketing and you

Cross-posted from Zolo Labs.

Marketing is defined as the act of promoting (and selling) your products or services. Folks in most industries consider it an important part of their business, especially in larger companies. For some reason, this seems less true in tech startups.

Instead, most founding teams concern themselves a lot with product and engineering. After all, if you don’t have a product, what are you going to market? While this may seem logical, I’ve come to realize this is a flawed view. I now believe that marketing is a critical function of all startup teams, right up there with product, engineering, recruiting, and, fund-raising.

To come to this realization, I first had to internalize that “marketing” wasn’t a bad word. While the above definition conjures up (at least in my mind) images of sleazy sales people, marketing is actually one of the most important ways of interacting with your customers. And really, are there any unimportant ways of interacting with customers?

As a startup, if there are even a few people out there who are actually willing to give you a few minutes (or seconds!) to listen to what you have to say, hallelujah! Marketing, then, is an opportunity for you to engage them in a dialogue, to explain to them why you exist at all. Most people filter out all forms of traditional marketing, not just because there are too many of them, but because they come across as insincere.

There was a time when running an ad would actually produce decent ROI. This isn’t true any more, of course, and in my mind, here lies the opportunity. Today’s connected world of blogs and social networks have presented us once again with a channel to actually connect with our customers and potential customers. To not just “market” to them, but to actually reach out and have a conversation.

While I filter out most forms of marketing as noise, what does catch my attention is authenticity. The new world of marketing then, is just this form of real and sincere social conversation. For tech startups, it’s the dialogue between the founding team, and their early customers, and their extended community. What an awesome chance to be yourself! It’s an outlet to express your philosophy, your beliefs, and your vision. And yes, perhaps to even talk about your products. It’s an opportunity to hear back from this audience, from those who actually care enough to respond! It’s an opportunity to help those who’re listening (or reading) in some small way, even if they don’t actually buy from you.

Do a search for “startup marketing” on your favorite search engine, and you’ll get thousands of results. But this is really it – this social conversation, where you can put yourself (and your company) out there. You can’t outsource this, this is you! As David Packard once said: marketing is too important to be left to the marketing department.

I’ve written before that of the two sides a startup (the product side, and the distribution side), it is the distribution side that’s the harder one. Marketing, defined the way we just talked about, is key in solving this distribution challenge. And, defined in this way, it doesn’t need to be looked down upon either… after all, you are the marketing 🙂

P.S. Check out what we’re building: Zolodeck.

In search of the viral loop

Cross-posted from Zolo Labs.

Just finished reading Viral Loop. As we think about the distribution side of things for Zolodeck, I figured it couldn’t hurt to read about how a bunch of companies achieved massive scale.

It’s a good read… and certainly describes how things were done at companies like Hotmail, Twitter, Facebook, Hot Or Not (remember?), Bebo (remember?), MySpace (remember?), eBay, Flickr, PayPal, and even Tupperware (yeah, really).

It isn’t, however, a book of recipes. I didn’t really expect a book to be able to just tell me the 10 things I can do to fix distribution, I guess, and on that count the book was merely describing how awesome it is when you do achieve massive scale. Not how to get it.

Still, a very decent read, and did trigger a few thoughts for what one can do to address this. A lot of people in the startup space know this already, but it’s been sinking in for me more and more over the past few months: distribution is the more important of the two key things a startup needs to solve (the other being product/market fit, or product in general). And also that distribution is the harder of the two problems, and more than product, it is distribution that will make or break the company.

As I said, we’re thinking a lot about this for Zolodeck… and since this is so important, we’re erring on the side of over-measuring things to ensure we can keep a pulse on what the distribution story is looking like. What else can one do at such an early stage?

Pretty-printing in Clojure logs

Cross-posted from Zolo Labs.

Logging is an obvious requirement when it comes to being able to debug non-trivial systems. We’ve been thinking a lot about logging, thanks to the large-scale, distributed nature of the Zolodeck architecture. Unfortunately, when logging larger Clojure data-structures, I often find some kinds of log statements a bit hard to decipher. For instance, consider a map m that looked like this:

When you log things like m (shown here with println for simplicity), you may end up needing to understand this:

Aaugh, look at that second line! Where does the data-structure begin and end? What is nested, and what’s top-level? And this problem gets progressively worse as the size and nested-ness of such data-structures grow. I wrote this following function to help alleviate some of the pain:

Remember to include clojure.pprint. And here’s how you use it:

That’s it, really. Not a big deal, not a particularly clever function. But it’s much better to see this structured and formatted log statement when you’re poring over log files in the middle of the night.

Just note that you want to use this sparingly. I first modified things to make ALL log statements automatically wrap everything being logged with pp-str: it immediately halved the performance of everything. pp-str isn’t cheap (actually, pprint isn’t cheap). So use with caution, where you really need it!

Now go sign-up for Zolodeck!

Why Java programmers have an advantage when learning Clojure

Cross-posted from Zolo Labs.

There is a spectrum of productivity when it comes to programming languages. I don’t really care to argue how much more productive dynamic languages are… but for those who buy that premise and want to learn a hyper-productive language, Clojure is a good choice. And for someone who has a Java background, the choice Clojure becomes the best one. Here’s why:

  • Knowing Java – obviously useful: class-paths, class loaders, constructors, methods, static methods, standard libraries, jar files, etc. etc.
  • Understanding of the JVM – heap, garbage collection, perm-gen space, debugging, profiling, performance tuning, etc.
  • The Java library ecosystem – what logging framework to use? what web-server? database drivers? And on and on….
  • The Maven situation – sometimes you have to know what’s going on underneath lein
  • Understanding of how to structure large code-bases – Clojure codebases also grow
  • OO Analysis and Design – similar to figuring out what functions go where

I’m sure there’s a lot more here, and I’ll elaborate on a few of these in future blog posts.

I’ve not used Java itself in a fairly long time (we’re using Clojure for Zolodeck). Even so, I’m getting a bit tired of some folks looking down on Java devs, when I’ve seen so many Clojure programmers struggle from not understanding the Java landscape.

So, hey Java Devs! Given that there are so many good reasons to learn Clojure – it’s a modern LISP with a full macro system, it’s a functional programming language, it has concurrency semantics, it sits on the JVM and has access to all those libraries, it makes a lot of sense for you to look at it. And if you’re already looking at something more dynamic than Java itself (say Groovy, or JRuby, or something similar), why not just take that extra step to something truly amazing? Especially when you have such an incredible advantage (your knowledge of the Java ecosystem) on your side already?