Signature

More jq fun

Jan 29, 2014

I’ve written about jq before, but I mention it again here as a very useful tool for parsing JSON on the command line.

I needed to grab some England Bank holidays for a quick hack the other day, and found that the GOV.UK team made web better by allowing you to append .json to any of their links to get to the data.

Hold onto yer hats:

2103 — the year's code and tooling in review

Dec 31, 2013

I saw the #code2013 hashtag on Twitter last night and joined in. It got me thinking about the stuff around the programming languages I use, and my professional skillset in general.

I’ll briefly touch on how a few major events in my life forced me to cut the fat and become more focused.

A few notes, then.

The programming languages

Scala

My new, old favourite. My go-to language since 2009. I can’t add much for 2013 apart from that I’m happy to see the community mature around it. We’ve gone from “How we adopted Scala at company X” to “How to go back in time using Akka, Play! and XYZ”. Oh, and then the reactive course was fun too, and highlighted a few holes in my Akka education.

Java

Java still mostly pays the bills, and I’m writing even better Java thanks to Scala. My editor is told to highlight everything in a shade of gray, apart from re-assignables which are red. It all just flows.

Python

I’m a big fan of Pylons and have used Python since 2006-ish for tiny webapps, but this year sees me going down the finance route with SciPy and Pandas thanks to a handful of courses I did recently.

R

Strangely addictive. I did a data analysis course with the famed Johns Hopkins university earlier this year, and thought I would be “learning” R along the way. If anything, it felt like I already knew R, and the thing that I learned was making a good analysis, writing it perfectly and succintly, and performing an honest peer review. R is perfect for slicing and dicing your data, and it doesn’t get in the way.

Dart

The main attraction to Dart for me was the promise of compiling to a better Javascript which you can ever hope to write by hand. The Dart syntax itself is familiar, and then you have the odd goodies like the cascade operator (borrowed from Smalltalk) which lets you chain methods without having to return the instance in your API, and being a Scala guy, I love having rockets (=>) scattered all over my code (Dart’s syntactic sugar for concise getters).

Javascript

Being a backend programmer by trade, I usually don’t go for frontend gigs, but a client this year said they don’t mind me learning on the job, and I’ve heard good things about AngularJS, so I dove right in. I’ve touched on Javascript a lot in the past, but this year I spent a significant tenth of my professional time fiddling with it.

Haskell

Yes, I really did spend the tiniest amount of time with Haskell, but enough to mention it here. I downloaded, chopped and changed the XMonad code and the take-away for me was that 2014 will see more Haskell.

Other languages, not-so-much-languages, and tools

My starter Dart project is a game (where do I find the time?), hence WebGL and subsequently GLSL. I’m particularly excited about getting my hands even dirtier with shaders with Mozilla recently announcing live-editing of shaders using Firefox.

I’ve always used Capistrano and Chef for my provisioning and configuration needs, but this year I gave Ansible a shot. I hope Ansible stays the way it is: small, simple, elegant. No agents; just SSH.

Conclusion

2013 was a bumpy ride. We had a beautiful daughter in June, and I quit my job soon after to freelance from home and be 2 flights of stairs away from her. To be able to learn a few new skills and do 4 courses, I had to make some changes. There are the 5AM starts, the highly-condensed exercise routine in the morning, diligently deleting anything which doesn’t contribute to the plan I have for our life (this includes articles, emails, etc), and just generally getting things done.

The Polar Bear who might have eaten Xmas The game I mentioned seems like a distraction, but I feel it’s important that I do it. I’ve been in love with doodling since I was a kid, and really took it up this year. I’m an able musician, and since I can code I thought I’d bring these ingredients together and make something nice. (Those umpteen characters and storylines in my head will finally be unleashed!).

Thanks for reading this far. May you have a great 2014.

I leave you with a doodle I made a few days ago entitled “The Polar Bear who might have eaten Xmas”. (Don’t worry — this isn’t one of my games ideas ;-)

Using single and double pipes in Scala conditionals

Dec 27, 2013

In a nutshell, use a single pipe if you want all the expressions in your conditionals to be evaluated, and a double pipe to conk out as soon as you have true.

scala> def a = { println("a") ; true }
a: Boolean

scala> def b = { println("b") ; true }
b: Boolean

scala> def c = { println("c") ; true }
c: Boolean

scala> if (a || b || c) println("true") else println("false")
a
true

scala> if (a | b | c) println("true") else println("false")
a
b
c
true

Heuristics

Dec 13, 2013

I recently saw a talk by Jack Dorsey where he iterates a list of commandments for this life, and number three is “drink only lemon water and red wine”.

I love red wine, and was intrigued by this lemon water idea. I investigated and it led me to an article which asks how much lemon water is needed. They quote a heuristic:

For those who weigh less than 150 pounds, squeeze half a lemon’s worth of juice into a glass of water. If over 150 pounds, use an entire lemon’s juice.

This is a heuristic, and it is beautiful. It saved the author from quoting an entire “weight in pounds to amount of lemon” table.

The heuristic is one of my favourite concepts of all time. During engineering school in the late nineties it became ever more apparent what a powerful concept it is, e.g. using lookup tables when designing flow systems with complex flow dynamics. It makes applied rocket science a bit easier. Or when solving path finding problems where the brute force method is just not a viable option in terms of time complexity.

Anyhoo, I’ve reached my 10 minute limit for writing ★, and since I can’t do this powerful concept any justice with just a few words, I’ll let the editors of Wikipedia englighten you further.

★ this is my own heuristic (aka “rule of thumb”) for not spending too much time on writing.

Project Euler 18 and 67 in Scala using foldRight, zip and sliding

Nov 28, 2013

I’ve recently applied to Toptal and sucked miserably at the entry exam. My algo chops were blunt and I thought I’d rectify it by revisiting Project Euler. With the startup last year and the baby this year I just haven’t been able to find the time for programming challenges, but that has to change.

Looking at my Project Euler source directory, I saw that I left it at problem 17, so next up will be 18. The problem description, however, mentions that the problem repeats itselt as 67, but with a bigger input that will run 20 billion years if you go the brute force route.

I worked a little on this problem last night, and decided to by-pass the brute force solution completely. It was a bit late, though, and I pulled my eyelids open far enough to make it to bed. I then went on to dream about the damn problem all night. I knew there had to be a simple bottom-up fold-based solution, and the voila moment came for me when I realised I had to seed the fold computation with the base layer in the triangle.

Here you go:

A little explanation

The triangle is represented as a nested list, like so: List(List(1), List(2, 3), List(4, 5, 6)) and so forth.


     1
    / \
   2   3
  / \ / \
 4   5   6    <- this is the "base layer" in my explanation.

Since you need to add the maximum of the two immediate children to the layer above, a foldRight wouldn’t give me all the info I need in the curent iteration. Foldright for me means “the data is coming FROM the right”, i.e. List(4, 5, 6) will be processed first, then List(2, 3) but at no point in the iteration will they be available together so I can do the sum. List(4, 5, 6) would need to come into the iteration with List(2, 3) in another way, and I realised I can use the foldRight’s accumulator for that by seeding the foldRight with the base layer in the triangle (aka the last list List(4, 5, 6)).

The easiest way was to just seed the foldRight with a list of zeros one element larger than the base layer. You then break it up into pairs using sliding(2,1), take the max of the pairs, and sum the max with the corresponding (thanks to zip) element in the layer above.


      1
     / \
    2   3
   / \ / \
  4   5   6
 / \ / \ / \
0   0   0   0    <- this becomes the new "base layer", or "seed"

No mutable state; no recursion; simple to understand. As Erik Meijer would say: “baby code”.

A webwords Chef cookbook

Nov 10, 2013

There are a lot of new examples up on the Typesafe website, a few with the word “Reactive” in them to drive home the credo behind the framework. One of my favourite examples, however, seem to have been demoted but still remains one of my favourites: webwords.

The old Heroku link to the live running instance of webwords does not seem to be up anymore, so I whipped up a quick Chef cookbook with which to spin up your own local instance.

You can find the Github repo here. Spin it up and have fun.

BTW, join us in doing the Principles of Reactive Programming course

Pretty-print JSON in vim

Nov 05, 2013

jq to the res-q. TL;DR A quick way to pretty-print JSON in vim with no dependencies (here’s looking at you, Perl). Download jq then :%!jq ‘.’

Computational statisticiansData scientists have some the funnest tools, and I found a good write-up of a few more I didn’t know about called 7 command-line tools for data science. The first thing the article mentions is a gem called jq.

I found jq to be especially useful for formatting JSON in vim. So, before ado can in any way be furthered:

curl http://stedolan.github.io/jq/download/linux64/jq -o ~/bin/jq   chmod +x ~/bin/jq

In vim, open your unformatted JSON file, then: :%!jq ‘.’.

There you go, ladies and gents — a command so short, you don’t even need to put it in your vimrc!

Dell XPS 13 LE

Nov 02, 2013

The portal to my livelihood. TL;DR If you’re a laptop-toting *nix user, then this is the laptop for you. Also, upgrade the kernel before you do anything else, for the best experience.

This advice is highly unqualified, though. The last time I ran *nix on a laptop was when I ran some flavour of Gentoo on an old Centrino-loaded HP in 2005. I’ve been a Mac user ever since those darned iPhones came out.

It’s just that I ran out of space on my 128GB SSD Air for the umpteenth time, and shuddered at the thought of breaking off glued-down components according to some iFixit guide. (Damnit! I just looked again, and there’s no glue involved in upgrading the SSD. Oh, well — the missus needed a new laptop anyway, and this was a welcome hand-me-down. Besides: new toys!)

First thing’s first

When you do get one of these bad boys, the first thing I recommend you do is this:

That will get you the latest kernel without upgrading everything. As any Linux user would know, upgrading to fix one problem introduces a thousand other problems, and this felt like a nice trade-off.

The reason for the above command is that I did have some wifi stability issues to start with, so I hope to save you that initial few dropped connections.

Once again, kudos to popey over on the #ubuntu-uk IRC channel for the advice.

Wifi has been stable and pretty quick. I suppose it’s not all down to the new kernel and drivers: I have Virgin 60Mb fibre optic, and my walls are refreshingly free of tin foil.

The gear

The XPS 13 comes packed with some great specs which you can gloss over here, but I’ll re-iterate the key components: Intel® Core™ i7-3537U CPU @ 2.00GHz × 4, 8GB RAM, 256GB SSD of which 230-odd usable and a mega bright 13.3" screen all on a pretty little laptop that feels like an 11" one. I never put the brightness above a third of the available level, and at 1920 x 1080 it looks great.

The keyboard still sports a Windows button, but it’s the new Windows logo which I’m not familiar with yet, and I pass it off in my mind as a generic “dashboard” type button, so it’s all good. A Tux button would probably be a bit weird. I mean, what would a penguin button do? Summon a troop of penguins? Nah. Besides, penguins — just like seals and dolphins — can be dirty little scumbags.

The ever-so-scientific bench test

I needed GIMP for something last night, and when I installed it and launched it for the first time, the realisation hit me that it all took about 30 seconds.

So, I purged GIMP and re-did it for your viewing pleasure:

Apart from this, I’m running about multiple Java server apps (for this thing I’m working on), plus a couple of VMs, plus some Python scripts that are running some numbers. My software does make efficient use of all the available cores, and CPU hasn’t been a problem for me in a while.

All the Linux games available on Steam runs like a dream. Just the addition of Steam of a *nix system feels like we’ve come full circle.

Qualms

It’s early days — a lot might still happen. In fact, Murphy’s Law states that I’ll publish this favourable review and it will be shortly followed by the i7 going up in smoke.

Anyway, here are some of my initial qualms:

  • The mouse is triggered sometimes whilst typing. This is configurable, and I have switched it to disable the mouse whilst typing, but to no avail.
  • The password box looses focus more times than not, and you end up typing your password in unfavourable places. This could be related to the previous issue discussed.
  • Keyboard is ever so slightly flimsy, because sometimes letters will double up. Perhaps this is just a keyboard I need to get used to. Besides, I’ve used the Mac chiclet-style keyboards exlusively since 2009.
  • The palm rests — even though they feel great to the touch — seems to get a bit streaky after a while. My fingers are constantly greasy from chowing on ginger biscuits and stuff like that, and it doesn’t wipe down well.
  • The backspace button does make a slight squeaky sound sometimes, but this could be because my pinky finger “pulls” on the button due to the angle I attack it with. Still — I don’t think this should happen.

Outro

If anything else pops up, or if the XPS 13 makes my life significantly better in any other way, I’ll come back here to amend. (I won’t do those silly UPDATE things at the bottom — you can look at the Git history of this page for changes.)

This is my first ever piece of Dell equipment, and it doesn’t look nor feel like a grey, impersonal piece of office equipment. Also, Mr. Mark Hails-From-The-Same-Home-Town-As-I Shuttleworth got things very right with Ubuntu. Linux has certainly come a long way since that day in 1995 when my mate dropped off 25-odd floppies of Slackware.

I’m more productive than ever and won’t be going back to Apple. I certainly don’t want to shoehorn OSX into a usable system with macports any time soon.

AngularJS and Internet Explorer woes

Nov 01, 2013

Aka endless clicking on crossbrowsertesting.com and why I love being a backend programmer.

AngularJS and Internet Exporer woes A few weeks ago, one of my clients tasked me with building a public-facing single page app for their credit monitoring business. They’ve heard good things about AngularJS and said they wouldn’t mind if I learned it on the job. (I love those kinds of clients!)

At this point we’ve already solidified the backend API (Scala + Play! 2.1.3) and felt like getting my hands dirty with our good friend Javascript.

At first, it was easy

I knocked the first version out quickly and got it working on the latest Chrome and Firefox on a Macbook Air. As we all know, my setup represents a small slice of humanity, so onto cross-browser testing I went.

And then, it wasn’t

I’ve been solely a backend programmer for a good few years now, and I soon remembered why: building something for to work with every browser/platform combination in the world is a royal PITA. I wasn’t going to give up, though, and the tool I used to achieve this goal comes with an A+ thumb’s up 5 stars from me. I was saved the cost and effort of running every other major browser/platform combination.

But, still — this is how it all panned out: load the page, load dev tools, and then the clicking begins. The endless clicking through thousands of lines of AngularJS and jQuery code to find the point where things fall over, and then just before the Eureka moment, the connection would drop. Or IE devtools would throw a wobbly and loose its place in the source code. All of this via a Flash frame stuck with IE devtools. IE DEVTOOLS, people! Not fun.

Which is why this isn’t a detailed bug report: I merely stumbled onto a solution after pulling all my hair out and trying something different.

The issue

Now, I’ve done exactly what the good AngularJS folks told me to do, but my scenario seemed unique. It turns out that if you use a custom directive from a partial in IE, it won’t be callable as an element (‘E’), but only as an attribute (‘A’), e.g. of a DIV.

I explain via two examples over here. If you are the lucky owner of a Windows box loaded with IE8 or smaller, please go and try it out and then lodge a proper bug report with the jQuery folks. Yes, yes — I know it sounds like this article is about AngularJS, but the real issue seemed to be with jQuery.

Something to do with target.appendChild(elem); on line 6,050 of the latest version of jQuery at the time of writing.

In the mean time, I hanker after my beloved Scala backend servers, where I get to choose exactly which platform and which flavour of VM I run on.

Failed startup #2

Jul 19, 2013

Pollarize is a decision-making platform that makes it quicker and easier for you to query your friends across multiple social networks and gather their responses into one concise answer. I co-founded pollarize at Startup Weekend London, March 2012, which we won. Shortly after, we were accepted into Wayra UK, whom gave us enough money to work on releasing pollarize on iOS and web in December 2012. Sadly, we have since run out of cash and have failed to secure more funding.

This article is a retrospective.

Startup weekend

Startup weekend final photo

That’s me in front behind the HP bag.

I won’t re-tell the story of pollarize’s beginnings, so I copy Mat’s post from the old pollarize blog:

If superhero reboots have taught me anything over the past few years it’s that people love an origin story so I thought I’d share ours with you:

At the most basic level Pollarize was born out of my long term courtship of Juan Uys, our CTO, and an indecisive iPad purchase.

Juan and I met for the first time at a Hacker News event in 2010 and I quickly identified him as the technical guru I needed to help me launch the startup I was working on. Unfortunately I discovered that Juan was working on his own project but the the two of us stayed in touch and ended up becoming friends instead.

Fast forward a year and a half and both of our previous projects have gone the way of the dodo so we decided to team up and attend London Startup Weekend. We met up towards the end of March of this year, a week or so before the event, and threw a few ideas around, including an idea borne out of my experience trying to decide which iPad version (3G vs. Extra Memory) I should buy…

Startup Weekend London was an awesome experience. I’d spoken to people that had attend similar events before so I knew that the key to doing well was forming a small, manageable, team and executing as quickly as possible. With Juan on board and taking care of the back end we set out to fill the gaps in our team as quickly as possible and recruited Michael Hobson to look after design and Sara Gozalo to focus on front end development.

We learnt a lot over the course of that weekend - I got further with a startup in those 54 hours than I had in a year of trying - but most importantly we went on to win. Winning that event had two major outcomes:

  • It was a great validation of the project and encouraged us to continue with it after the event.
  • It introduced us to the team behind Wayra UK - the weekend’s primary sponsors.

In light of this we decided to continue the project after we left Startup Weekend and also began the process of applying to accelerators.

Unfortunately we had to part ways with Sara at this point due to her other commitments and Juan quickly brought Phil Cole on board to take over front end development work.

A month or so later we made it through the final stage of selection at Wayra and became one of the first 16 teams to participate in Wayra UK.

A month after that we moved into the Wayra Academy as one of, if not the, earliest stage teams in the program and set about starting again from scratch.

It’s been 8 months since the Startup Weekend pitch that kicked all of this off and we’re still to break the ‘heads-down and execute like hell’ initiative that brought us that initial success. We have a wait list of 1,500 potential users signed up already, the second version of our iOS app in the app store and a brand new web app experience. It’s very early days for us but you can be sure that we’re giving it our best shot.

I’d like to take this quick opportunity to thank my team, our early adopters and all the people that have supported us behind the scenes - Pollarize may have been born out of an iPad purchase but it’s growing into something bigger thanks to the awesome group of people involved in the project.

Cheers,

Mat

Wayra Academy

W is for Wayra

Mat’s post above already aluded to Wayra Academy. I’ll fill in some more blanks.

Wayra was the hardest I’ve ever worked in my life. Apart from having to keep my day job (the guys didn’t — bless ‘em), I squeezed pollarize deliverables into every spare minute I had. As can be read from my old pollarize posts, I used a few technologies which enabled me to iterate very, very quickly. I worked 3 hours before work in the morning, one hour over lunch, then another few hours in the evening. Weekends were 2 ten hour days. Needless to say, I never burned out. I followed a healthy diet and went to the gym every morning to stay limber. I did regret not spending the week days in the shiny new office with the guys, but with the help of good old email, Pivotal Tracker and some Basecamp, I did a mighty good job of getting those tickets out and keeping the bugs at bay.

I rushed through hours and hours of Scala, and used some Python for some add-on NLP and recommendation projects which we never launched. Mike designed everything we threw at him perfectly, whilst also juggling what was to become his tech events business. Phil became my personal hero when he learned, developed and launched a gorgeous iOS app in 3 months. And Mat kept hustling all the way through it all, meeting, greeting, wining, dining and basically trying to secure us some funding.

We were going for the Big Bang, one-in-a-million, “get lucky” exit — something one is especially hungry for when your app is social and free.

Dublin Web Summit

'Stout' in Afrikaans means 'Naughty'

Mike won 2 Web Summit tickets, and Wayra sponsored another 2, so we were all able to attend. We implemented and refined about double the amount of features from the previous 6 months at the Web Summit weekend.

We had a successful iPhone app launch, got some good PR going, and met with more angels and investors you can shake an executive summary at.

App Store, here we come!.

We spoke, we demo’d, we networked with other startups and got the buzz going. This was the best times of our respective professional lives.

Big Bang

Big bang? No: crash and burn. We failed to secure funding. We pivoted, then re-pivoted in a panic to make ourselves matter. We desperately wanted to stick together as a team and make more stuff. We do sometimes wonder if we were in a different startup environment if things would have panned out differently. Besides, we weren’t the only ones making opinion apps: there are Amen, Thumb, Polar and others.

We started an agency called Strange, heavily inspired by BERG. All of us, however, went on to separate things, because bills needed paying, and our respective better halves couldn’t sustain us too much longer. (My story is slightly different. Having kept my day job, the missus really just needed some help with the renovations already!)

Exodus

A couple of days ago I put a stop to all our bills: Github, Pivotal, Heroku Postgres, SSL, Mongo, etc etc. The free instance on Heroku is in maintenance mode, and our splash page will remain until the domain expires, and we probably won’t renew it. The demise of pollarize is sad for the 4 of us considering the hours we put in.

We are doing great, though. Phil went on to work (and take equity in) one of the Wayra shining stars: Night Zookeeper. Mike, like I said, is crafting the best tech events, now worldwide. (Silicon Drinkabout Copenhagen, anyone?). Mat continues to conquer the world, one idea at a time. The missus and I just had a baby, and I’m planning my next move.

Now, I just have to wait for the guys to become slightly more available again so we can hack up our Next Big Thing. We did everything perfectly before, but the world just wasn’t ready. Our next product will be perfect again, but it won’t be free ;-)

Thanks for reading.