On the modern web, we almost take a well designed RESTful JSON-based API for granted. But what actually goes into designing a good API? Is it really just one simple formula or are all APIs not created equal? Following is some of the thoughts and considerations we went thru, designing the new Iconfinder API.
Mostly any respectable Web service these days has an API – be it Twitter or Facebook, or much smaller sites such as Dribbble. And, for good reason. The Web as we know it today is no longer just a bunch of silos with each their own data, but rather an amazing constellation of integrated services, where your data can follow you around. This makes for the user experience we’ve gotten so used to, no matter if you just want your latest Instagram photos on your personal site or you, like our friends at Lucidchart, run a massive diagramming tool and want users to be able to easily use illustrations in their diagrams.
Iconfinder has since 2010 had a minimal API exposed to developers. Over the years, we’ve seen some pretty cool things built on top of this simple API. However, as Iconfinder has grown immensely in terms of features and complexity, just as the Web has evolved greatly in terms of what we as developers regard as a “good” API, it became clear that we needed to give the developers out there something more – something better.
But, with the new Web landscape, designing an API is no longer just a question of making “something” available. A good API nowadays has to be well designed, and has to keep its options open for the future.
The illusive REST
The world of software development is awash with abbreviations – some of them clear, some of them not, and some of them which deterioate over time. One such term is REST, which you’ve surely come across if you’ve as much as glanced at any of the previously mentioned APIs. A REST API, or more commonly RESTful API, is in its most strict of senses an API that follows the Representational State Transfer architectural style. Confused? Good. So is the rest of the Web, really, despite this term now being slapped on almost any product out there.
You see, calling the Iconfinder API 2.0 – or most any other API out there – RESTful is technically a misnomer, when the term is considered in its original academic sense as first described by the father of the term, Apache HTTP Server co-founder Roy T. Fielding. The reason for this is that, just like many other strict, academic concepts, REST at its purest is hard and makes implementation of both server and client code needlessly complex (give Fielding’s summation of the rules of pure REST a read for a solid headache.)
What’s happened, as it so often does, is that the Web community has taken the best and most practical parts of REST, added a bit of extra “best practise” and kept the label. As Michael Bleigh points out, this is absolutely okay. In fact, I’d dare say it’s the best general API structure we have come up with so far, which is why we decided to latch onto the popular definition of REST (which one might be tempted to call a “resourceful API” instead, but I digress), which follows a few simple rules (people might have opinions on more specific rules):
- The API at its base exposes resources. A resource can be basically any entity in your application – for Iconfinder, this includes icons, icon sets, users etc.
- Resources have a well-defined URI, and resources are the structural components of URIs. Icons for example are available under
/iconswith a specific icon being adressable as
/icons/<unique icon ID>. The same goes for resource relationships, with the icons in a specific icon set being adressable as
/iconsets/<unique icon set ID>/icons– see the pattern?
- HTTP request methods are used to indicate the action to perform on a resource. Commonly,
GETis used to retrieve a resource,
POSTto create one,
PUTto replace one,
PATCHto update one, and
DELETEto get ride of one.
The end result of following these rules is a nice, logical URI structure with requests making sense.
GET /iconsets/5261 gives you the information about Timothy Miller’s awesome Wthr Color icon set. If you were Timothy Miller,
PATCH /iconsets/5261 would let you update some of this information. Nice, readable and – probably more importantly – easy to play around with using a tool like curl or even jQuery.
Shortcomings of REST
For simple CRUD (Create-Read-Update-Delete) applications, in which resources are accessed in a straight-forward manner, this set of rules is usually enough. But, more often than not, you may have some unique functionality, that doesn’t fit these simple rules, like, say, exposing a way to search for icons. As the HTTP specification does not offer a
SEARCH request method, we have to come up with some other way of making this possible.
This is one of the definite shortcomings of REST – a shortcoming, so-called Remote Procedure Call or RPC APIs like WSDL based services don’t have, because these consider everything to be an action with resources simply being the result of an action – like “getting an icon” or “search for an icon.” Those of you who’ve actually worked with this kind of API however knows that it’s a hell hole that we’re thankful REST helped us get out of, for so many reasons. The most prominent reason is the massive complexity overhead in explicitly having to define actions for common things like “getting an icon,” with no accepted “community standard” for how to do this, which REST gives us for free.
So, when you don’t want RPC but REST can’t cut it, what do you do? Well, this is where design gets tricky and you need to meet somewhere in the middle. Different APIs have taken different approaches to this, but generally always in the form of defining custom actions like search as what would appear to be a unique resource, such as:
This might seem ugly, but if we don’t want to violate the idea of using resources as the structural URI component, endpoints like
/search-icons simply won’t do. The Elasticsearch API has taken an interesting approach in prefixing custom actions with an underscore and disallowing resource names beginning with an underscore to make it easy to distinguish between actions and actual resources. As we only use numerical unique IDs, we picked the easy way out and decided to simply use a verb to describe the action, such that searching for icons means accessing
OAuth 2.0, of course
Compared to two years ago, being a user on Iconfinder now comes with a lot of extra functionality. For example, you can now buy premium icons, favorite icons and so forth. So, of course we wanted to make it possible for third parties to take advantage of all of this. However, this requires giving developers a way to authenticate a user with the Iconfinder API.
A handful of years ago, the Web was awash with all sorts of custom authentication systems until 2007 when OAuth slowly started making its way onto the stage, mainly driven by Twitter. OAuth provided a great standard for authenticating between Web applications, albeit in a somewhat cumbersome way – especially for newcomers. To address this issue, and the issue of OAuth only really working for Web applications, which once again lead to a wealth of non-standards like XAuth for mobile and desktop applications, a new, more comprehensive standard, OAuth 2.0, was introduced.
Nowadays, OAuth 2.0 is the de facto standard for doing authentication against Web services. Facebook, Twitter, Google, Dropbox and virtually any other big API provider out there use it. Almost every programming language you can think of has great libraries for using it. Really then, it was a no-brainer for the Iconfinder API 2.0 to also pick OAuth 2.0 for authentication.
APIs are forever
Okay, maybe not quite at the level of diamonds, but remember: when designing an API, you should be aware that if people start using it, they’re probably going to use it for quite a long time. So, do the hard work up front – it’ll keep your options open down the road and hopefully make your API relevant and useful for a long time to come. At least, that’s what we hope for with the Iconfinder API 2.0.
That being said, versioning is almost unavoidable as things change and evolve. Sadly, though, we haven’t yet settled on a community standard for how to do this. Some swear by prefixing URIs for an API version, such that icons in API version 2 are actually addressed as
/v2/icons rather than
/icons. Others swear by a much more recent approach, which involves using so-called vendor media types.
If you look at the HTTP response for a normal HTML page, you’ll notice that it usually has a
Content-Type header – it’s media type – of
GET / HTTP/1.1
HTTP/1.1 200 OK
Similarly, a CSS file is identified as
text/css and a PNG image
image/png. These content types are part of an agreed upon Internet standard, which makes it easy for browsers to figure out exactly what it is receiving. This standard also allows for developers to be a bit more creative with content types, as long as they start with
application/vnd.. Like any standard, it’s a bit of a murky read, but the trick is that we can rely on HTTP to make it easy for us to get the data we want in the version we want. Just like the previously mentioned
Last-Modified headers provide related control mechanisms, we can use the
Accept header to specificy the content type(s) we want. Suppose we have a version 2.0 and 2.1 of the icon resource, we could give these media types of
application/vnd.iconfinder.v2.1+json respectively. Depending on what the
Accept header contains, users of the API can then easily explicitly pick a version and be sure that it stays the same way forever.
The truth of the matter is, that it’s hard to argue that the vendor media type approach isn’t smart. But, on the other hand, it’s also pretty complex compared to just appending
/v2 to the resource URIs. Like so many design choices, the right solution is probably somewhere between those two extremes, something which Github seems to be embracing. For now, the Iconfinder API 2.0 is still brand new, so, we decided to take the safe approach and keep our options open. Therefore, you’ll see that all API 2.0 URIs are indeed prefixed with
/v2. In the mean time, we’re waiting to see what the Web as a whole does until we actually need finer grained versioning.
Documentation is half the API
While getting the design of the API “right” was our primary concern, we also had another learning from the previous version of the Iconfinder API as well as using a host of good and bad APIs out there: documentation is key.
The initial Iconfinder API suffered heavily from being underdocumented, which meant that we were mortally afraid of changing it in any way. When you don’t specify exactly how things work, users start making assumptioms. If you then change something, chances are that you’ll wind up breaking people’s integrations with your API – the exact opposite of what you’re after, unless of course you’re Facebook and don’t really give a damn.
Traditionally, RESTful APIs have been primarily documented via comprehensive documentation such as Instagram’s beautiful developer site. RPC style APIs have on the other hand been much more strictly defined with for example machine-readable languages like WSDL (Web Service Definition Language) which makes it possible to generate documentation and even complete clients from the specification. In theory, this is an awesome approach – it should make everything nice and easy for everyone – so of course attempts are now being made to replicate this with RESTful APIs, partly driven by the fact that Fielding’s original intention was for academically proper RESTful APIs to be “discoverable.” These attempts are currently covered by the “hypermedia” umbrella term, and while great in theory, any developer who’s had to actually work with for example WSDL will tell you to “go to hell.” Furthermore, many of the current attempts at combining these concepts, like the proposed Hypertext Application Language standard, involve sending a lot of often useless data over the wire, when all the user wants is the actual resource data. A waste of everyone’s time and energy. David Heinemeier Hansson of Basecamp fame summed up the arguments against hypermedia beautifully:
Let’s not repeat the same mistakes twice. Some times fewer standards and less ceremony is exactly what’s called for.
When designing the API 2.0 we therefore decided to go for the “old-school”, humanly written documentation approach. This gave us the added benefit of using it as sort of a specification for how we wanted everything to work, rather than doing it as an afterthought, as well as the opportunity to invest some time into trying to make it really nice rather than just what a tool could spit out. A lot more work, for sure, but we’re pretty damn happy with the result.
We’re pretty lucky
Designing and building an API these days is in many ways not trivial and requires a lot of time of effort. But, luckily, the Web as a whole is slowly maturing and coming up with great, generalized solutions to a lot of the problems involved with APIs. In that sense, designing and building a good API is a lot easier today than it was five years ago. We’re pretty lucky.