Building APIs

Lalit Kapoor

CTO, Goodybag

Goodybag

We're a neato tech-startup in
Austin, TX

Goodybag

We pretty much just write {code} all day.

We of course wrote the
Goodybag API - which powers:

  • Our Loyalty Platform
  • Our Consumer and Business web interface
  • Our Mobile Application
  • Tons of other stuff too

What
are API's?

Hold on to your beers. We're about to find out.

What are APIs?

APIs are the exposed pieces of functionality within a given system

  • Each function performs a single task
  • Set of functionality enabling products and services
  • For example, Twitter has an awesome API

What are APIs?

Why do I need an API?

  • Streamline functionality across multiple products (e.g. Mobile App, Admin Panel, Analytics Dashboard, etc.)
  • Separate back-end logic from the front-end interface
  • Consolidate a system's critical functional pieces
  • Modular application design
  • Makes Unit Testing a breeze

What are APIs?

Enough, with the fancy talk, Lalit, let's dig in.

How to Build an API

...Like a Boss

  • Lots of planning
  • And more planning
  • And then plan some more

How to Buid an API

Determine which system functions to expose

  • Expose only what is necessary
  • Be as light-weight as possible (minimalism is the best way to start)

How to Buid an API

Determine how you will expose your API

Is it important for your API to work on the web?

  • Good API's are the ones that are easy to work with
  • Serve up just the data, no extraneous HTML markup
  • Use open standards like HTTP and REST
  • Use well-known formats like JSON and XML to exchange information

How to Buid an API

Determine which technologies you will use

At Goodybag, we chose Node.js and MongoDb

Why did Goodybag choose Node.js and MongoDb to build their API?

node.js and mongo

Why Node.js

Because Javascript is flippin' awesome

Node is a V8 powered Javascript framework designed for writing event-driven and network based applications.

V8 is just Google's Javascript engine.

node.js and mongo

Why Node.js

Because Javascript is flippin' awesome

  • It's very light-weight
  • Our applications are mostly I/O bound
  • We do a lot of network communication
  • Node's event loop is great for 1000s of network connections on a single machine
  • Fast growing community
  • Code re-usability

node.js and mongo

It's not all peaches and cream

There are some pain points when using node

  • Change to an asynchronous work-flow
  • Callback Hell - we use async.js to assist with control-flow
  • Exceptions can get messy
  • CPU intensive tasks will hold up everyone!

node.js and mongo

Why MongoDb?

WHY DID YOU NOT USE MYSQL?!

  • Schema-less designs means we can be flexible with our data model as we iterate quickly and stabilize
  • Nested documents
  • BSON/JSON
  • Decent Querying interface

node.js and mongo

Challenges with MongoDb

  • Lack of transactions - we did some custom engineering to resolve those issues
  • Cannot run operations on data during an update
  • Cannot easily retrieve specific items from an array in a document - some solutions might exist in release v2.2 - aggregation framework, look at v2.1

Let's build a miniature
Twitter Platform

Twitter Platform

Identify the components of our system

  • Users
  • Tweets (140 character status messages)
  • Streams (user's own tweets + the tweets of people the user follows)

Twitter Platform

What functionality do we want to support

  • [public] We want people to be able to register and create unique identities
  • [public] We want users to login before they can do anything with their own accounts
  • [public] We want users to be able to view information about other users
  • [public, auth] We want users to be able to follow other users

Twitter Platform

What functionality do we want to support continued

  • [public, auth] We want users to be able to post 140 character status messages that are visible to the public
  • [public] We want to be able to retrieve single tweets
  • [private] We want tweets published into our follower’s streams
  • [public, auth] We want to be able to view our own streams, but no one else's

Twitter Platform

Determine how to expose the public functions

We want to consume JSON requests and return data in a JSON format as well

Functionality

Determine how to expose the public functions

Users

  • register (alias, email, password, name, url)
    returns successful or error code
  • login (alias, password)
    returns success or error code
  • get (alias)
    returns information about the user (in this case name, email address, list of follower’s aliases) or an error code
  • follow (alias, followee_alias)
    returns successful or error code

Functionality

Determine how to expose the public functions

Tweets

  • get (id)
    returns a single tweet message or error code
  • list (alias)
    returns a list of tweets for a specific user sorted by most recent first or an error code
  • add (alias, message)
    returns successful or error code

Functionality

Determine how to expose the public functions

Streams

  • list (alias)
    returns a list of tweets for the alias and the users that alias follows sorted by most recent first or an error code
  • addTweet (alias, message)
    add a tweet message to the specified user’s stream

Twitter Platform

Determining the Routes

  • [POST] - api.mini-twitter.com/v1/users/register
  • [POST] - api.mini-twitter.com/v1/users/login
  • [GET] - api.mini-twitter.com/v1/users/:alias
  • [POST] - api.mini-twitter.com/v1/relationships/follow
  • [POST] - api.mini-twitter.com/v1/tweets
  • [GET] - api.mini-twitter.com/v1/tweets{?alias=:alias}
  • [GET] - api.mini-twitter.com/v1/tweets/:id
  • [GET] - api.mini-twitter.com/v1/tweets/stream

Twitter Platform

Setup the initial app

var express  = require('express')
  , mongojs  = require('mongojs')
  , sanitize = require('./middleware/sanitize')
  , users    = require('./routes/users')
  , tweets   = require('./routes/tweets')
  , streams  = require('./routes/streams')
  , app      = express.createServer();

mongojs.connect(process.env['MONGO_URL']);

app.use(express.bodyParser());
app.use(express.cookieParser());
app.use(express.session({ secret: 'secret'}));
app.use(express.methodOverride());

Twitter Platform

Expose the Routes

app.post('/v1/users/register', sanitize, users.register);
app.post('/v1/users/login', sanitize, users.login);
app.get('/v1/users/:alias', sanitize, users.get);
app.post('/v1/relationships/follow', auth.check, sanitize, users.follow);
app.post('/v1/tweets', auth.check, sanitize, tweets.add);
app.get('/v1/tweets', sanitize, tweets.list);
app.get('/v1/tweets/:id', sanitize, tweets.get);
app.get('/v1/tweets/stream', auth.check, sanitize, streams.list);

app.listen(parseInt(process.env['PORT_WWW']));

Twitter Platform

Handle the Routes

users.js

// users.js

var api    = require('./api')
  , routes = module.exports;

routes.register = function (req, res) {
  api.users.register(req.body, function(err)){
    if (err) {
      res.send(err.message, 500);
      return;
    }
    res.send(200);
  }
}

Twitter Platform

Handle the Routes

api/users.js

// api/users.js

...
op.add = function (data, callback) {
  var user = {
    email: data.email
  , password: data.password
  , alias: data.alias
  , name: data.name
  };

  db.users.schema.validate.add(user), function(err){
    if (err) callback(errs.SCHEMA_VALIDATION_ERROR);
    else db.users.collection.insert(, { safe: true }, callback || noop);
  });
};
...

Twitter Platform

Don't forget

  • Build clients to your APIs
  • Write unit tests
  • Thorough documentation means high adoptability

Platforms

And that's it - You've built an API

You've built a platform for succinct communication.

Now you can:

  • Built web or mobile apps
  • Completely separate products that utilize your platform

Platforms

Building a platform is no easy task - we only touched on a few components

Each system will have its own set of challenges. It's important that you identify them and address them head on.

Platforms

Our platform was simple, but the real world is much crazier

Twitter has TONS of users. So when Justin Bieber tweets, All hell breaks loose.

Twitter has obviously optimized for the Bieber Case.

The Goodybag Platform

Goodybag our platform faced some of the following challenges

  • Financial matters – consistency, reliability, and availability
  • Low latency responses
  • And most important to us is security and privacy

Keeping these things in mind we thought hard about our technologies and setting up our architecture.

Let's see a TapIn at
Goodybag

Adding Value

Every one of our products uses our API

  • Consumer Site - SPA
  • Business Site - SPA
  • Consumer Mobile App (Native/Web) – iOS/Android/Web
  • Business Mobile App (Native) – iOS/Android/Web
  • Tablet App (Native) – Java (Where actual redemption occurs)
  • Admin Interface – SPA
  • Analytics Dashboard - SPA

So, let's recap

Why you should build APIs almost all of your apps

  • You don’t look like a n00b – Just Kidding
  • Awesome segregation of code
  • It’s all about enabling others to doing things
  • Enables you to hire more than just full stack engineers
  • Easily integrate with other services
  • Enables 3rd party developers
  • Even Obama is requiring all Federal Agencies to build APIs