The Story of a Thousand Services

@brandur

thousand-services.herokuapp.com

2013

Sure .. we have a public API ..

Grown From the CLI

heroku create

A Diverse API

Ecosystem Sprung Forth

Successful Login

$ curl -X POST https://api.heroku.com/login \
  -d "email=..." -d "password=good"
HTTP/1.1 200 OK
Content-Type: application/json

{ "email": "..." }

Unsuccessful Login

$ curl -X POST https://api.heroku.com/login \
  -d "email=..." -d "password=bad"
HTTP/1.1 404 Not Found
Content-Type: text/plain

""
$ curl -X POST https://api.heroku.com/login \
  -d "login_user[email]=..."

or . . .

$ curl -X POST https://api.heroku.com/login \
  -d "username=..."

or . . .

$ curl -X POST https://api.heroku.com/login \
  -d "email=..."

Read. Think. Discuss.

UUIDs

vs.

Integer IDs

Request Bodies

application/json vs.

application/x-form-www-urlencoded

Naming Things

There are two hard problems in computer science ...

Nullable Request

Parameters

URL vs. Header Versioning

V3

Re-use

Convention can be hard-earned

Design Guidelines

Principles derived from designing the public API.

github.com/interagent/http-design-guidelines

Appropriate Status Codes

201 or 202 on create; 200 elsewhere.

Paginate With Range

Range: id ..; max=1

Times in ISO8601

2012-01-01T12:00:00Z

Downcase paths and attributes

Hyphens in paths (e.g. ssl-endpoints); underscores in params (e.g. created_at)

Support Caching with Etag

If-None-Match: ...

Transparent Rate Limit Status

Include RateLimit-Remaining header

JSON Schema

json-schema.org

Validates a JSON Value

{
  "type": "string"
}

✓ "foo"

✗ 42

More Complex Rules

{
  "pattern": "^[a-z][a-z0-9-]{3,30}$",
  "type": "string"
}

✓ "my-app"

✗ "my_app"

Schemas Nest

{
  "properties": {
    "name": {
      "pattern": "^[a-z][a-z0-9-]{3,30}$",
      "type": "string"
    }
  },
  "required": ["name"],
  "type": "object"
}

✓ { "name": "my-app" }

✗ {}

Uses JSON Reference

{
  "definitions": {
    "name": {
      "pattern": "^[a-z][a-z0-9-]{3,30}$",
      "type": "string"
    }
  },
  "properties": {
    "name": {
      "$ref": "#/definitions/name"
    }
  },
  "required": ["name"],
  "type": "object"
}

`$ref` under "properties" is a JSON Reference to "definitions"

Schemas Nest to Any Depth

{
  "definitions": {
    "domain": {
      "properties": {
        "name": {
          "format": "hostname",
          "type": "string"
        }
      }
    },
    "app": {
      "properties": {
        "domains": {
          "items": {
            "$ref": "#/definitions/domain"
          },
          "type": "array"
        }
      }
    }
  }
  ...
}

{
  "name": "my-app",
  "domains": [
    { "name": "example.com },
    { "name": "heroku.com" }
  ]
}

Suitable for all kinds of validation

JSON

HYPER

Schema

Gives Schemas Links

{
  "links": [
    {
      "description": "Create a new app.",
      "href": "/apps",
      "method": "POST",
      "rel": "create"
    },
    {
      "description": "List apps.",
      "href": "/apps",
      "method": "GET",
      "rel": "instances"
    }
  ],
  "properties": {
    "name": {
       ...
    }
  },
  "required": ["name"],
  "type": "object"
}

Schema of a Link's Request

{
  "description": "Create a new app.",
  "href": "/apps",
  "method": "POST",
  "rel": "create",
  "schema": {
    "properties": {
      "name": {
        "$ref": "#/definitions/app/definitions/name"
       }
     },
     "required": ["name"],
     "type": "object"
  },
  "title": "Create"
}

$ curl -X POST http://example.com/apps \
  -H "Content-Type: application/json" \
  -d '{"name":"my-app"}'

Schema of a Link's Response

{
  "description": "List apps.",
  "href": "/apps",
  "method": "GET",
  "rel": "instances",
  "targetSchema": {
    "items": {
      "$ref": "#/definitions/app"
    },
    "type": "array"
  },
  "title": "List"
}

Note the reference to the link's parent

$ curl -X GET http://example.com/apps
[
  {
    "name": "my-app",
    "domains": [
      { "name": "example.com },
      { "name": "heroku.com" }
    ]
  }
]

Check out Heroku's!

$ curl -H "Accept: application/vnd.heroku+json; version=3" \
  https://api.heroku.com/schema -o schema.json

Meta-schemas

http://json-schema.org/draft-04/schema#

http://json-schema.org/draft-04/hyper-schema#

JSON Which Validates JSON Which Validates JSON

{
  "definitions": {
    "domain": {
      links: [
        ...
      ]
      ...
    },
    "app": {
      links: [
        ...
      ]
      ...
    }
  }
  ...
}

Define Your Own!

{
  "definitions": {
    "resource": {
      "properties": {
        "links": {
          "items": {
            "$ref": "#/definitions/link"
          },
          "type": "array"
        }
      }
    },
    "link": {
      "required": [ "method", "targetSchema" ],
      "type": "object"
    }
  },
  ...
}

Requires links to have `method` and `targetSchema`

{
  "resource": {
    "properties": {
      ...,
      "properties": {
        "additionalProperties": false,
        "patternProperties": {
          "^[a-z][a-z_]+[a-z]$": {}
        }
      }
    }
  }
}

Requires params to use only `[a-z]` and underscores

✓ parameter_name

✗ parameter-name

Check out Heroku's Meta-schema!

$ curl https://interagent.github.io/interagent-hyper-schema -o meta.json

Human + Machine Readable

Heroics

Ruby client generation

github.com/interagent/heroics
$ heroics-generate schema.json > client.rb
> heroku.app.create
=> { ... }

Schematic

Go client generation

github.com/interagent/schematic
$ schematic schema.json > client.go
> heroku.app.createapp, err := h.AppCreate()
if err != nil {
   panic(err)
}
fmt.Println(app.Name)

Prmd

Documentation & tools

github.com/interagent/prmd

Committee

Validation & Stubbing

github.com/interagent/committee

$ committee-stub -p 3000 schema.json
$ curl http://localhost:3000/apps/my-app
{
  "created_at": "2012-01-01T12:00:00Z",
  "git_url": "git@heroku.com:example.git",
  "id": "01234567-89ab-cdef-0123-456789abcdef",
  "name": "example"
}

Stub Distributed Services

help_stub:    bundle exec committee-stub -p 3000 help/schema.json
logplex_stub: bundle exec committee-stub -p 3001 logplex/schema.json
maestro_stub: bundle exec committee-stub -p 3002 maestro/schema.json
psmgr_stub:   bundle exec committee-stub -p 3003 psmgr/schema.json
vault_stub:   bundle exec committee-stub -p 3004 vault/schema.json

Test Public Contracts

def schema_path
  "./schema.json"
end

it "conforms to schema" do
  @app = App.create! name: "my-app"
  get "/apps/my-app"
  assert_equal 200, last_response.status
  assert_schema_conform
end
  • HTTP API guidelines codifies best practices
  • Schema generates clients
  • Schema generations documentation
  • Schema stubs remote services
  • Schema tests contracts
  • Meta-schema validates conventions

Sell Internally

@brandur