Serverside Webscripting [JLW384]

07.RESTful.APIs

RESTful APIs

The key to victory is discipline,
and that means a well made bed.

RESTful API?

An API that is REST compliant

SRSLY?

Not Sure If ...

Not sure if trolling or just “technically correct”

API

Application Programming Interface
- Abstract Interface to allow communication between software components
- eg. Photoshop has a JS based API to remotely control Photoshop
- eg. Browsers have an API that allow creation of extensions
- eg. Browsers have a Device API that give JS devvers access to device features such as geolocation, camera, vibration, etc.

Web API

Websites can also offer an API
- Allow you to push/pull data to/from a web site
- Scripting language independent (PHP, ASP, RoR, ..)
- Sometimes referred to as a “Web Service” (blame Microsoft)
- Implemented via JavaScript, XML-RPC, SOAP, REST, …
- Return formats JS, XML, HTML, plaintext, JSON(P), …

Why provide an API? (1)

Technical Limitations
- You cannot connect to a MySQL server from an iOS app you made
- You cannot connect to a MySQL server from within JavaScript
Security Issues
- It is not smart to connect to a MySQL server from an iOS app you made

Why provide an API? (2)

See cool stuff built on top of your technology platform

Let your plants tell you via Twitter that they're thirsty with botanicalls

Know when your baby kicked via Twitter with http://kickbee.net/

Why provide an API? (3)

See cool stuff built using your data

Visualize your Gowalla (RIP) checkins with GowallaHEAT

Write your own Twitter Client (and eventually get bought by Twitter)

REST

REST = REpresentational State Transfer
- Architectural style for distributed hypermedia systems described by Roy Fielding, in 2000
REST constraints
1. Client-server
2. Stateless Server
3. Cacheable
4. Uniform interface
  1. Identification of resources (URI)
  2. Manipulation of resources through representations (modify/delete)
  3. Self-descriptive messages (message and meta-data)
  4. Hypermedia As The Engine Of Application State (state transitions)
5. Layered System
6. Code-On-Demand (optional)

REST API

API that follows the REST constraints
Constraint #4 (Uniform interface) is the tricky one

Why choose REST?

As a consumer, it's easy to work with
As a developer, it's (fairly) easy to implement
It's well structured
It's popular

Figure by jmusser

Richardson Maturity Model

This chronological wang-dang-doodle
could destroy the very matrix of reality.

Richardson Maturity Model

Model that breaks down the principal elements of a REST approach in a few steps
The model embraces 4 levels
1. Level 0: The Swamp of POX
2. Level 1: Identify and organize resources in a hierarchy
3. Level 2: Use HTTP verbs for operations
4. Level 3: HATEOAS

0. The Swamp of POX

Richardson Maturity Model Level 0

What?

Use HTTP as a transport mechanism
- Mostly one URI (/api) and one METHOD (POST)
- Based on the sent-in data (in $_POST) you get back some data
POX = Plain Old XML
- Of course you can use a different format if you like
  - Plaintext is just ... plain
  - JSON is concise and quite readable too
  - JSONP does not allow custom headers and only works with GET
- → We'll go for JSON

API Call Example

Do a POST request to /api with $_POST['action'] set to getLecturers and get back a list of lecturers as the actual content

{
	"lecturers":
	[
		{
			"id": 1,
			"name": "Davy De Winne"
		},
		{
			"id": 2,
			"name": "Bramus Van Damme"
		}
	]
}

1. Resources

Richardson Maturity Model Level 1

What?

Interact with individual resources, instead of a singular resource endpoint.
- Resources are identified by a URI
- Subordinate resources are ordered in a hierarchy

Resource Identification & Hierarchy

Basic naming conventions
- Verbs are bad, nouns are good
- Use plurals
- Respect the Hierarchy
Examples
- ~~/product~~ → /products
- ~~/product/1234~~ → /products/1234
- ~~/photos/product/1234~~ → /products/1234/photos
- ~~/photos/product/1234/5678~~ → /products/1234/photos/5678

Identification & Hierarchy bis

Use concrete naming, yet don't be too specific
- ~~/things~~
- ~~/animals~~
- /dogs
- ~~/beagles~~
Some cases will require a verb instead of a noun, yet consider these exceptions
- /search?course=WMD
- /convert?from=EUR&to=USD&amount=100

URL Structure Example

/
/movies
/movies/{id}
/movies/{id}/photos
/actors
/actors/{id}
/actors/{id}/movies
/about
/contact

API Call Example

Do a POST request to /api/lecturers and get back a list of lecturers as the actual content

{
	"lecturers":
	[
		{
			"id": 1,
			"name": "Davy De Winne"
		},
		{
			"id": 2,
			"name": "Bramus Van Damme"
		}
	]
}

URL Design Sidenotes (1)

API Versioning
- Don't place it in the URL (e.g. api.twitter.com/v1/… is wrong)
- Solution: see further
Filtering & Sorting
- Use URL params ?filter=foo&sort=asc
Limiting fields
- Use an URL param ?fields=firstname,lastname,age

URL Design Sidenotes (2)

Pagination
- Mandatory for huge resources
- Use limit and offset URL params
  - Defaulting to ?limit=10&offset=<max_id>
  - Avoid using numbers for offset, opt for id-constraint based on sorting method; this avoids doubles when paginating over a frequently updated result set.

2. HTTP Verbs & HTTP Response Codes

Richardson Maturity Model Level 2

What?

Don't limit yourself to POST, but use all HTTP verbs
- e.g. POST, GET, PUT, PATCH, DELETE, …
- Depending on the HTTP verb used, a specific action will be taken
When an action has been taken, correctly use the HTTP response codes
- e.g. 404, 500, …

HTTP Verbs

HTTP Verbs = CRUD = Queries
- POST = Create = INSERT
- GET = Read = SELECT
- PUT = Update = UPDATE
- PATCH = Update = UPDATE
- DELETE = Delete = DELETE
Examples
- POST /products (data via $_POST)
- GET /products or GET /products/1234
- PUT /products/1234 (data via “$_PUT”)
- DELETE /products/1234

HTTP Verbs Sidenote: PATCH vs. PUT

What's the difference?
- PUT = replace a resource entirely
- PATCH = partially replace a resource
But ...

The PATCH method applies a delta (diff) rather than replacing the entire resource. The request entity to PATCH is of a different content-type that the resource that is being modified. Instead of being an entire resource representation, it is a resource that describes changes to apply on a resource.

Source: Please don't patch like an idiot

HTTP Status Codes (1)

When accessing a resource, you'll get back the data, along with an HTTP status code
General Codes (applicable to all verbs/resources)
- 400 — Bad Request (e.g. Request Body is malformed / not parseable)
- 401 — Not Authorized (e.g. No API Key given)
- 405 — Method Not Allowed (e.g. wrong verb for resource)
- 415 — Unsupported Media Type
- 422 — Unprocessable Entity (Validation Errors)
- 429 — Too Many Requests (Rate Limiting)
- 500 — Internal Server Error (Developer screwed up ;))

HTTP Status Codes (2)

GET (SQL SELECT)
- 200 — OK (requested data sent via response body)
- 404 — Not Found (resource not found, not “API not found”)
POST (SQL INSERT)
- 201 — Created (=OK)
- 400 — Bad Request (Missing Parameters)

HTTP Status Codes (3)

PUT (SQL UPDATE)
- 200 — OK (requested data sent via response body)
- 400 — Bad Request (Missing Parameters)
- 404 — Not Found
DELETE (SQL DELETE)
- 204 — No Content (=OK — Note: no response body sent!)
- 404 — Not Found
Warning!
- HTTP Statuses 4XX and 5XX will invoke $.ajax().error() in jQuery!

API Call Example (1)

Do a GET request to /api/lecturers and get back an HTTP 200 status code and a list of lecturers as the content

{
	"lecturers":
	[
		{
			"id": 1,
			"name": "Davy De Winne"
		},
		{
			"id": 2,
			"name": "Bramus Van Damme"
		}
	]
}

API Call Example (2)

Do a POST request to /api/lecturers with some data in $_POST to insert a new lecturer
- Get back 201 — Created if all OK
- Get back 400 — Bad Request if parameters are missing
Do a PUT request to /api/lecturers/2 with some data in $_PUT to update a lecturer
- Get back 200 — OK if all OK
- Get back 400 — Bad Request if parameters are missing
- Get back 404 — Not Found if the lecturer does not exist

3. Hypermedia Controls

Richardson Maturity Model Level 3

What?

Represent resources using a Hypermedia Format.
- Hypermedia = The simultaneous presentation of information and controls such that the information becomes the affordance through which the user obtains choices and selects actions ⚑
Consequence: HATEOAS
- HATEOAS = Hypermedia as the Engine of Application State
- Application State = Where the user is in the process of completing a task (e.g. a page of the application)
Roughly translated:
- The response must be hypertext and include links so that the user/automation can transition to other pages of the application

HATEOAS (2)

Essentially, an API consumer should only know
1. The URI starting point
2. How to parse the response (viz. what media type it is)
With this, the consumer has enough info to parse out the links and transition to other states when needed
When an API embraces HATEOAS, we can call it a Hypermedia API

Response Media Types (1)

Responses must⚑ be hypertext to be RESTful
- XML and JSON are not hypermedia types; they only define a syntax
- (X)HTML, ATOM, RDF/XML, etc. are hypermedia types
  - These are all specified
  - Each has their own MIME Type, e.g. application/atom+xml
JSON based hypermedia types are in the making
- Collection+JSON
- Siren
- HAL
- JSON-LD
- …

Response Media Types (2)

It's perfectly possible to roll your own hypermedia type
- e.g. application/vnd.ikdoeict.whatever
- General consensus amongst hypermedia gurus: don't
An API can support multiple hypermedia types
- The client indicates in an accept header which
  - e.g Accept: application/collection+json
- Content negotiation:
  - e.g. Accept: application/xml; q=0.8, application/json
- Also use this header for versioning
  - e.g. Accept: application/vnd.ikdoeict.whatever-v2+json
    or Accept: application/collection+json;v=2

What will we do?

We're going to break the rules and use our own format
- JSON based
- Served as plain application/json (Sorry Roy, gotta pave the cowpaths)
- The returned HTTP status code will also be part of the response
  (but we'll also send it via the response headers as RMM Level 2 requires)
  - The response is split out in a status and content object
- When links are needed, inject a links array holding the links
  - Links have a rel and href property
  - rel values are based on RFC5988
    - self, up, index, next, prev, ...
    - When no value can be found in the RFC, pick your own (e.g. photos)

API Call Example

Do a GET request to /lecturers and get back a custom JSON response with HTTP status code 200 containing a list of lecturers (with resource links) as the response content

{
	"status":
	{
	   "code": 200,
	   "text": "OK"
	},
	"content":
	{
		"size": 2,
		"lecturers":
		[
			{
				"id": 1,
				"name": "Davy De Winne",
				"links": [
					{
						"rel": "self",
						"href": "/users/1"
					}
				]
			},
			{
				"id": 2,
				"name": "Bramus Van Damme",
				"links": [
					{
						"rel": "self",
						"href": "/users/2"
					}
				]
			}
		]
	}
}

Summary

Real RESTful APIs or Hypermedia APIs are APIs that consist of two things
1. Usage of HTTP to its fullest (RMM Levels 0 - 2)
2. Responses are served as hypermedia that manages application state (RMM Level 3)
We'll be RMM level 3 compliant-ish and use our own (simple) hypermedia format but serve it as application/json — It cuts the mustard

RESTful API Case Study

Hey, look! I found a robot fossil!

Twitter API (1)

One of the first RESTful APIs, suffering from historically grown bloat.
- Spoiler: it's not RESTful at all!
Get a status
- Endpoint
  - GET api.twitter.com/1/statuses/show/id.format
- Problems
  - API Version in URL
  - Operation show included in URL
  - id not a child of statuses
  - Content Negotiation via the URL (.format at the end)
- Better
  - GET api.twitter.com/statuses/{id}

Twitter API (2)

Set (update) your status
- Endpoint
  - POST api.twitter.com/1/statuses/update.format
- Problems
  - API Version in URL
  - Operation update included in URL
  - Content Negotiation via the URL (.format at the end)
- Better
  - POST api.twitter.com/statuses

MoviesDB (1)

MoviesDB (2)

Example API
Play around with the API Explorer to check it out
- FYI: You can Fork the API Explorer
Recommended tool: Chrome Postman plugin

RESTful APIs in PHP

Would it cheer you up if I punch Fry in the groin?
Cause I'll do it, regardless.

It's all about the routes

What we essentially need is a router of some sorts that takes the URL and the HTTP method into account
- If you do a GET request to /movies, show a list of movies
- If you do a GET request to /movies/{id}, show the one specific movie
- If you do a POST request to /movies, insert it
- …

Codewise, it should be something like this

$router = new Router();

$router->get('/', function() { … });
$router->get('/movies', function() { … });
$router->get('/movies/\d+', function() { … });
$router->post('/movies', function() { … });

$router->run();

It's all about the responses

To easily work with the (semi-)hypermedia format we've described, let's introduce a Response class
Class requirements
- Provide $status and $content datamembers + mutators
- Keep list of allowed HTTP Status Codes
- By default, set status to 200

All together now

Routes?
→ Silex does this!
Response?
→ Silex uses \Symfony\Component\HttpFoundation\Response!⚑
- Just extend this class and:
  - Override getContent() to transform the content to the format we need
  - Force sending of the Content-Type: application/json header

One more thing

There's no such thing as $_PUT, you'll need to fake it

<?php

//…

$router->put('/movies/\d+', function() {

	// Fake $_PUT
	$_PUT  = array();
	parse_str(file_get_contents('php://input'), $_PUT);

	// …

});

//…

RESTful APIs in JavaScript

To the flying machine!

RESTful APIs in jQuery

Since the API returns both OK and errorous HTTP status codes, you need to hook both success and error functions

$.ajax({
	url : 'http://api.myapp.tld/',
	type: 'get|post|put|delete',
	dataType : 'json',
	data: 'foo=bar'
})
.success(function(data, textStatus, jqXHR) {
	if (data) {
		// All OK and data returned (200, 201)
	} else {
		// All OK but no data returned (e.g. 204 after a DELETE)
	}
})
.error(function(jqXHR, textStatus, errorThrown) {
	var data = jqXHR.responseText ? JSON.parse(jqXHR.responseText) : {};
	// Got a 400 (params), 401, 404, 405 (method), 500, or 501 (not impl)
});

TIP: Since we include the status in the response, you can easily check it via data.status.code

CORS

API Authentication

Who are those horrible orange creatures over there? — Grunka-Lunkas. — Tell them I hate them!

API Authentication

Two needs for authentication
1. API Consumers (apps using your API)
2. User Authentication (allowing apps using your API to act on behalf of a user)

1. API Consumers (1)

Make use of API-Keys

Require API-Key to be sent in via a header
Check API-Key as a before middleware on all routes

$router->before('GET|POST|PUT|DELETE', '.*', function() use ($response) {
	$headers = apache_request_headers();
	if(!isset($headers['X-Api-Key']) || !ApiDB::isValidApiKey($headers['X-Api-Key'])) {
		$response->setStatus(401);
		$response->setContent('Missing or invalid API Key.');
		$response->finish();
	}
});

Note: Custom headers must start with X and all words must be ucfirst'd and separated with -

1. API Consumers (2)

Note: apache_request_headers() not always available (IIS)!

Paste in the code below to always have it available

if (!function_exists('apache_request_headers')) {
	eval('
		function apache_request_headers() {
			foreach($_SERVER as $key=>$value) {
				if (substr($key,0,5)=="HTTP_") {
					$key=str_replace(" ","-",ucwords(strtolower(str_replace("_"," ",substr($key,5)))));
					$out[$key]=$value;
				}
			}
			return $out;
		}
	');
}

2. User Authentication (1)

API Consumers should never ask the user his/her username/password (password anti-pattern!) but should use OAuth2 which works with time-limited access tokens to accessing user data

2. User Authentication (2)

OAuth2, schematic

2. User Authentication (3)

OAuth2 in words
1. User is sent to the OAuth provider (along the consumer app's API-Key (client_id) for identification), asking if consumer app may access user data at a certain scope (read/write)
2. If user accepts the OAuth provider will redirect back to the consumer app's redirect_uri with an authorization_code
3. In the background, the consumer app must make a request to the OAuth provider with that authorization_code, along with its API-Key (client_id) and secret key (client_secret)
4. The OAuth provider will issue an access_token and refresh_token
5. The access_token must passed with all calls made and is valid for a limited time
6. When the access_token has expired, the consumer app needs to do a call to the OAuth Prodider with the refresh_token to get a new pair access_token & refresh_token

API Authentication

API-Keys are easily implemented, both consumer and provider side
Once you get the gist of it, OAuth2 is fairly easy …
- Roll your own implementation, or use a prefab solution (#lmgtfy)
- Remark: Treat OAuth tokens as passwords and encrypt them!

Questions?

Sources

ikdoeict.be