Web & Mobile Development JLW288

07.RESTful.APIs

RESTful APIs

RESTful API?

• An API that is REST compliant

SRSLY?

Not sure if trolling or just “technically correct”

API

• Application Programming Interface
  • Abstract Interface to allow communication between software components
  • eg. Photoshop has a JS based API to remotely control Photoshop
  • eg. Browsers have an API that allow creation of extensions
  • eg. Browsers have a Device API that give JS devvers access to device features such as geolocation, camera, vibration, etc.

Web API

• Websites can also offer an API
  • Allow you to push/pull data to/from a web site
  • Scripting language independent (PHP, ASP, RoR, ..)
  • Sometimes referred to as a “Web Service” (blame Microsoft)
  • Implemented via JavaScript, XML-RPC, SOAP, REST, …
  • Return formats JS, XML, HTML, plaintext, JSON(P), …

Why provide an API? (1)

• Technical Limitations
  • You cannot connect to a MySQL server from an iOS app you made
  • You cannot connect to a MySQL server from within JavaScript
• Security Issues
  • It is not smart to connect to a MySQL server from an iOS app you made

Why provide an API? (2)

• See cool stuff built on top of your technology platform

  

  

Why provide an API? (3)

• See cool stuff built using your data

  

  

REST

• REST = REpresentational State Transfer
  • Architectural style for distributed hypermedia systems described by Roy Fielding, in 2000
• REST constraints
  1. Client-server
  2. Stateless Server
  3. Cacheable
  4. Uniform interface
     1. Identification of resources (URI)
     2. Manipulation of resources through representations (modify/delete)
     3. Self-descriptive messages (message and meta-data)
     4. Hypermedia As The Engine Of Application State (state transitions)
  5. Layered System
  6. Code-On-Demand (optional)

REST API

• API that follows the REST constraints
• Constraint #4 (Uniform interface) is the tricky one

Why choose REST?

• As a consumer, it's easy to work with
• As a developer, it's (fairly) easy to implement
• It's well structured
• It's popular

  

Richardson Maturity Model

Richardson Maturity Model

• Model that breaks down the principal elements of a REST approach in a few steps
• The model embraces 4 levels
  0. Level 0: The Swamp of POX
  1. Level 1: Identify and organize resources in a hierarchy
  2. Level 2: Use HTTP verbs for operations
  3. Level 3: HATEOAS

0. The Swamp of POX

What?

• Use HTTP as a transport mechanism
  • Mostly one URI (/api) and one METHOD (POST)
  • Based on the sent-in data (in $_POST) you get back some data
• POX = Plain Old XML
  • Of course you can use a different format if you like
    • Plaintext is just ... plain
    • JSON is concise and quite readable too
    • JSONP does not allow custom headers and only works with GET
  • → We'll go for JSON

API Call Example

• Do a POST request to /api with $_POST['action'] set to getLecturers and get back a list of lecturers as the actual content

  {
  	"lecturers":
  	[
  		{
  			"id": 1,
  			"name": "Davy De Winne"
  		},
  		{
  			"id": 2,
  			"name": "Bramus Van Damme"
  		}
  	]
  }

1. Resources

Resource Identification & Hierarchy

• Basic naming conventions
  • Verbs are bad, nouns are good
  • Use plurals
  • Respect the Hierarchy
• Examples
  • /product → /products
  • /product/1234 → /products/1234
  • /photos/product/1234 → /products/1234/photos
  • /photos/product/1234/5678 → /products/1234/photos/5678

Identification & Hierarchy bis

• Use concrete naming, yet don't be too specific
  • /things
  • /animals
  • /dogs
  • /beagles
• Some cases will require a verb instead of a noun, yet consider these exceptions
  • /search?course=WMD
  • /convert?from=EUR&to=USD&amount=100

URL Structure Example

• /
• /movies
• /movies/{id}
• /movies/{id}/photos
• /actors
• /actors/{id}
• /actors/{id}/movies
• /about
• /contact

API Call Example

• Do a POST request to /api/lecturers and get back a list of lecturers as the actual content

  {
  	"lecturers":
  	[
  		{
  			"id": 1,
  			"name": "Davy De Winne"
  		},
  		{
  			"id": 2,
  			"name": "Bramus Van Damme"
  		}
  	]
  }

URL Design Sidenotes (1)

• API Versioning
  • Don't place it in the URL (e.g. api.twitter.com/v1/… is wrong)
  • Solution: see further
• Filtering & Sorting
  • Use URL params ?filter=foo&sort=asc
• Limiting fields
  • Use an URL param ?fields=firstname,lastname,age

URL Design Sidenotes (2)

• Pagination
  • Mandatory for huge resources
  • Use limit and offset URL params
    • Defaulting to ?limit=10&offset=<max_id>
    • Avoid using numbers for offset, opt for id-constraint based on sorting method; this avoids doubles when paginating over a frequently updated result set.

2. HTTP Verbs

HTTP Verbs

• HTTP Verbs = CRUD = Queries
  1. POST = Create = INSERT
  2. GET = Read = SELECT
  3. PUT = Update = UPDATE
  4. DELETE = Delete = DELETE
• Examples
  1. POST /products (data via $_POST)
  2. GET /products or GET /products/1234
  3. PUT /products/1234 (data via “$_PUT”)
  4. DELETE /products/1234

HTTP Status Codes (1)

• When accessing a resource, you'll get back the data, along with an HTTP status code
• General Codes (applicable to all verbs/resources)
  • 401 — Not Authorized (e.g. No API Key given)
  • 405 — Method Not Allowed (e.g. wrong verb for resource)
  • 500 — Internal Server Error (Developer screwed up ;))
• GET (SQL SELECT)
  • 200 — OK (requested data sent via response body)
  • 404 — Not Found (resource not found, not “API not found”)
• POST (SQL INSERT)
  • 201 — Created (=OK)
  • 400 — Bad Request (Missing Parameters)

HTTP Status Codes (2)

• PUT (SQL UPDATE)
  • 200 — OK (requested data sent via response body)
  • 400 — Bad Request (Missing Parameters)
  • 404 — Not Found
• DELETE (SQL DELETE)
  • 204 — No Content (=OK — Note: no response body sent!)
  • 404 — Not Found
• Warning!
  • HTTP Statuses 4XX and 5XX will invoke $.ajax().error() in jQuery!

API Call Example (1)

• Do a GET request to /api/lecturers and get back an HTTP 200 status code and a list of lecturers as the content

  {
  	"lecturers":
  	[
  		{
  			"id": 1,
  			"name": "Davy De Winne"
  		},
  		{
  			"id": 2,
  			"name": "Bramus Van Damme"
  		}
  	]
  }

API Call Example (2)

• Do a POST request to /api/lecturers with some data in $_POST to insert a new lecturer
  • Get back 201 — Created if all OK
  • Get back 400 — Bad Request if parameters are missing
• Do a PUT request to /api/lecturers/2 with some data in $_PUT to update a lecturer
  • Get back 200 — OK if all OK
  • Get back 400 — Bad Request if parameters are missing
  • Get back 404 — Not Found if the lecturer does not exist

3. Hypermedia Controls

HATEOAS (1)

• Hypermedia as the Engine of Application State
  • Application State = Where the user is in the process of completing a task (e.g. a page of the application)
  • Hypermedia = The simultaneous presentation of information and controls such that the information becomes the affordance through which the user obtains choices and selects actions ⚑
• Roughly translated
  • The response must be hypertext and include links so that the user/automation can transition to other pages of the application

HATEOAS (2)

• Essentially, an API consumer should only know
  1. The URI starting point
  2. How to parse the response (viz. what media type it is)
• With this, the consumer has enough info to parse out the links and transition to other states when needed
• When an API embraces HATEOAS, we can call it a Hypermedia API

Response Media Types (1)

• Responses must⚑ be hypertext to be RESTful
  • XML and JSON are not hypermedia types; they only define a syntax
  • (X)HTML, ATOM, RDF/XML, etc. are hypermedia types
    • These are all specified
    • Each has their own MIME Type, e.g. application/atom+xml
• JSON based hypermedia types are in the making
  • Collection+JSON
  • Siren
  • HAL
  • JSON-LD
• Collection+JSON looks very promising

Response Media Types (2)

• It's perfectly possible to roll your own hypermedia type
  • e.g. application/vnd.ikdoeict.whatever
  • General consensus amongst hypermedia gurus: don't
• An API can support multiple hypermedia types
  • The client indicates in an accept header which
    • e.g Accept: application/collection+json
  • Content negotiation:
    • e.g. Accept: application/xml; q=0.8, application/json
  • Also use this header for versioning
    • e.g. Accept: application/vnd.ikdoeict.whatever-v2+json
      or Accept: application/collection+json;v=2

What will we do?

• We're going to break the rules and use our own format
  • JSON based
  • Served as plain application/json (Sorry Roy, gotta pave the cowpaths)
  • The returned HTTP status code will also be part of the response
    (but we'll also send it via the response headers as RMM Level 2 requires)
    • The response is split out in a status and content object
  • When links are needed, inject a links array holding the links
    • Links have a rel and href property
    • rel values are based on RFC5988
      • self, up, index, next, prev, ...
      • When no value can be found in the RFC, pick your own (e.g. photos)

API Call Example

• Do a GET request to /lecturers and get back a custom JSON response with HTTP status code 200 containing a list of lecturers (with resource links) as the response content

  {
  	"status":
  	{
  	   "code": 200,
  	   "text": "OK"
  	},
  	"content":
  	{
  		"size": 2,
  		"lecturers":
  		[
  			{
  				"id": 1,
  				"name": "Davy De Winne",
  				"links": [
  					{
  						"rel": "self",
  						"href": "/users/1"
  					}
  				]
  			},
  			{
  				"id": 2,
  				"name": "Bramus Van Damme",
  				"links": [
  					{
  						"rel": "self",
  						"href": "/users/2"
  					}
  				]
  			}
  		]
  	}
  }

Summary

• Real RESTful APIs or Hypermedia APIs are APIs that consist of two things
  1. Usage of HTTP to its fullest (RMM Levels 0 - 2)
  2. Responses are served as hypermedia that manages application state (RMM Level 3)
• We'll be RMM level 3 compliant-ish and use our own (simple) hypermedia format but serve it as application/json — It cuts the mustard

RESTful API Case Study

Twitter API (1)

• One of the first RESTful APIs, suffering from historically grown bloat.
  • Spoiler: it's not RESTful at all!
• Get a status
  • Endpoint
    • GET api.twitter.com/1/statuses/show/id.format
  • Problems
    • API Version in URL
    • Operation show included in URL
    • id not a child of statuses
  • Better
    • GET api.twitter.com/statuses/{id}

Twitter API (2)

• Set (update) your status
  • Endpoint
    • POST api.twitter.com/1/statuses/update/id.format
  • Problems
    • API Version in URL
    • Operation update included in URL
    • Uses authenticated user implicitly
  • Better
    • POST api.twitter.com/users/{id}/statuses

Twitter API (3)

• Remove a status udpate
  • Endpoint
    • POST api.twitter.com/1/statuses/destroy/id.format
  • Problems
    • API Version in URL
    • Operation destroy included in URL
    • Uses authenticated user implicitly
  • Better
    • DELETE api.twitter.com/users/{id}/statuses/{id}

Twitter API (4)

• Get retweets of a status
  • Endpoint
    • GET api.twitter.com/1/statuses/retweets/id.format
  • Problems
    • API Version in URL
    • Hierarchy is wrong
  • Better
    • GET api.twitter.com/statuses/{id}/retweets

MoviesDB (1)

MoviesDB (2)

• Example API
• Play around with the API Explorer to check it out
  • FYI: You can Fork the API Explorer
• Recommened tool: Chrome Postman plugin

RESTful APIs in PHP

Prerequisite: Supporting nice URLs

• Route all requests to non-existing files/folders to index.php
• Build index.php in such a way that it decides what to do based on the URL (found in $_SERVER['REQUEST_URI'])

Nice URLs in Apache

• Possible using the mod_rewrite Apache Module
  • Wampserver: not enabled by default (instructions)
  • MAMP Pro: enabled by default
• Put a file named .htaccess in your root to redirect requests to non-existing files to index.php

  <IfModule mod_rewrite.c>
  Options -MultiViews
  RewriteEngine On
  # RewriteBase /
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule ˆ /index.php [L]
  </IfModule>

Nice URLs in IIS

• Possible using the IIS Rewrite Module
• Put a file named web.config in your root to redirect requests to non-existing files to index.php

  <?xml version="1.0" encoding="UTF-8"?>
  <configuration>
  	<system.webServer>
  		<rewrite>
  			<rules>
  				<rule name="Main Rule" stopProcessing="true">
  					<match url=".*" />
  					<conditions logicalGrouping="MatchAll">
  						<add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
  						<add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
  					</conditions>
  					<action type="Rewrite" url="index.php" />
  				</rule>
  			</rules>
  		</rewrite>
  	</system.webServer>
  </configuration>

Nice URLs in the PHP 5.4 server

• All requests route to index.php by default
  • You'll need to add some extra PHP code to index.php to force non-404 requests route to the correct file

  <?php
  
  $filename = __DIR__ . preg_replace('#(\?.*)$#', '', $_SERVER['REQUEST_URI']);
  if (php_sapi_name() === 'cli-server' && is_file($filename)) {
  	return false;
  }
  
  // rest of your code here

It's all about the routes

• What we essentially need is a router of some sorts that takes the URL and the HTTP method into account
  • If you do a GET request to /movies, show a list of movies
  • If you do a GET request to /movies/{id}, show the one specific movie
  • If you do a POST request to /movies, insert it
  • …
• Codewise, it should be something like this

  $router = new Router();
  
  $router->get('/', function() { … });
  $router->get('/movies', function() { … });
  $router->get('/movies/\d+', function() { … });
  $router->post('/movies', function() { … });
  
  $router->run();

Router Demo

Let's take a look at assets/07/examples/router/ →

Router Recap (1)

• Definitions
  • A route is a combination of a pattern and an HTTP method
  • A route is handled by a handling function which returns/does x/y
• Hook routes using match()

  $router->match('GET|POST', 'pattern', function() { … });

• Or use a shorthand

  $router->get('pattern', function() { … });

Router Recap (2)

• Route patterns can be static or dynamic
  • The dynamic parts of the pattern (\d+, \w+, .*, or .+) will be converted to route variables which you pass into the handling function

  $router->get('/movies/\d+', function($movieId) {
  	echo 'Movie #' . $movieId . ' detail';
  });

  $router->get('/movies/\d+/photos/\d+', function($movieId, $photoId) {
  	echo 'Movie #' . $movieId . ' photo #' . $photoId);
  });

• Only the first route matched will be handled
• When no matching route is found, a 404 will be returned
  • Override the default 404 handler using $router->set404(function);

Router Recap (3)

• Connect middleware before the routes are handled

  $router->before('GET|POST', 'pattern', function() {
  	// ...
  });

• Unlike regular routes it's possible to hook several middlewares to one and the same route

  $router->before('GET|POST', '/admin/.*', function() {
  	if (!isset($_SESSION['userId'])) { … }
  });

• Possible to run one middleware after a route was handled
  • Pass it along the run function

  $router->run(function() { … });

It's all about the responses (1)

• To easily work with the hypermedia format we've described, let's introduce a Response class
• Class requirements
  • Provide $status and $content datamembers + mutators
  • Keep list of allowed HTTP Status Codes
  • By default, set status to 200

It's all about the responses (2)

<?php
namespace Ikdoeict\Rest;

class Response {

	private $status, $content;

	public function __construct() {
		$this->setStatus(200);
		$this->setContent('');
	}

	final public function setStatus($statusCode) {
		$codes = array(
			200 => 'OK',
			// ... List of statuscode with status texts
		);
		$this->status = array(
			'code' => $statusCode,
			'text' => $codes[$statusCode]
		);
		header('HTTP/1.1 ' . $this->status['code'] . ' ' . $this->status['text']);
	}

	final public function setContent($content) {
		$this->content = $content;
	}

	public function finish() {

		$json = json_encode(
			array(
				'status' => $this->status,
				'content' => $this->content
			)
		);

		header('Cache-Control: no-cache, must-revalidate');
		header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');
		header('Content-type: application/json');
		echo $json;

		exit();
	}

}
// EOF

It's all about the responses (4)

• Supporting JSONP is also fairly easy: pass $_GET['callback'] to the finish function

public function finish($jsonp = null) {

	$json = …

	header('Cache-Control: no-cache, must-revalidate');
	header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');

	if ($jsonp) {
		header('Content-type: application/javascript');
		echo $jsonp . '(' . $json . ');';
	} else {
		header('Content-type: application/json');
		echo $json;
	}

	exit();

}

Response Demo

Let's take a look at assets/07/examples/response/ →

And now, Router & Response together

<?php

$router = new Ikdoeict\Routing\Router();
$response = new Ikdoeict\Rest\Response();

// Override the standard router 404
$router->set404(function() use ($response) {
	$response->setStatus('404');
	$response->setContent('Invalid resource');
	$response->finish();
});

// Define our routes
$router->get('', function() use ($response) {
	$response->setContent('Welcome!');
});

// … (more routes here)

// Run the router
$router->run(function() use ($response) {
	$response->finish(isset($_GET['callback']) ? $_GET['callback'] : null);
});

// EOF

One more thing

• There's no such thing as $_PUT, you'll need to fake it

<?php

//…

$router->put('/movies/\d+', function() {

	// Fake $_PUT
	$_PUT  = array();
	parse_str(file_get_contents('php://input'), $_PUT);

	// …

});

//…

RESTful APIs in jQuery

RESTful APIs in jQuery

• Since the API returns both OK and errorous HTTP status codes, you need to hook both success and error functions

  $.ajax({
  	url : 'http://api.myapp.tld/',
  	type: 'get|post|put|delete',
  	dataType : 'json',
  	data: 'foo=bar'
  })
  .success(function(data, textStatus, jqXHR) {
  	if (data) {
  		// All OK and data returned (200, 201)
  	} else {
  		// All OK but no data returned (e.g. 204 after a DELETE)
  	}
  })
  .error(function(jqXHR, textStatus, errorThrown) {
  	var data = jqXHR.responseText ? JSON.parse(jqXHR.responseText) : {};
  	// Got a 400 (params), 401, 404, 405 (method), 500, or 501 (not impl)
  });

API Authentication

API Authentication

• Two needs for authentication
  1. API Consumers (apps using your API)
  2. User Authentication (allowing apps using your API to act on behalf of a user)

1. API Consumers (1)

• Make use of API-Keys
  • Require API-Key to be sent in via a header
  • Check API-Key as a before middleware on all routes

  $router->before('GET|POST|PUT|DELETE', '.*', function() use ($response) {
  	$headers = apache_request_headers();
  	if(!isset($headers['X-Api-Key']) || !ApiDB::isValidApiKey($headers['X-Api-Key'])) {
  		$response->setStatus(401);
  		$response->setContent('Missing or invalid API Key.');
  		$response->finish();
  	}
  });

• Note: Custom headers must start with X and all words must be ucfirst'd and separated with -

1. API Consumers (2)

• Note: apache_request_headers() not always available (IIS)!
  • Paste in the code below to always have it available

  if (!function_exists('apache_request_headers')) {
  	eval('
  		function apache_request_headers() {
  			foreach($_SERVER as $key=>$value) {
  				if (substr($key,0,5)=="HTTP_") {
  					$key=str_replace(" ","-",ucwords(strtolower(str_replace("_"," ",substr($key,5)))));
  					$out[$key]=$value;
  				}
  			}
  			return $out;
  		}
  	');
  }

2. User Authentication (1)

• API Consumers should never ask the user his/her username/password (password anti-pattern!) but should use OAuth2 which works with time-limited access tokens to accessing user data

  

2. User Authentication (2)

• OAuth2, schematic

  

2. User Authentication (3)

• OAuth2 in words
  1. User is sent to the OAuth provider (along the consumer app's API-Key (client_id) for identification), asking if consumer app may access user data at a certain scope (read/write)
  2. If user accepts the OAuth provider will redirect back to the consumer app's redirect_uri with an authorization_code
  3. In the background, the consumer app must make a request to the OAuth provider with that authorization_code, along with its API-Key (client_id) and secret key (client_secret)
  4. The OAuth provider will issue an access_token and refresh_token
  5. The access_token must passed with all calls made and is valid for a limited time
  6. When the access_token has expired, the consumer app needs to do a call to the OAuth Prodider with the refresh_token to get a new pair access_token & refresh_token

API Authentication

• API-Keys are easily implemented, both consumer and provider side
• Once you get the gist of it, OAuth2 is fairly easy …
  • Roll your own implementation, or use a prefab solution (#lmgtfy)
  • Remark: Treat OAuth tokens as passwords and encrypt them!

Questions?

Sources

• http://en.wikipedia.org/wiki/Representational_state_transfer
• http://www.slideshare.net/jmusser/pw-glue-conmay2010
• http://www.slideshare.net/al3x/the-interaction-design-of-apis
• http://www.slideshare.net/Wombert/phpjp-urls-rest
• http://www.slideshare.net/apigee/restful-api-design-second-edition
• http://restpatterns.org/HTTP_Status_Codes
• http://httpstatus.es/
• http://restcookbook.com/
• http://barelyenough.org/blog/2008/05/versioning-rest-web-services/
• http://en.wikipedia.org/wiki/HATEOAS
• http://martinfowler.com/articles/richardsonMaturityModel.html
• http://www.smartjava.org/content/presentation-rest-hateoas
• http://www.infoq.com/articles/mark-baker-hypermedia
• http://www.recessframework.org/page/towards-restful-php-5-basic-tips
• http://www.gen-x-design.com/archives/create-a-rest-api-with-php/
• http://en.wikipedia.org/wiki/Oauth
• http://www.slideshare.net/leahculver/oauth-open-api-authentication