Web & Mobile Development JLW288

07.RESTful.APIs

RESTful APIs

The key to victory is discipline,
and that means a well made bed.

RESTful API?

  • An API that is REST compliant

SRSLY?

Not Sure If ...

Not sure if trolling or just “technically correct”

API

  • Application Programming Interface
    • Abstract Interface to allow communication between software components
    • eg. Photoshop has a JS based API to remotely control Photoshop
    • eg. Browsers have an API that allow creation of extensions
    • eg. Browsers have a Device API that give JS devvers access to device features such as geolocation, camera, vibration, etc.

Web API

  • Websites can also offer an API
    • Allow you to push/pull data to/from a web site
    • Scripting language independent (PHP, ASP, RoR, ..)
    • Sometimes referred to as a “Web Service” (blame Microsoft)
    • Implemented via JavaScript, XML-RPC, SOAP, REST, …
    • Return formats JS, XML, HTML, plaintext, JSON(P), …
You've already implemented a SOAP-based one in the course Webprogrammatie

Why provide an API? (1)

  • Technical Limitations
    • You cannot connect to a MySQL server from an iOS app you made
    • You cannot connect to a MySQL server from within JavaScript
  • Security Issues
    • It is not smart to connect to a MySQL server from an iOS app you made

Why provide an API? (2)

  • See cool stuff built on top of your technology platform
    Let your plants tell you via Twitter that they're thirsty with botanicalls
    Know when your baby kicked via Twitter with http://kickbee.net/

Why provide an API? (3)

  • See cool stuff built using your data
    Visualize your Gowalla (RIP) checkins with GowallaHEAT
    Write your own Twitter Client (and eventually get bought by Twitter)

REST

  • REST = REpresentational State Transfer
    • Architectural style for distributed hypermedia systems described by Roy Fielding, in 2000
  • REST constraints
    1. Client-server
    2. Stateless Server
    3. Cacheable
    4. Uniform interface
      1. Identification of resources (URI)
      2. Manipulation of resources through representations (modify/delete)
      3. Self-descriptive messages (message and meta-data)
      4. Hypermedia As The Engine Of Application State (state transitions)
    5. Layered System
    6. Code-On-Demand (optional)

REST API

  • API that follows the REST constraints
  • Constraint #4 (Uniform interface) is the tricky one

Why choose REST?

  • As a consumer, it's easy to work with
  • As a developer, it's (fairly) easy to implement
  • It's well structured
  • It's popular
    Figure by jmusser

Richardson Maturity Model

This chronological wang-dang-doodle
could destroy the very matrix of reality.

Richardson Maturity Model

  • Model that breaks down the principal elements of a REST approach in a few steps
  • The model embraces 4 levels
    1. Level 0: The Swamp of POX
    2. Level 1: Identify and organize resources in a hierarchy
    3. Level 2: Use HTTP verbs for operations
    4. Level 3: HATEOAS
Most self-proclaimed “RESTful APIs” out there aren't even level 1 compliant

0. The Swamp of POX

Richardson Maturity Model Level 0

What?

  • Use HTTP as a transport mechanism
    • Mostly one URI (/api) and one METHOD (POST)
    • Based on the sent-in data (in $_POST) you get back some data
  • POX = Plain Old XML
    • Of course you can use a different format if you like
      • Plaintext is just ... plain
      • JSON is concise and quite readable too
      • JSONP does not allow custom headers and only works with GET
    • → We'll go for JSON

API Call Example

  • Do a POST request to /api with $_POST['action'] set to getLecturers and get back a list of lecturers as the actual content
    {
    	"lecturers":
    	[
    		{
    			"id": 1,
    			"name": "Davy De Winne"
    		},
    		{
    			"id": 2,
    			"name": "Bramus Van Damme"
    		}
    	]
    }

1. Resources

Richardson Maturity Model Level 1

Resource Identification & Hierarchy

  • Basic naming conventions
    • Verbs are bad, nouns are good
    • Use plurals
    • Respect the Hierarchy
  • Examples
    • /product/products
    • /product/1234/products/1234
    • /photos/product/1234/products/1234/photos
    • /photos/product/1234/5678/products/1234/photos/5678
Note: This doesn't apply for “normal pages” of your site

Identification & Hierarchy bis

  • Use concrete naming, yet don't be too specific
    • /things
    • /animals
    • /dogs
    • /beagles
  • Some cases will require a verb instead of a noun, yet consider these exceptions
    • /search?course=WMD
    • /convert?from=EUR&to=USD&amount=100

URL Structure Example

  • /
  • /movies
  • /movies/{id}
  • /movies/{id}/photos
  • /actors
  • /actors/{id}
  • /actors/{id}/movies
  • /about
  • /contact

API Call Example

  • Do a POST request to /api/lecturers and get back a list of lecturers as the actual content
    {
    	"lecturers":
    	[
    		{
    			"id": 1,
    			"name": "Davy De Winne"
    		},
    		{
    			"id": 2,
    			"name": "Bramus Van Damme"
    		}
    	]
    }

URL Design Sidenotes (1)

  • API Versioning
    • Don't place it in the URL (e.g. api.twitter.com/v1/… is wrong)
    • Solution: see further
  • Filtering & Sorting
    • Use URL params ?filter=foo&sort=asc
  • Limiting fields
    • Use an URL param ?fields=firstname,lastname,age

URL Design Sidenotes (2)

  • Pagination
    • Mandatory for huge resources
    • Use limit and offset URL params
      • Defaulting to ?limit=10&offset=<max_id>
      • Avoid using numbers for offset, opt for id-constraint based on sorting method; this avoids doubles when paginating over a frequently updated result set.

2. HTTP Verbs

Richardson Maturity Model Level 2

HTTP Verbs

  • HTTP Verbs = CRUD = Queries
    1. POST = Create = INSERT
    2. GET = Read = SELECT
    3. PUT = Update = UPDATE
    4. DELETE = Delete = DELETE
  • Examples
    1. POST /products (data via $_POST)
    2. GET /products or GET /products/1234
    3. PUT /products/1234 (data via “$_PUT”)
    4. DELETE /products/1234

HTTP Status Codes (1)

  • When accessing a resource, you'll get back the data, along with an HTTP status code
  • General Codes (applicable to all verbs/resources)
    • 401 — Not Authorized (e.g. No API Key given)
    • 405 — Method Not Allowed (e.g. wrong verb for resource)
    • 500 — Internal Server Error (Developer screwed up ;))
  • GET (SQL SELECT)
    • 200 — OK (requested data sent via response body)
    • 404 — Not Found (resource not found, not “API not found”)
  • POST (SQL INSERT)
    • 201 — Created (=OK)
    • 400 — Bad Request (Missing Parameters)

HTTP Status Codes (2)

  • PUT (SQL UPDATE)
    • 200 — OK (requested data sent via response body)
    • 400 — Bad Request (Missing Parameters)
    • 404 — Not Found
  • DELETE (SQL DELETE)
    • 204 — No Content (=OK — Note: no response body sent!)
    • 404 — Not Found
  • Warning!
    • HTTP Statuses 4XX and 5XX will invoke $.ajax().error() in jQuery!

API Call Example (1)

  • Do a GET request to /api/lecturers and get back an HTTP 200 status code and a list of lecturers as the content
    {
    	"lecturers":
    	[
    		{
    			"id": 1,
    			"name": "Davy De Winne"
    		},
    		{
    			"id": 2,
    			"name": "Bramus Van Damme"
    		}
    	]
    }

API Call Example (2)

  • Do a POST request to /api/lecturers with some data in $_POST to insert a new lecturer
    • Get back 201 — Created if all OK
    • Get back 400 — Bad Request if parameters are missing
  • Do a PUT request to /api/lecturers/2 with some data in $_PUT to update a lecturer
    • Get back 200 — OK if all OK
    • Get back 400 — Bad Request if parameters are missing
    • Get back 404 — Not Found if the lecturer does not exist

3. Hypermedia Controls

Richardson Maturity Model Level 3

HATEOAS (1)

  • Hypermedia as the Engine of Application State
    • Application State = Where the user is in the process of completing a task (e.g. a page of the application)
    • Hypermedia = The simultaneous presentation of information and controls such that the information becomes the affordance through which the user obtains choices and selects actions
  • Roughly translated
    • The response must be hypertext and include links so that the user/automation can transition to other pages of the application

HATEOAS (2)

  • Essentially, an API consumer should only know
    1. The URI starting point
    2. How to parse the response (viz. what media type it is)
  • With this, the consumer has enough info to parse out the links and transition to other states when needed
  • When an API embraces HATEOAS, we can call it a Hypermedia API

Response Media Types (1)

  • Responses must be hypertext to be RESTful
    • XML and JSON are not hypermedia types; they only define a syntax
    • (X)HTML, ATOM, RDF/XML, etc. are hypermedia types
      • These are all specified
      • Each has their own MIME Type, e.g. application/atom+xml
  • JSON based hypermedia types are in the making
  • Collection+JSON looks very promising

Response Media Types (2)

  • It's perfectly possible to roll your own hypermedia type
    • e.g. application/vnd.ikdoeict.whatever
    • General consensus amongst hypermedia gurus: don't
  • An API can support multiple hypermedia types
    • The client indicates in an accept header which
      • e.g Accept: application/collection+json
    • Content negotiation:
      • e.g. Accept: application/xml; q=0.8, application/json
    • Also use this header for versioning
      • e.g. Accept: application/vnd.ikdoeict.whatever-v2+json
        or Accept: application/collection+json;v=2

What will we do?

  • We're going to break the rules and use our own format
    • JSON based
    • Served as plain application/json (Sorry Roy, gotta pave the cowpaths)
    • The returned HTTP status code will also be part of the response
      (but we'll also send it via the response headers as RMM Level 2 requires)
      • The response is split out in a status and content object
    • When links are needed, inject a links array holding the links
      • Links have a rel and href property
      • rel values are based on RFC5988
        • self, up, index, next, prev, ...
        • When no value can be found in the RFC, pick your own (e.g. photos)

API Call Example

  • Do a GET request to /lecturers and get back a custom JSON response with HTTP status code 200 containing a list of lecturers (with resource links) as the response content
    {
    	"status":
    	{
    	   "code": 200,
    	   "text": "OK"
    	},
    	"content":
    	{
    		"size": 2,
    		"lecturers":
    		[
    			{
    				"id": 1,
    				"name": "Davy De Winne",
    				"links": [
    					{
    						"rel": "self",
    						"href": "/users/1"
    					}
    				]
    			},
    			{
    				"id": 2,
    				"name": "Bramus Van Damme",
    				"links": [
    					{
    						"rel": "self",
    						"href": "/users/2"
    					}
    				]
    			}
    		]
    	}
    }
Note: we've broken the rules here ... this ain't a genuine Hypermedia API!

Summary

  • Real RESTful APIs or Hypermedia APIs are APIs that consist of two things
    1. Usage of HTTP to its fullest (RMM Levels 0 - 2)
    2. Responses are served as hypermedia that manages application state (RMM Level 3)
  • We'll be RMM level 3 compliant-ish and use our own (simple) hypermedia format but serve it as application/json — It cuts the mustard

RESTful API Case Study

Hey, look! I found a robot fossil!

Twitter API (1)

  • One of the first RESTful APIs, suffering from historically grown bloat.
    • Spoiler: it's not RESTful at all!
  • Get a status
    • Endpoint
      • GET api.twitter.com/1/statuses/show/id.format
    • Problems
      • API Version in URL
      • Operation show included in URL
      • id not a child of statuses
    • Better
      • GET api.twitter.com/statuses/{id}

Twitter API (2)

  • Set (update) your status
    • Endpoint
      • POST api.twitter.com/1/statuses/update/id.format
    • Problems
      • API Version in URL
      • Operation update included in URL
      • Uses authenticated user implicitly
    • Better
      • POST api.twitter.com/users/{id}/statuses

Twitter API (3)

  • Remove a status udpate
    • Endpoint
      • POST api.twitter.com/1/statuses/destroy/id.format
    • Problems
      • API Version in URL
      • Operation destroy included in URL
      • Uses authenticated user implicitly
    • Better
      • DELETE api.twitter.com/users/{id}/statuses/{id}

Twitter API (4)

  • Get retweets of a status
    • Endpoint
      • GET api.twitter.com/1/statuses/retweets/id.format
    • Problems
      • API Version in URL
      • Hierarchy is wrong
    • Better
      • GET api.twitter.com/statuses/{id}/retweets

MoviesDB (1)

(click image to launch)

MoviesDB (2)

RESTful APIs in PHP

Would it cheer you up if I punch Fry in the groin?
Cause I'll do it, regardless.

Prerequisite: Supporting nice URLs

  • Route all requests to non-existing files/folders to index.php
  • Build index.php in such a way that it decides what to do based on the URL (found in $_SERVER['REQUEST_URI'])

Nice URLs in Apache

  • Possible using the mod_rewrite Apache Module
    • Wampserver: not enabled by default (instructions)
    • MAMP Pro: enabled by default
  • Put a file named .htaccess in your root to redirect requests to non-existing files to index.php
    <IfModule mod_rewrite.c>
    Options -MultiViews
    RewriteEngine On
    # RewriteBase /
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ˆ /index.php [L]
    </IfModule>

Nice URLs in IIS

  • Possible using the IIS Rewrite Module
  • Put a file named web.config in your root to redirect requests to non-existing files to index.php
    <?xml version="1.0" encoding="UTF-8"?>
    <configuration>
    	<system.webServer>
    		<rewrite>
    			<rules>
    				<rule name="Main Rule" stopProcessing="true">
    					<match url=".*" />
    					<conditions logicalGrouping="MatchAll">
    						<add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
    						<add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
    					</conditions>
    					<action type="Rewrite" url="index.php" />
    				</rule>
    			</rules>
    		</rewrite>
    	</system.webServer>
    </configuration>

Nice URLs in the PHP 5.4 server

  • All requests route to index.php by default
    • You'll need to add some extra PHP code to index.php to force non-404 requests route to the correct file
    <?php
    
    $filename = __DIR__ . preg_replace('#(\?.*)$#', '', $_SERVER['REQUEST_URI']);
    if (php_sapi_name() === 'cli-server' && is_file($filename)) {
    	return false;
    }
    
    // rest of your code here
Note: You can start a server by running php -S localhost:8080 in your project www-root

It's all about the routes

  • What we essentially need is a router of some sorts that takes the URL and the HTTP method into account
    • If you do a GET request to /movies, show a list of movies
    • If you do a GET request to /movies/{id}, show the one specific movie
    • If you do a POST request to /movies, insert it
  • Codewise, it should be something like this
    $router = new Router();
    
    $router->get('/', function() { … });
    $router->get('/movies', function() { … });
    $router->get('/movies/\d+', function() { … });
    $router->post('/movies', function() { … });
    
    $router->run();

Router Demo

Let's take a look at assets/07/examples/router/

Router Recap (1)

  • Definitions
    • A route is a combination of a pattern and an HTTP method
    • A route is handled by a handling function which returns/does x/y
  • Hook routes using match()
    $router->match('GET|POST', 'pattern', function() { … });
  • Or use a shorthand
    $router->get('pattern', function() { … });

Router Recap (2)

  • Route patterns can be static or dynamic
    • The dynamic parts of the pattern (\d+, \w+, .*, or .+) will be converted to route variables which you pass into the handling function
    $router->get('/movies/\d+', function($movieId) {
    	echo 'Movie #' . $movieId . ' detail';
    });
    $router->get('/movies/\d+/photos/\d+', function($movieId, $photoId) {
    	echo 'Movie #' . $movieId . ' photo #' . $photoId);
    });
  • Only the first route matched will be handled
  • When no matching route is found, a 404 will be returned
    • Override the default 404 handler using $router->set404(function);

Router Recap (3)

  • Connect middleware before the routes are handled
    $router->before('GET|POST', 'pattern', function() {
    	// ...
    });
  • Unlike regular routes it's possible to hook several middlewares to one and the same route
    $router->before('GET|POST', '/admin/.*', function() {
    	if (!isset($_SESSION['userId'])) { … }
    });
  • Possible to run one middleware after a route was handled
    • Pass it along the run function
    $router->run(function() { … });

It's all about the responses (1)

  • To easily work with the hypermedia format we've described, let's introduce a Response class
  • Class requirements
    • Provide $status and $content datamembers + mutators
    • Keep list of allowed HTTP Status Codes
    • By default, set status to 200

It's all about the responses (2)

<?php
namespace Ikdoeict\Rest;

class Response {

	private $status, $content;

	public function __construct() {
		$this->setStatus(200);
		$this->setContent('');
	}

	final public function setStatus($statusCode) {
		$codes = array(
			200 => 'OK',
			// ... List of statuscode with status texts
		);
		$this->status = array(
			'code' => $statusCode,
			'text' => $codes[$statusCode]
		);
		header('HTTP/1.1 ' . $this->status['code'] . ' ' . $this->status['text']);
	}

	final public function setContent($content) {
		$this->content = $content;
	}

	public function finish() {

		$json = json_encode(
			array(
				'status' => $this->status,
				'content' => $this->content
			)
		);

		header('Cache-Control: no-cache, must-revalidate');
		header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');
		header('Content-type: application/json');
		echo $json;

		exit();
	}

}
// EOF
Call $response->finish(); to let it show your response in the format we agreed upon

It's all about the responses (4)

  • Supporting JSONP is also fairly easy: pass $_GET['callback'] to the finish function
public function finish($jsonp = null) {

	$json = …

	header('Cache-Control: no-cache, must-revalidate');
	header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');

	if ($jsonp) {
		header('Content-type: application/javascript');
		echo $jsonp . '(' . $json . ');';
	} else {
		header('Content-type: application/json');
		echo $json;
	}

	exit();

}

Response Demo

Let's take a look at assets/07/examples/response/

And now, Router & Response together

<?php

$router = new Ikdoeict\Routing\Router();
$response = new Ikdoeict\Rest\Response();

// Override the standard router 404
$router->set404(function() use ($response) {
	$response->setStatus('404');
	$response->setContent('Invalid resource');
	$response->finish();
});

// Define our routes
$router->get('', function() use ($response) {
	$response->setContent('Welcome!');
});

// … (more routes here)

// Run the router
$router->run(function() use ($response) {
	$response->finish(isset($_GET['callback']) ? $_GET['callback'] : null);
});

// EOF

One more thing

  • There's no such thing as $_PUT, you'll need to fake it
<?php

//…

$router->put('/movies/\d+', function() {

	// Fake $_PUT
	$_PUT  = array();
	parse_str(file_get_contents('php://input'), $_PUT);

	// …

});

//…

RESTful APIs in jQuery

To the flying machine!

RESTful APIs in jQuery

  • Since the API returns both OK and errorous HTTP status codes, you need to hook both success and error functions
    $.ajax({
    	url : 'http://api.myapp.tld/',
    	type: 'get|post|put|delete',
    	dataType : 'json',
    	data: 'foo=bar'
    })
    .success(function(data, textStatus, jqXHR) {
    	if (data) {
    		// All OK and data returned (200, 201)
    	} else {
    		// All OK but no data returned (e.g. 204 after a DELETE)
    	}
    })
    .error(function(jqXHR, textStatus, errorThrown) {
    	var data = jqXHR.responseText ? JSON.parse(jqXHR.responseText) : {};
    	// Got a 400 (params), 401, 404, 405 (method), 500, or 501 (not impl)
    });
TIP: Since we include the status in the response, you can easily check it via data.status.code

API Authentication

Who are those horrible orange creatures over there? — Grunka-Lunkas. — Tell them I hate them!

API Authentication

  • Two needs for authentication
    1. API Consumers (apps using your API)
    2. User Authentication (allowing apps using your API to act on behalf of a user)

1. API Consumers (1)

  • Make use of API-Keys
    • Require API-Key to be sent in via a header
    • Check API-Key as a before middleware on all routes
    $router->before('GET|POST|PUT|DELETE', '.*', function() use ($response) {
    	$headers = apache_request_headers();
    	if(!isset($headers['X-Api-Key']) || !ApiDB::isValidApiKey($headers['X-Api-Key'])) {
    		$response->setStatus(401);
    		$response->setContent('Missing or invalid API Key.');
    		$response->finish();
    	}
    });
  • Note: Custom headers must start with X and all words must be ucfirst'd and separated with -

1. API Consumers (2)

  • Note: apache_request_headers() not always available (IIS)!
    • Paste in the code below to always have it available
    if (!function_exists('apache_request_headers')) {
    	eval('
    		function apache_request_headers() {
    			foreach($_SERVER as $key=>$value) {
    				if (substr($key,0,5)=="HTTP_") {
    					$key=str_replace(" ","-",ucwords(strtolower(str_replace("_"," ",substr($key,5)))));
    					$out[$key]=$value;
    				}
    			}
    			return $out;
    		}
    	');
    }

2. User Authentication (1)

  • API Consumers should never ask the user his/her username/password (password anti-pattern!) but should use OAuth2 which works with time-limited access tokens to accessing user data

2. User Authentication (2)

  • OAuth2, schematic

2. User Authentication (3)

  • OAuth2 in words
    1. User is sent to the OAuth provider (along the consumer app's API-Key (client_id) for identification), asking if consumer app may access user data at a certain scope (read/write)
    2. If user accepts the OAuth provider will redirect back to the consumer app's redirect_uri with an authorization_code
    3. In the background, the consumer app must make a request to the OAuth provider with that authorization_code, along with its API-Key (client_id) and secret key (client_secret)
    4. The OAuth provider will issue an access_token and refresh_token
    5. The access_token must passed with all calls made and is valid for a limited time
    6. When the access_token has expired, the consumer app needs to do a call to the OAuth Prodider with the refresh_token to get a new pair access_token & refresh_token

API Authentication

  • API-Keys are easily implemented, both consumer and provider side
  • Once you get the gist of it, OAuth2 is fairly easy …

Questions?

Sources

ikdoeict.be