Saturday, May 12, 2018

Scala Syntax: 7 points

A few years back I dipped into some Scala as a hobby language. Recently, in order to get a quick overview of Spark I did the 'Big Data Analysis with Scala and Spark' from Coursera. It's a great course. But, one aspect I found challenging was just getting my head around Scala syntax again. Some of it, yeah the basic stuff can be counter-intuitive depending on your perspective.

1. Method / Function Definition

Typing on the right rather than the left. Consider this simple function definition:
def sayHello(param: String): String = {
    "Hello" + param
}
Javaholics will note:
  • The return is specified at the end of the method definition, rather than the beginning. 
  • The type of the parameter is specified after the parameter name rather than before. 
  • Before the function body there is a = 
  • There are two colons (:), one between the parameter and the type and one before the return type.

2.  Unit

Google "Unit" and you be quickly told you that Unit is the Scala's version of Java void.  But, Java’s void is a keyword.  Scala’s Unit is a final class which only has one value: () - which is like an alias for no information. Unit indicates a method returns nothing and therefore has side effects, something we don't want to do much of in Scala. So is that counter intuitive? No.
But here is what I find is. If a function has no return type in the function definition and no equals it means Unit is implicitly the return type. Example:
def procedure {
    println "This String is not returned"
}

procedure: ()Unit
Big deal? Of course not. But what about:
def procedure {
     "This String is not returned"
}
Expect the String to be returned, it wont be. How about this?
def addNumbers(a: Integer, b: Integer) {
    return a + b
}
This will give a compile warning: :12: warning: enclosing method addNumbers has result type Unit: return value discarded return a + b It will compile but nothing will be returned:
def addNumbers(a: Integer, b: Integer) {
    a + b
}
will give no compile warning and will also return nothing.

3.  Underscore

In anonymous Scala functions, _ is like Groovy's it. In Groovy we can to multiple all numbers between 1 and 5 we can do:
(1..5).collect {it * 2}
In Scala we can do:
(1 to 5).map{_*2}
However, in Scala, the second time _ is referenced, it refers to the second parameter
val ns = List(1, 2, 3, 4)
val s0 = ns.foldLeft (0) (_+_) //10

4. Passing anonymous functions. 

Pass one anonymous function and you don't need any curly parenthesis. Pass two and you do.
def compose(g:R=>R, h:R=>R) = (x:R) => g(h(x)) 
val f = compose({_*2}, {_-1})

5. Arity-0 

When a method has no arguments, (arity-0), the parentheses can be omitted in invocation
size()
...
size  // do it like this 
But this technique should never be used when method has side effects. So,
queue.size // ok
println // not ok do println()

6. Declare parameter types

Function defiinitions / Method definition have to declare parameter types but function literals don’t.
def addNumbers(a, b): Number {
:1: error: ':' expected but ',' found.

7. Ternary Operator

There is no ternary operator in Scala. There is one in Java, Groovy, JavaScript. Python 2.5 added support for it. Instead you can do if else on one line and since if / else is an expression you can return a value. For example: In Java we would do:
(eurovision.winner == "Ireland") ? "Yippee" : "It's a fix"
Scala, it's:
if (eurovision.winner == "Ireland") "Yippee" else "It's a fix"

Friday, May 11, 2018

And some more REST tips

In previous blog posts I have covered some ideas and tips for achieving a REST architecture. In this post, I cover a few more ideas and tips.

Caching

  • Caching is a big part of the original dissertation.  See section 5.1.4 
  • Strategies include validation (client checks it has the latest version) and expiration (client assumes it has the latest version until a specified time)
  • Expiration:
    • Expires header tells client when resource is going to expire. The value 0 means avoid caching
    • Cache-Control
      • Use max-age directive to specify how long response should be considered valid for; s-maxage for shared caches
      • Can also be used in requests no-cache means re validate response with server
  • Validation
    • Etag - unique version of resource. Used in conjunction with If-none-match request header
    • Last-Modified - tells client when resource last changed

Controller APIs

  • When something does fit neatly to a CRUD operation, consider a Controller API

Handling Dates

  • Use ISO-8601 for your dates - better for natural sorting, handles timezone, locale nuetral, support from most programming languages
  • Accept any timezone as anyone in the world may call your API
  • Store in UTC, not in your server's timezone.  There should be no offset when persisted.
  • Return in UTC.  Allow the client to adjust to its timezone as necessary
  • Don't use time if you don't need it.  If Date only suffices, only persist Date. This means, timezone complexity goes away. 

HEAD

Headers

  • Always return what headers are useful.  Consider: 
    • Content-Type
    • Content-Length
    • Last-Modified
    • ETag
    • Location

Hypermedia (advantages)

  • Less coupling
  • Consistent format for links => cleaner client code
  • Developer productivity: API's easier to navigate 
  • Make easier to introduce services in a more granular way
  • Code easier to debug - messages always have the URL that created them via the self link

Hypermedia (choices)

  • HAL - reduces Address coupling 
  • SIREN - reduces Address and Actions coupling
  • Collection+JSON (CJ) - reduces Address, Action and Object coupling

Idempotent

  • Can be called several times and return the same result
  • OPTIONS, GET, HEAD, PUT and DELETE are all idempotent

Long Running Requests

  • Some operations take a long time.  In such cases, consider returning a 202 with the location field set to a URL the client can poll to check for operation progress.

Method not allowed

  • If an API only supports GET, it should return a 405 for any PUT, POST, DELETEs etc

Must Ignore Principle

  • Clients should ignore data they are not interested in. This makes it much easier for APIs to be backwardly compatible .  If an API returns extra data and some clients aren't expecting it they will just ignore it. 

Not acceptable

  • When a resource doesn't support a specific media type, it should return 406  (see Masse, Rule: 406 (“Not Acceptable”) must be used when the requested media type cannot be served

OPTIONS

  • OPTIONS should return what actions are available on a resource

Partial Update

  • Handle partial updates with  PATCH

Query

  • The query component of a URI should be used to filter collections

Resource Creation

  • When a Resource has been successfully created a 201 should be returned 
  • The location header should indicate the URL to get the Resource. 

Safe

  • Actions are considered Safe if they Do not modify resources
  • OPTIONS, GET and HEAD are safe

Self link

  • Response bodies should always include a self link - the URL that was used to return the resource. 

Singular or Plural?

  • Use Singular for Singular Document type resource  - when there can only be one.  For example: /humans/12343343/head
  • Otherwise plural

REST: Using a Controller endpoint?

In REST architectures, the fundamental concept is a Resource.   A Resource represents anything that’s important enough to be referenced as a thing in itself.   For example, a Shopping Cart, a Book or a Car.  The next fundamental concept is the Uniform Interface for accessing and manipulating the Resources.  In HTTP land usually means:
  • Create is POST 
  • Read is GET 
  • Update is PUT (or PATCH for Partial Update) 
  • Delete is DELETE
There are of course other concepts (statelessness, caching etc) but for this blog post, let's just focus on Resources.

In the real world,  many things map nicely to Resources.  However, inevitably somethings won't map so nicely to resources. This is usually a minority of operations for example reset password. It's possible to model these as either
  •  a PUT on /password/ 
or as
  •  a Controller endpoint and a POST to /resetpassword 
The latter may be considered to be closer to programmatic REST than pure REST, but there are times when clients and customers will want you to be pragmatic. This article gives suggestions regarding when to consider using the Controller option.

Does the action Map to a CRUD? 

Several actions in a real world application will not map nicely to a Create Read Update Delete (CRUD). For example, Paypal's cancel billing agreement API is:
POST /v1/payments/billing-agreements/agreement_id/cancel
The cancel action rarely maps nicely to a CRUD for a resource. It could be interpreted as:
  • some resource gets be created (A cancel record) 
  • some resource gets updated (some status column could be getting set to cancelled) 
  • or some resource gets deleted (a order request gets deleted). 
Why should the client have to care about how cancel is handled?  Couldn't it always change? In some case API's have got around the doesn't map nicely to a CRUD problem using HTTP tunneling. For cancelling a billing agreement this would like:
POST /v1/payments/billing-agreements/agreement_id
with body:
{
  "operation":"cancel"
}
This is considered an anti-pattern and should never be used. Instead a Controller end point should be used.

 

Resource State or Workflow? 

In a REST architecture, every request between Client or Server will usually change a Resource State (write operation) or the Application State (a query or read operation). However, in the real world workflows are inevitable. For example, a reset password flow usually consists of:
  • Asking the user for the userId (usually email) 
  • System checking that email exists on the system 
  • Sending the user an email with a link to reset the password 
  • Ensuring the user only has a set amount of time to click the link 
  • When the user clicks the link they may be asked a bunch of questions 
  • They will be asked to retype their new password to ensure there's no typos 
When an client action is part of a complex workflow, Resource state and Application state changes may not be easy to model. They may not happen synchronously and they could change based on how the workflow is modelled or when the workflow needs to add an extra step. In such scenarios, consider using a Controller end point.

 

REST without PUT 

In some situations, arguments can be made for avoiding PUT and instead using POST to a different endpoint which signifies intent. For example, to change address instead of invoking a PUT to /address/, the client would invoke a POST to /changeaddress and avoid PUTs altogether.  One example where this approach is useful is when handling asynchronous operations and you are trying to make clear atomic consistent operation.  So for example, if changing address takes a long time and you would rather return a 202, with a location field for the client to poll, if you use the /changeaddress you can then leave /address endpoints as those that are only atomically consistent.

So, any PUT or POST to address, means if you were to immediately do a GET you would get the consistent view of the Resource.  This approach is also useful if you want to model the Business event rather than the actual resource that is changing.  So for example, suppose 6 or 7 things need to take place when a Bank account has been closed for a Business process perspective.  All on the back end in the same thread / transaction.  Again, here POST to controller endpoint such as /accountclosed makes more sense then /DELETE to /account.

See this article for more info. 

Summary


So why there may be subjectivity involved on when to use a controller style endpoint.  The above may at least help to you to make a decision.  Remember, it should always only be a minority of APIs where you consider this approach. You are outside the conventional Uniform Interface for unique style operations but you want to still make them feel intuitive to clients of the API.