Exceptions

Throwing exceptions

TL;DR:

  • treat errors in the categories "illegal input" and "system error"
  • "illegal input" = IllegalArgumentException, System error = "IllegalStateException"
    • Make specific exceptions from these for expectable errors
  • for expectable errors, return Result from your function; for unexpectable/unrecoverable errors fail-fast (propagate upwards)
    • "Is it an exception? Throw an exception. Is it the norm? Handle with Result(Success/Failure)."
  • carry exceptions through; show full stack traces, not just "Error: $message"

Exception vs Result

When do I throw an exception? When do I return Result?

What errors there are / what errors to throw

There can be many reasons why something went wrong while processing a function, but in production, almost all of it can be boiled down to either:

  1. Illegal (/ unexpected/invalid) input, or a
  2. System Error

Let's view this in an example:

You wrote a function that takes a URL as parameter, resolves the URL, parses the JSON response, reads a certain value off of it, does some processing on it, and returns the result. Now, at every step of that, something could go wrong:

  • the user might have provided an invalid URL (expected "https://example.org/abcxyz", but got "ajskldhjal", "", etc...)
  • the URL is not resolvable/reachable ("https://example.org" is down)
  • the URL is resolvable, but the server responds with an error (503 Service Unavailable)
  • the URL is reachable, but the server provides invalid JSON (possibly a text message instead of a JSON response)
  • the server provides valid JSON, but it does not fit the schema the server is supposed to respond
  • the JSON the server responded with might fit the schema, but the attribute we read from the JSON is invalid
  • and various stuff in between, of course

All the above-mentioned are cases where nothing went wrong with your function, but instead the input to it was invalid. As such, all of these fit "IllegalArgumentException".

Now, even if the JSON of the server was in fact correct, but your function was not programmed in a way to handle it this way (the server uses a newer version of a protocol than what the function was designed for) - and as such the server is in the right and your function is in the wrong - to the function that still equals to "illegal arguments".

Now, there are other things that could go wrong with the above-mentioned example, e.g.:

  • failed to write to a cache file
  • unable to load
  • invalid charset
  • IOException during reading from the socket
  • Out of memory
  • Hardware errors (e.g. RAM bitflips)

This is the other category of stuff that could go wrong: System errors. You might recognize, these are the errors that the checked exception system of Java has been heavily criticized for. These are the kinds of errors that are mostly extremely uncommon, and cannot be effectively recovered from for the operation either way (or have not been thought of being programmed to do so).

Some examples from the Java world:

  • byte[] bytes = "hello".getBytes("UTF-8"); // must handle UnsupportedEncodingException, even though UTF-8 will always exist

  • Thread.sleep(1000); // InterruptedException: even though sleeping is not critical, we must handle an exception that rarely occurs.

  • File file = new File("existing.txt");
if (file.exists()) {
  FileReader reader = new FileReader(file); // must handle FileNotFoundException
}

  • InputStream is = new ByteArrayInputStream(new byte[]{1, 2, 3});
is.read(); // must handle IOException for in-memory stream where IO errors are impossible

  • URI uri = new URI("https://example.com"); // must handle URISyntaxException

  • Cipher cipher = Cipher.getInstance("AES"); // must handle NoSuchAlgorithmException
MessageDigest digest = MessageDigest.getInstance("SHA-256"); // must handle NoSuchAlgorithmException
  • Process process = new ProcessBuilder("cmd", "/c", "dir").start(); // must handle IOException
  • List<String> lines = Files.readAllLines(Path.of("file.txt")); // must handle IOException
  • ZipFile zip = new ZipFile("archive.zip"); // must handle IOException
  • Desktop.getDesktop().browse(new URI("https://example.com")); // must handle IOException, URISyntaxException
  • ServerSocket server = new ServerSocket(0); // Port 0 dynamically selects an available port, still must handle IOException

All of these are examples where in Java, you must handle exceptions in the most uncommon or even impossible circumstances.

In Kotlin, this was changed: you are no longer forced to catch such ridiculous cases. You may still choose to do so (just wrap in runCatching { ... } or similar), but there will no longer be a compilation error because you did do not wrap "hello".getBytes("UTF-8") in a try-catch for UnsupportedEncodingException.

Expectable errors vs unexpectable/unrecoverable errors

In the above Java examples, we saw some cases that are either impossible, or if they would happen would be mostly unrecoverable from anyways (because apparently something is extremely wrong with the installation / version).

Generally, all the "system errors" cannot be recovered from, even if you tried to catch these runtime exceptions (what do you do if UTF-8 is suddenly no longer a valid charset/AES is no longer a valid cipher/SHA256 is no longer a valid digest? What do you do if the config file you just read from suddenly no longer exists? If it still exists but you no longer have read permissions?). Realistically, these could be data corruption, unresolvable services (e.g. database), or internal logic inconsistencies.

In Java, some developers handle / wrap checked exceptions with code which add little value (as they have to catch them, even when it's impossible to add value), and then they call it "defensive programming".

An alternative approach is "fail-fast", or as Joe Armstrong (Erlang) calls it "let-it-crash". This is what we try to do.

For unrecoverable errors where it is useless to try to handle them, dispose of Java "defensive programming" stance (if it can even be called that way), and instead let things that fail crash (rather than polluting your code with needless guards trying to keep track of the wreckage).

In our specific case, our main work are REST APIs. Exceptions are handled by a layer above yours. You can just have such a system error exception flow upwards - the server process will not fail, instead the HTTP REST request you are processing will just fail with a 500 with the appropriate error message, e.g. (HTTP 500: "Cannot connect to database"). No need to handle this error case at every single database call manually.

Using Result

We already established that it is in almost all cases useless or impossible to handle system errors, there simply is no way of adding value to that.

However, the other exception cause (illegal function input) can be handled with added-value, or sometimes even be recovered from, e.g.: Prompting user for alternative, using backup service to resolve data, etc... This value-add is particularly possible because we can already expect that everything with remote input (including user input) has a considerable likelihood to fail.

In the scope of REST APIs, nothing much changes (in a service, you can have the exception flow upward, and the HTTP server will show an appropriate error message). But in other circumstances, we might want to do something more specific, e.g. show a popup to the user in a wallet application. For this, we need to be able to:

  • know that a certain operation has a high likelihood of failing
  • want to explicitly know if a certain operation was successful or failed
  • want to get information about the failure

While in some programming languages it is common to return null, or have some error code return, and name functions in a special way if they might return null / an error, we have a much more elegant construct in Kotlin available to achieve these 3 points: Result.

It works like this:

fun parseKeyFromPEM(pem: String): Result<Key> =
    runCatching { /*... parse method 1 ...*/ }
        .recoverCatching { /*... parse method 2 ...*/ }
  • It is immediately recognizable that the function might not succeed, because it returns Result
  • with the Result.isSuccess/Result.isFailure we know if the operation was successful or failed
  • we have all information available about any internal-thrown exception for later processing (e.g. showing error message in popup)

The Result type provides us with some useful capabilities surrounding handling exceptions:

  • We can choose to ignore the possibility of a failure with result.getOrThrow(), propagating any exceptions upward
  • We can choose to ignore the possibility of a failure with println("The result is: ${result.getOrDefault("Unknown")}")
  • We can choose to handle success or error (or both) cases with result.onSuccess { println("Result is $it") } and result.onFailure { println("Error is ${it.message}")) }, result.fold
  • We can (try to) recover from errors with result.recover {} / result.recoverCatching {}
  • We can continue with Results (carrying errors through) with result.map {}
  • ...

Throw specific exceptions

We established that exceptions are grouped either into IllegalArgumentException or IllegalStateException. It is also very effortless to do so with

  • require() { "" } throwing IllegalArgumentException; and
  • check() { "" }, error("") throwing IllegalStateException

For example:

fun updateUsername(name: String) {
    require(user.isNameChangeAllowed()) { "You do not have the permission to update usernames yourself." }

    require(name.isNotBlank()) { "Provided name '$name' is blank." }
    require(name.length() >= 3) { "Provided name '$name' is less than the minimum 3 characters long." }
    require(name !in setOf("admin", "root", "system")) { "Name '$name' is not allowed" }

    // ... Update procedure here ...
    val updatedRows: Long = someSqlCommand()
    check(updatedRows != 0) { "Could not update user row" }
}

(In this example, none of the errors are recoverable in any way, so catching them in the function would serve zero purpose).

As you can see, it is extremely easy and effortless to indicate errors that way, but it has a drawback: While these functions do include the (hopefully helpful) provided error message, they are still generic IllegalArgumentException/IllegalStateException. This means that an error message is shown, and due to the inclusion of the (hopefully helpful) error message, indeed actually also makes sense to users — but not to computers, because these messages are meant for humans and not for computers, and they (usually) do not understand human messages.

Especially in our REST APIs, this would make it basically impossible to handle errors for the caller, as typically it's not humans making these HTTP requests in the Swagger interface, but they are made by server backends, wallet applications, etc.

These require a unique error code that special code can be written for to somehow handle it. In the above example this could be, we can already determine that neither we nor the caller can effectively recover from the SQL exception (a typical operation error, where you could only try to execute the operation again at a later point).

However, everything involving the illegal arguments a user could provide could be covered in more specific ways:

  • InvalidPermissionException
    • or even more specifically: NameChangeNotAllowedException
  • InvalidProvidedNameException
    • or even more specifically: BlankNameProvidedException, MinimumNameLengthNotMetException, BlacklistedNameProvidedException

This is a consideration (effort trade-off) you have to make yourself. Again, it would be unrealistic to try to catch every single thing that could ever possibly fail, but stuff that you already expect beforehand, and could be very common could very well deserve its custom specific-exception.

For example:

data class InvalidNameProvidedException(val name: String, override val message: String) :
    IllegalArgumentException(message = message)

data class BlankNameProvidedException : InvalidNameProvidedException(name = "", message = "Name is blank")
data class MinimumNameLengthNotMetException(name: String, minimumLength: Int) : InvalidNameProvidedException(
    name = name,
    message = "Name '$name' is too short (${name.length}). Minimum length is $minimumLength."
)
data class BlacklistedNameProvidedException(name: String) :
    InvalidNameProvidedException(name = name, message = "Name '$name' is not allowed")

Throw useful exceptions

Carry exceptions through

There is a reason why Exceptions are used, instead of e.g. returning null. Exceptions provide a way of determining "what went wrong where why" (when → provided by log system).

However, for this to be fulfilled, you have to carry through exceptions that happen, e.g.

fun doSomething(str: String) =
    runCatching {
        /* ... */
        Base64.UrlSafe.decode(str)
        /* ... */
    }.getOrElse { error -> throw IllegalArgumentException("Failed doing X on: $str", error) }

doSomething("/wD+AA==")

Here, the original error is carried through as cause of the exception. This causes a useful stack trace to be generated:

Exception in thread "main" java.lang.IllegalArgumentException: Failed doing X on: /wD+AA==
    at id.walt.mypackage.something.DoSomethingKt.doSomething(DoSomething.kt:35)
    at id.walt.mypackage.something.DoSomethingKt.main(DoSomething.kt:38)
    at id.walt.mypackage.something.DoSomethingKt.main(DoSomething.kt)
Caused by: java.lang.IllegalArgumentException: Invalid symbol '/'(57) at index 0
    at kotlin.io.encoding.Base64.decodeImpl(Base64.kt:524)
    at kotlin.io.encoding.Base64.decode(Base64.kt:285)
    at kotlin.io.encoding.Base64.decode$default(Base64.kt:279)
    at kotlin.io.encoding.Base64.decode(Base64.kt:351)
    at kotlin.io.encoding.Base64.decode$default(Base64.kt:349)
    at id.walt.mypackage.something.DoSomethingKt.doSomething(DoSomething.kt:33)
    ... 2 more

We can see everything effectively:

  • what went wrong: Base64.decode
  • where: id.walt.mypackage.something.DoSomethingKt.doSomething(DoSomething.kt:33)
  • why: Invalid symbol '/'(57) at index 0

Changing the above function to not carry the error through when throwing causes us to miss out on all of these:

fun doSomething(str: String) =
    runCatching {
        /* ... */
        Base64.UrlSafe.decode(str)
        /* ... */
    }.getOrElse { error -> throw IllegalArgumentException("Failed doing X on: $str" /* no cause set here */) }

Exception in thread "main" java.lang.IllegalArgumentException: Failed doing X on: /wD+AA ==
at id.walt.mypackage.something.DoSomethingKt.doSomething(DoSomething.kt:35)
at id.walt.mypackage.something.DoSomethingKt.main(DoSomething.kt:38)
at id.walt.mypackage.something.DoSomethingKt.main(DoSomething.kt)

Suddenly we lose all the error information:

  • what went wrong: ? - something in the call stack of doSomething
  • where: ? — somewhere in the call stack of doSomething
  • why: ? - ?
The message is not the exception

Something I've seen often is along the lines of this:

{ // ...
}.onFailure { error ->
    logger.debug { "error: $error" }
    context.respond(
        HttpStatusCode.BadRequest,
        error.message ?: "Unknown error",
    )
}

(which is obviously completely useless — why do that?)

Always show the full stack traces, not just "Error: $message". Otherwise, you throw away two of the three answers to the "what where why" questions.

IdiomsNaming variables