Even better interop using customizable variants

Interop with JavaScript/TypeScript is a top priority for ReScript. Recently we've made some changes to the runtime representation of variants that'll allow you to use variants for a large number of new interop scenarios, zero cost. This is important, because variants is the feature of ReScript, enabling great data modeling, pattern matching and more. Pattern matching is really the key here. The changes we've made will enable you to write idiomatic ReScript leveraging pattern matching in many more scenarios.

These changes will ship in v11 of ReScript, and in this post we'll tell you all you need to know about them.

TLDR;

Customizable runtime representation

We're making the runtime representation of variants customizable. This will allow you to cleanly map variants external data and APIs in many more cases than before.

That in turn will let you write idiomatic ReScript in more cases when interfacing with external data, leveraging the power of pattern matching and other ReScript features.

Zero cost bindings to discriminated unions

This covers variants with a discriminator, which will map cleanly to JavaScript/TypeScript discriminated unions.

Discriminated unions are getting more and more popular in TypeScript (for good reason!) and this will open up a whole new venue for simple interop for ReScript. This will let you effortlessly utilize the power of ReScript's pattern matching and similar features, without needing any runtime conversion. In addition it'll also greatly improve debuggability, because the runtime representation will map cleanly to the variant type you've defined.

Unboxed (untagged) variants

We also introduce untagged variants, meaning variants where the underlying runtime representation is a primitive. This will let you cleanly map to things like heterogenous array items, nullable values, and more.

The current state

We have several tools available today in ReScript for binding to external values without having to convert them to a ReScript runtime representation. Most notably, we have polymorphic variants, allowing us to bind zero cost to string and number literals. An example:

RESCRIPT
// These will both be
type myType = [#"store-or-network" | #"network-only"]

However, polymorphic variants have many downsides when compared to regular variants. They still have many good and valid use cases, but more often than not the reason for picking them has been their runtime representation mapping cleanly to external data. Meaning they are typically chosen even though choosing regular variants would almost always have been a better choice, all things equal.

In addition to this, there's currently no way to map to discriminated unions in TypeScript (which are getting more and more popular) without needing to do manual runtime conversion of the discriminated union into a ReScript variant. This is cumbersome in the default case, and even more cumbersome when dealing with things like recursive structures, where each level of recursion needs to do explicit conversion.

With ReScript v11, these issues will finally be solved. Let us take you on a tour of what has changed, and how it improves on the current situation.

Tagged variants

Variants with payloads have always been represented as a tagged union at runtime. Here's an example:

RESCRIPT
type entity = User({name: string}) | Group({workingName: string})

let user = User({name: "Hello"})

Is represented as:

JAVASCRIPT
var user = {
  TAG: /* User */ 0,
  name: "Hello",
};

However, this has been problematic when binding to external data, because there has been no way to customize the discriminator (the TAG property) or how its value is represented for each variant case (0 representing User here).

Let's examplify this by imagining we're binding to an external union that looks like this in TypeScript:

TYPESCRIPT
type LoadingState =
  | { state: "loading"; ready: boolean }
  | { state: "error"; message: string }
  | { state: "done"; data: Data };

Currently, there's no good way to use a ReScript variant to represent this type without resorting to manual and error prone runtime conversion. However, with the new funcionality, binding to the above zero cost is easy:

RESCRIPT
@tag("state")
type loadingState = | @as("loading") Loading({ready: bool}) | @as("error") Error({message: string}) | @as("done") Done({data: data})

let state = Error({message: "Something went wrong!"})

This will compile to:

JAVASCRIPT
var state = {
  state: "error",
  message: "Something went wrong!",
};

Let's break down what we've done to make this work:

• The @tag attribute lets you customize the discriminator. We're setting that to "state" so we map to what the external data looks like.

• Each variant case has an @as attribute. That controls what each variant case is discriminated on. We're setting all of the cases to their lowercase equivalent, because that's what the external data will look like.

The end result is clean and zero cost bindings to the external data, in a way that previously would require manual runtime conversion.

Let's look at a few more real world examples.

Binding to TypeScript enums

TYPESCRIPT
// direction.ts
/** Direction of the action. */
enum Direction {
  /** The direction is up. */
  Up = "UP",

  /** The direction is down. */
  Down = "DOWN",

  /** The direction is left. */
  Left = "LEFT",

  /** The direction is right. */
  Right = "RIGHT",
}

export const myDirection = Direction.Up;

Previously, you'd be forced to use a polymorphic variant for this if you wanted clean, zero-cost interop:

RESCRIPT
type direction = [ #UP | #DOWN | #LEFT | #RIGHT ]
@module("./direction.js") external myDirection: direction = "myDirection"

Notice a few things:

• We're forced to use the names of the enum payload, meaning it won't fully map to what you'd use in TypeScript

• There's no way to bring over the documentation strings, because polymorphic variants are structural, so there's no one source definition for them to look for docstrings on. This is true even if you annotate with your explicitly written out polymorphic variant definition.

With the new runtime representation, this is how you'd bind to the above enum instead:

RESCRIPT
/** Direction of the action. */
type direction =
  | /** The direction is up. */
  @as("UP")
  Up

  | /** The direction is down. */
  @as("DOWN")
  Down

  | /** The direction is left. */
  @as("LEFT")
  Left

  | /** The direction is right. */
  @as("RIGHT")
  Right

@module("./direction.js") external myDirection: direction = "myDirection"

Now, this maps 100% to the TypeScript code, including letting us bring over the documentation strings so we get a nice editor experience.

String literals

The same logic is easily applied to string literals from TypeScript, only here the benefits is even larger, because string literals have the same limitations in TypeScript that polymorphic variants have in ReScript.

TYPESCRIPT
// direction.ts
type direction = "UP" | "DOWN" | "LEFT" | "RIGHT";

There's no way to attach documentation strings to string literals in TypeScript, and you only get the actual value to interact with.

With the new customizable variants, you could bind to the above string literal type easily, but add documentation, change the name you interact with i. And there's no runtime cost.

Untagged variants

We've also implemented support for untagged variants. This will let you use variants to represent values that are primitives and literals in a way that hasn't been possible before. We'll explain what this is and why it's useful by showing a number of real world examples. Let's start with a simple one on how we can now represent a heterogenous array.

RESCRIPT
@unboxed type listItemValue = String(string) | Boolean(bool) | Number(float)

let myArray = [String("Hello"), Boolean(true), Boolean(false), Number(13.37)]

Here, each value will be unboxed at runtime. That means that the variant payload will be all that's left at runtime, no "box" holding the value.

It therefore compiles to this JS:

JAVASCRIPT
var myArray = ["hello", true, false, 13.37];

This was previously possible to do, leveraging a few tricks, when you didn't need to potentially read the values from the array again in ReScript. But, if you wanted to read back the values, you'd have to do a number of manual steps.

In the above example, reaching back into the values is as simple as pattern matching on them.

Let's look at a few more examples.

Pattern matching on nullable values

Previously, any value that might be null would need to be explicitly converted to an option, via for example Nullable.toOption (in Core), before you could use pattern matching on it. With the new possibility of defining unboxed variants, you'll now be able to define variants that will allow you to pattern match directly on the nullable values, without requiring explicit conversion. Let's look at what usage looks like before, vs now:

RESCRIPT
type userAge = {ageNum: Nullable.t<int>}
type rec user = {
  name: string,
  age: Nullable.t<userAge>,
  bestFriend: Nullable.t<user>,
}

let getBestFriendsAge = user =>
  switch user.bestFriend->Nullable.toOption {
  | Some({age}) =>
    switch age->Nullable.toOption {
    | None => None
    | Some({ageNum}) => ageNum->Nullable.toOption
    }
  | None => None
  }

As you can see, you need to convert each level of nullables explicitly, which breaks the pattern matching flow. With the new unboxed variant representation, we'll instead be able to do this:

RESCRIPT
// The type definition below is inlined here to examplify, but this definition will live in Core and be easily accessible
module Nullable = {
  @unboxed type t<'a> = Present('a) | @as(null) Null
}

type userAge = {ageNum: Nullable.t<int>}
type rec user = {
  name: string,
  age: Nullable.t<userAge>,
  bestFriend: Nullable.t<user>,
}

let getBestFriendsAge = user =>
  switch user.bestFriend {
  | Present({age: Present({ageNum: Present(ageNum)})}) => Some(ageNum)
  | _ => None
  }

Pattern matching goodness, just like you'd expect to be able to do!

This has a few implications:

• Dealing with external data, that is often nullable and seldom guaranteed to map cleanly to option without needing conversion, becomes much easier, and zero cost

• Special handling like @return(nullable) becomes redundant. This is good because that functionality does not work in all cases, but the new functionality will

Decoding and encoding JSON idiomatically

With unboxed variants, we have everything we need to define an actual, valid JSON type:

RESCRIPT
@unboxed
type rec json =
  | @as(false) False
  | @as(true) True
  | @as(null) Null
  | String(string)
  | Number(float)
  | Object(Js.Dict.t<json>)
  | Array(array<json>)

let myValidJsonValue = Array([String("Hi"), Number(123.)])

This above makes it impossible to define anything but valid JSON. You can also easily use this to pattern match on parsed JSON values. Here's an example of how you could write your own JSON decoders easily, leveraging pattern matching:

RESCRIPT
@unboxed
type rec json =
  | @as(false) False
  | @as(true) True
  | @as(null) Null
  | String(string)
  | Number(float)
  | Object(Js.Dict.t<json>)
  | Array(array<json>)

type rec user = {
  name: string,
  age: int,
  bestFriend: option<user>,
}

let rec decodeUser = json =>
  switch json {
  | Object(userDict) =>
    switch (
      userDict->Dict.get("name"),
      userDict->Dict.get("age"),
      userDict->Dict.get("bestFriend"),
    ) {
    | (Some(String(name)), Some(Number(age)), Some(maybeBestFriend)) =>
      Some({
        name,
        age: age->Float.toInt,
        bestFriend: maybeBestFriend->decodeUser,
      })
    | _ => None
    }
  | _ => None
  }

let decodeUsers = json =>
  switch json {
  | Array(array) => array->Array.map(decodeUser)->Array.keepSome
  | _ => []
  }

The point of this example isn't to showcase a perfect JSON decoder strategy, but rather show that the language itself will now have the building blocks to interact with JSON style data natively.

Wrapping up

• Major new features that will enable whole new use cases and scenarios