JSON and term formats

Terms everywhere

Any Erlang term can turn into a binary and back. term_to_binary and binary_to_term are the pair every Erlang developer knows.

The format is stable across OTP releases within reason, and it round-trips every value exactly. The first byte is always 131, the external term format tag.

erl

1> term_to_binary({ok, [1, 2]}).
<<131,104,2,119,2,111,107,107,0,2,1,2>>
2> binary_to_term(term_to_binary({ok, [1, 2]})).
{ok,[1,2]}

JSON libraries

Since OTP 27 the standard library ships a json module, so a fresh install encodes and decodes without a dependency. Older projects still use jiffy or jsx through rebar3, and all three encode maps and lists.

json:encode answers iodata, not a binary, so wrap it in iolist_to_binary when the result has to be one. Text values are binaries: a charlist encodes as an array of integers. This module hides both details; save it as wire.erl.

-module(wire). -export([json_encode/1, json_decode/1, pack/1, unpack/1]). json_encode(Term) -> iolist_to_binary(json:encode(Term)). json_decode(Bin) -> json:decode(Bin). pack(Term) -> base64:encode(term_to_binary(Term)). unpack(Bin) -> binary_to_term(base64:decode(Bin)).

erlc wire.erl
erl

1> wire:json_encode(#{name => <<"Ada">>}).
<<"{\"name\":\"Ada\"}">>
2> wire:json_encode([1, true, null]).
<<"[1,true,null]">>
3> wire:json_decode(<<"{\"name\":\"Ada\"}">>).
#{<<"name">> => <<"Ada">>}
4> wire:json_encode(#{name => "Ada"}).
<<"{\"name\":[65,100,97]}">>

The last answer is the trap: the charlist was encoded as an array of code points, not as text. The decode answer is a map with binary keys, so the maps tools of lesson 14 take over from there.

Base64 and friends

The base64 module encodes binaries to ASCII text and back. It is the standard way to carry binary payloads over text-only channels.

Packing a term means term_to_binary and then base64 in one step, and unpacking is the reverse. The payload survives a channel that would mangle raw bytes.

erl

1> base64:encode("hello").
<<"aGVsbG8=">>
2> base64:decode("aGVsbG8=").
<<"hello">>
3> wire:unpack(wire:pack({ok, [1, 2]})).
{ok,[1,2]}

Which format

Use term_to_binary inside your own cluster, JSON at the edges with other languages, and Base64 when text channels force it.

The rule of thumb: match the format to the reader, never to the writer.

% inside the cluster: term_to_binary % across languages: json:encode % text-only wire: base64:encode

Next: Processes and message passing

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does term_to_binary({ok, [1, 2]}) start with?

    Answer

    <<131>>

    The first byte 131 is the external term format tag that binary_to_term expects.

  2. What does binary_to_term(term_to_binary(Term)) answer?

    Answer

    The original term

    The pair round-trips every term exactly, including nested lists and tuples.

  3. Which module decodes JSON in OTP 27?

    Answer

    The stdlib json module

    OTP 27 added json to the standard library; jiffy and jsx predate it and remain common.

  4. What does json:encode/1 answer?

    Answer

    Iodata

    It answers iodata, so the lesson wraps it in iolist_to_binary/1 to get a binary.

  5. What does wire:json_encode([1, true, null]) answer?

    Answer

    <<"[1,true,null]">>

    Lists encode as JSON arrays, and true and null are the matching JSON literals.

  6. How does a charlist such as "Ada" encode in JSON?

    Answer

    As an array of code points [65,100,97]

    A charlist is a list, so JSON sees an array of integers; use a binary for text.

  7. What kind of keys does json:decode/1 produce?

    Answer

    Binary keys

    Decoded objects are maps with binary keys, as in #{<<"name">> => <<"Ada">>}.

  8. What does base64:encode("hello") answer?

    Answer

    <<"aGVsbG8=">>

    Base64 turns the five bytes into ASCII text, padded with one equals sign.

  9. What does wire:pack(Term) do?

    Answer

    term_to_binary then base64:encode

    Packing turns the term into the external format and then into text, and unpack reverses both steps.

  10. Which format belongs inside your own cluster?

    Answer

    term_to_binary

    Both ends are Erlang, so the native term format is exact and cheap; JSON belongs at the edges.

The examples on this page were compiled and run with Erlang/OTP 27 (erts-15.2.7.14) on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev