Symmetric Encryption with ExCrypto in Elixir/Phoenix

The problem

Imagine you are building an Elixir Phoenix API called MyApp.

MyApp needs to interact with an external service on behalf of its users.

Let's call this external service Aliens.

You build a workflow where users can authenticate with Aliens.

At this point Aliens is sending you an api_secret that is unique to some particular user.

The challenge now is to securely persist all api_secret values we get from Aliens.

The solution

We cannot use the common crypto technique of "hashing" because hashing is (effectively) irreversible and we want to retrieve a particular user's api_secret at any time.

So we will use "symmetric encryption".

It's called "symmetric" because the same "key" is used for both encrypting and decrypting.

ExCrypto (click here for github repo)

There is a lib called :ex_crypto that is just a wrapper around Erlang's :crypto module.

Add :ex_crypto lib to the deps in your mix.exs and mix deps.get (the usual way to install libs).

We will use only 3 functions from :ex_crypto:

generate_aes_key/2 (click here for docs)

Open an iex shell with your project deps loaded by doing:

$ iex -S mix

In the iex shell that opens, do the following:

iex [1] > ExCrypto.generate_aes_key(:aes_256, :bytes)

That should return a tuple similar to this:

{:ok,
 <<227, 203, 213, 134, 28, 137, 72, 106, 172, 214, 138, 61, 177, 26, 128, 200,
   221, 80, 231, 150, 234, 238, 137, 1, 24, 18, 99, 155, 218, 72, 240, 88>>}

Later we will "hard-code" that <<227, 203, ..., 88>> value in our code as @aes_256_key.

encrypt/3 (click here for docs)

The encrypt/3 function accepts 3 arguments:

encrypt(key, auth_data, clear_text)

The key argument is an aes_256 key, just like the one we generated above.

The auth_data is just a string that we will be using as our cryptographic "salt".

The clear_text argument is the thing you want to encrypt.

For us, clear_text will be the api_secret we got from Aliens.

This function uses something called GCM mode which basically means it won't devour CPU.

If you read the docs linked above, you will see it returns a tuple like this:

{:ok, {_ad, {init_vec, cipher_text, cipher_tag}}} = ExCrypto.encrypt(aes_256_key, auth_data, clear_text)

init_vec is an abbreviation for initialisation_vector.

We will need all 3 variables init_vec, cipher_text and cipher_tag for decoding.

decrypt/5 (click here for docs)

The decrypt/5 function accepts 5 arguments:

decrypt(key, auth_data, init_vec, cipher_text, cipher_tag)

The key argument is the same aes_256 key used for encrypting.

The auth_data is also the same "salt" we used for encrypting.

The variables init_vec, cipher_text and cipher_tag variables are returned by encrypt/3.

Simple summary

Your "encryption key" is effectively comprised of 2 parts:

1. the aes_256_key generated above
2. some auth_data string

You can encrypt any clear_text string using encrypt/3, giving you 3 variables that effectively comprise your "encrypted data":

1. init_vec
2. cipher_text
3. cipher_tag

You can recover the clear_text string using decrypt/5, by passing 5 variables:

1. Your "encryption key" (2 variables)
2. Your "encrypted data" (3 variables)

We will be hard-coding aes_256_key but we will have the auth_data in the application configuration (which can be via e.g. an environment variable in :prod env).

The auth_data (application configuration)

For a production deployment, there are many ways to get this value into your host's environment and I won't explore that here.

For this article, let's assume this value is an environment variable in the host environment.

We put the auth_data in our application configuration by editing 3 files:

In dev.exs:

config :my_app, :alien_credential_cipher, System.get_env("ALIEN_CREDENTIAL_CIPHER", "alien-dev-cipher")

In test.exs:

config :my_app, :alien_credential_cipher, System.get_env("ALIEN_CREDENTIAL_CIPHER", "alien-test-cipher")

In prod.exs:

config :my_app, :alien_credential_cipher, System.get_env("ALIEN_CREDENTIAL_CIPHER")

In :dev and :test the app will "just work" with the default auth_data values "alient-dev-cipher" and "alien-test-cipher".

Note how the :alien_credential_cipher will be nil in production if we have not set the environment variable.

We will see later how a nil value for auth_data will result in loud errors (that's what we want).

You can generate a good secret like this:

$ mix phx.gen.secret

You should write this value down if you intend to use it in :prod.

(Optional) custom auth_data in :dev env

For a dev environment, you can do something like this (run this in your project root directory):

$ echo "export ALIEN_CREDENTIAL_CIPHER=\"$(mix phx.gen.secret)\"" > ./.env

That will make a file called .env in your project root directory.

Then just make sure to do this before running your Phoenix app if you want the env variable to be used:

$ source ./.env

The database

We are going to create a table called alien_credentials to hold the encrypted api_secret we get for a user.

Generate a migration like this:

$ mix ecto.gen.migration "create_alien_credentials"

Edit the change function in that generated file to look like this:

def change do
  create table(:alien_credentials) do
    add :user_id, references(:users, on_delete: :nothing), null: false
    add :init_vec_api_secret, :binary, null: false
    add :cipher_text_api_secret, :binary, null: false
    add :cipher_tag_api_secret, :binary, null: false
    timestamps()
  end

  create unique_index(:alien_credentials, [:user_id])
end

Note there is no :api_secret column.

We are only persisting the 3 variables that comprise our "encrypted data"

The crypto

This is a generic module that encrypts fields on Ecto changesets and decrypts fields on structs.

It exposes 2 public functions: encrypt_changeset_field/4 and decrypt_field/4.

Note both functions have the same arity and their signatures differ only in the first argument (since we encrypt some field on an %Ecto.Changeset{} but decrypt some field on a generic struct)

Note that if auth_data is nil then encryption will loudly put an error on the changeset but decryption will quietly set the decrypted value to nil.

defmodule MyApp.Crypto do
  @spec encrypt_changeset_field(%Ecto.Changeset{}, atom(), binary(), binary() | nil) ::
          %Ecto.Changeset{}
  def encrypt_changeset_field(changeset, field, _key, nil) do
    changeset
    |> Ecto.Changeset.add_error(field, "Auth data not provided")
  end

  def encrypt_changeset_field(changeset, field, key, auth_data) do
    clear_text =
      Ecto.Changeset.get_field(changeset, field)
      |> to_string()  # warning: raises exception for certain types, e.g. tuples

    case ExCrypto.encrypt(key, auth_data, clear_text) do
      {:ok, {_ad, {init_vec, cipher_text, cipher_tag}}} ->
        changeset
        |> Ecto.Changeset.put_change(init_vec_field(field), init_vec)
        |> Ecto.Changeset.put_change(cipher_text_field(field), cipher_text)
        |> Ecto.Changeset.put_change(cipher_tag_field(field), cipher_tag)
        |> Ecto.Changeset.delete_change(field)

      _ ->
        changeset
        |> Ecto.Changeset.add_error(field, "Data is not encryptable")
    end
  end

  @spec decrypt_field(term(), atom(), binary(), binary()) :: term()
  def decrypt_field(nil, _field, _key, _auth_data), do: nil
  def decrypt_field(struct, _field, _key, nil), do: struct

  def decrypt_field(struct, field, key, auth_data) do
    init_vec_field_name = init_vec_field(field)
    cipher_text_field_name = cipher_text_field(field)
    cipher_tag_field_name = cipher_tag_field(field)

    init_vec = Map.get(struct, init_vec_field_name)
    cipher_text = Map.get(struct, cipher_text_field_name)
    cipher_tag = Map.get(struct, cipher_tag_field_name)

    decrypted_field =
      case ExCrypto.decrypt(key, auth_data, init_vec, cipher_text, cipher_tag) do
        {:ok, decrypt_result} -> decrypt_result
        _ -> nil
      end

    struct
    |> Map.put(field, decrypted_field)
    |> Map.put(init_vec_field_name, nil)
    |> Map.put(cipher_text_field_name, nil)
    |> Map.put(cipher_tag_field_name, nil)
  end

  defp init_vec_field(field), do: String.to_atom("init_vec_" <> to_string(field))
  defp cipher_text_field(field), do: String.to_atom("cipher_text_" <> to_string(field))
  defp cipher_tag_field(field), do: String.to_atom("cipher_tag_" <> to_string(field))
end

The model

Here is the model, which makes use of the Crypto module defined above.

The model has access to our effective "encryption key" which is made of 2 parts:

1. The hard-coded @aes_256_key which should be specific to each model that wants to do this.
2. The ephemeral ALIEN_CREDENTIAL_CIPHER which is specific to the host.

defmodule MyApp.AlienCredential do
  use Ecto.Schema

  import Ecto.Changeset

  alias MyApp.Accounts.User
  alias MyApp.AlienCredential
  alias MyApp.Crypto

  # generated using `ExCrypto.generate_aes_key(:aes_256, :bytes)`

  @aes_256_key <<227, 203, 213, 134, 28, 137, 72, 106, 172, 214, 138, 61, 177, 26, 128, 200, 221, 80, 231, 150, 234, 238, 137, 1, 24, 18, 99, 155, 218, 72, 240, 88>>

  schema "alien_credentials" do
    field :init_vec_api_secret, :binary
    field :cipher_text_api_secret, :binary
    field :cipher_tag_api_secret, :binary
    field :api_secret, :string, virtual: true
    belongs_to :user, User

    timestamps()
  end

  def changeset(, attrs) do
    user
    |> cast(attrs, [:api_secret, :user_id])
    |> validate_api_secret()
  end

  def decrypt_field(%AlienCredential{} = alien_credential, field_name),
    do:
      Crypto.decrypt_field(
        alien_credential,
        field_name,
        @aes_256_key,
        Application.get_env(:my_app, :alien_credential_cipher)
      )

  defp encrypt_field(%Ecto.Changeset{} = changeset, field_name),
    do:
      Crypto.encrypt_changeset_field(
        changeset,
        field_name,
        @aes_256_key,
        Application.get_env(:my_app, :alien_credential_cipher)
      )

  defp validate_api_secret(changeset) do
    changeset
    |> validate_required([:api_secret])
    |> validate_length(:api_secret, min: 6, max: 80)
    |> prepare_changes(&encrypt_api_secret/1)
  end

  defp encrypt_api_secret(changeset), do: encrypt_field(changeset, :api_secret)
end

Note that encryption of any field on any changeset can be accomplished with this simple bit of code:

Crypto.encrypt_changeset_field(
  changeset,
  field_name,
  @aes_256_key,
  Application.get_env(:my_app, :alien_credential_cipher)
)

Similarly, decryption is done like this:

Crypto.decrypt_field(
  alien_credential,
  field_name,
  @aes_256_key,
  Application.get_env(:my_app, :alien_credential_cipher)
)

It's easy to use.

You can re-use this pattern in other modules - just make sure to generate a new @aes_256_key and a new ALIEN_CREDENTIAL_CIPHER.

In practice

You only need to remember to decrypt fields on structs when you need them.

For example, you might fetch credentials from the database like this:

def get_alien_credential_by_user_id(user_id) do
  case Repo.get_by(AlienCredential, user_id: user_id) do
    %AlienCredential{} = alien_credential ->
      alien_credential
      |> AlienCredential.decrypt_field(:api_secret)

    _ ->
      nil
  end
end

Warning

The price of security with this approach is the risk of losing the ALIEN_CREDENTIAL_CIPHER.

Since it's an environment variable in :prod environment, a naive deployment won't automatically have it. Also, you can't regenerate it.

Motivation

The reward for this approach is that even if the database is compromised, any attackers will only have encrypted gibberish.

Even if the same attackers have access to the source code, they won't have the auth_data (the ALIEN_CREDENTIAL_CIPHER) so they cannot decrypt the data.

The only way to decrypt the data is to gain access to the running server and if that happens it's game over anyway.