Tokenization

What is Tokenization?

Akeyless Platform provides Tokenization as a service. Tokenization is a process of encrypting sensitive data, such as social security, phone, and credit card numbers, in a way that will preserve data format and uniqueness, and allow for data decryption later on.

Akeyless Platform performs Tokenization using a Format Preserving Encryption (FPE) algorithm.

Why is it important to preserve the original format of data?

First, a Format Preserving Encryption (FPE) is a method of encrypting data for systems that require it to be in a specific format and to pass certain validation checks in order to use it. By using it, companies can avoid costly and time-consuming changes to their data models and code.

The second major advantage FPE has over conventional encryption is that data such as credit cards or Social Security numbers can still be used as a unique key to identify a row in a database.

Vaultless Tokenization

Akeyless Platform supports Vaultless tokenization where "vaultless" means that a tokenized secret is not stored in the Akeyless Platform at any time during or after the tokenization process.

Akeyless Platform provides built-in tokenization templates for the following data types:

• Credit/Debit/Pre-paid card numbers (must be compliant with the Luhn algorithm)
• Social Security Numbers (SSN)
• US phone numbers

And custom tokenization templates (where you can provide a custom alphabet, tokenization pattern, and data output format) are also supported.

Working With a Tokenizer from the CLI

This section of the guide explains the workflow with the Akeyless CLI. If you wish to work from the Akeyless Console UI, go to this section.

Create a Tokenizer from the CLI

To create a Tokenizer from the CLI, run the following command:

👍

Note

Please note that custom tokenization has additional mandatory parameters.

Tokenization using Templates

akeyless create-tokenizer \
--name <Tokenizer name> \
--tokenizer-type <vaultless> \
--template-type <SSN, CreditCard, USPhoneNumber> \
--tweak-type <Supplied, Generated, Internal, Masking>

Custom Tokenization

akeyless create-tokenizer \
--name <Tokenizer name> \
--tokenizer-type <vaultless> \
--template-type <Custom> \
--tweak-type <Supplied, Generated, Internal, Masking> \
--alphabet <Symbols to use for tokenization> \
--pattern <A regexp pattern to extract tokenized parts> \
--encoding-template <An expression to alter the template of the encryption output> \
--decoding-template <An expression to alter the template of the decryption output>

Where:

• name: A unique name of the tokenizer. The name can include the path to the virtual folder where you want to create the new tokenizer, using the slash / separators. If the folder does not exist, it will be created together with the tokenizer.

• tokenizer-type: Currently, Akeyless platform supports only vaultless tokenization. So, the default value of this parameter is vaultless.

• template-type:The format of the tokenized secret. The following templates are available:
  SSN, CreditCard, USPhoneNumber, and Custom.

• tweak-type: The tweak type to use in tokenization. The following tweak types are available:
  Supplied, Generated, Internal, and Masking.

  • Supplied: Provided by the customer for each encrypt/decrypt operation.

  • Generated: Created by Akeyless for each encryption operation and returned to the customer. The customer must provide it when decrypting.

  • Internal: Generated by Akeyless when creating a tokenizer and saved with the tokenizer data. Thus the same tweak is used when encrypting or decrypting all tokens of this tokenizer.

  • Masking: Created by Akeyless for each encryption operation, but it is not returned to the customer. Thus encryption with a Masking tweak is a one-way operation (i.e., the data cannot be decrypted).

• alphabet: A string of allowed symbols for Custom tokenization.

• pattern: A regexp pattern that is used to extract the parts that should be tokenized during Custom tokenization. It can use named and non-named capture groups (e.g., (?P<first>\d{3})-(?P<last>\d{3}) or (\d{3})-(\d{3}) ).

• encoding-template: An expression to alter the template of the encryption output for Custom tokenization (e.g., $1-$2-$groupname, where $1 or $2 are non-named capture groups, and $groupname is a named capture group).

• decoding-template: An expression to alter the template of the decryption output for Custom tokenization.

You can find the complete list of parameters for this command in the CLI Reference - Encryption Keys section.

Tokenize and Detokenize a Secret from the CLI

To tokenize a secret from the CLI, run the following command:

Shell

akeyless tokenize \
--tokenizer-name <Tokenizer name> \
--plaintext <Data to be encrypted> \
--tweak <Base64-encoded tweak value>

To detokenize a secret from the CLI, use this command:

Shell

akeyless detokenize \
--tokenizer-name <Tokenizer name> \
--ciphertext <Data to be decrypted> \
--tweak <Base64-encoded tweak value that was used for encryption>

Working With a Tokenizer from the Console

This section of the guide explains the workflow with the Akeyless Console. If you wish to work from the Akeyless CLI, go to this section.

Create a Tokenizer in the Akeyless Console

1. Log in to the Akeyless Console, and go to Items > New > Tokenizer.

2. Define a Name of the tokenizer, and specify the Location as a path to the virtual folder where you want to create the new tokenizer, using slash / separators. If the folder does not exist, it will be created together with the tokenizer.

3. Select Tweak Type to use in tokenization. The following tweak types are available:
   Supplied, Generated, Internal, and Masking.

   • Supplied: Provided by the customer for each encrypt/decrypt operation.

   • Generated: Created by Akeyless for each encryption operation and returned to the customer. The customer must provide it when decrypting.

   • Internal: Generated by Akeyless when creating a tokenizer and saved with the tokenizer data. Thus the same tweak is used when encrypting or decrypting all tokens of this tokenizer.

   • Masking: Created by Akeyless for each encryption operation, but it is not returned to the customer. Thus encryption with a Masking tweak is a one-way operation (i.e., the data cannot be decrypted).

👍

Note

Best Practice - The Masking tweak is meant mainly for situations in which you want to allow exploration and analysis of certain data sets without exposing the data, and so take the measure of keeping its structure with a tokenizer but still encrypting it in a non-reversible manner. If you are not familiar with operations that require Masking, we suggest you select a different tweak type.

4. Click Next to go to the second step of the wizard.

5. Select Template Type to use in tokenization. The following templates are available: Credit Card, SSN, US Phone number, and Custom.

If you select the Custom template, you'll need to provide values for the following parameters:

• Alphabet: A string of allowed symbols for tokenization.

• Custom Pattern: A regexp pattern that is used to extract the parts which should be tokenized. It can use named and non-named capture groups (e.g., (?P<first>\d{3})-(?P<last>\d{3}) or (\d{3})-(\d{3}) ).

• Encoding Template: An expression to alter the template of the encryption output for tokenization (e.g., $1-$2-$groupname, where $1 or $2 are non-named capture groups, and $groupname is a named capture group).

• Decoding Template: An expression to alter the template of the decryption output for tokenization.

6. Click on Finish.

Tokenize and Detokenize a Secret in the Akeyless Console

1. Log in to the Akeyless Console, and go to Items.

2. Browse to the folder where you created a tokenizer.

3. Select the tokenizer and click the Tokenize / Detokenize button.

4. To tokenize a secret:
   a. In the pop-up, select the Tokenize tab.
   b. Provide the secret value (and a Tweak - for a Tokenizer with a "Supplied" tweak type).
   c. Click Tokenize.
   d. Copy and save the tokenized value (and a Tweak - for a Tokenizer with a "Generated" tweak type).

5. To detokenize a secret:
   a. In the pop-up, select the Detokenize tab.
   b. Provide the tokenized value (and a Tweak - for a Tokenizer with "Supplied" and "Generated" tweak types).
   c. Click Detokenize.
   d. Copy and save the detokenized value.

Tutorial

Check out our tutorial video on Creating and Using a Tokenizer.