Protecting Replicate passwords

Replicate stores secrets (e.g. passwords and keys) in its internal repository, enabling it to perform secure operations during runtime, such as connecting to an endpoint, connecting to a web server, connecting to an email server, connecting to a remote file transfer server, and so on.

As a rule, all UI values displayed as asterisks are stored encrypted and never transmitted or exposed by the API. For instance, when exporting task definitions, all passwords are encrypted and can only be decrypted if the source machine’s mk.dat file - and possibly also the account or machine identity (on Windows) - is copied to the target machine. See also Master key considerations when exporting and importing tasks.

Secrets that appear in Replicate settings are stored in an encrypted form with the following properties:

• The secret is encrypted using the AES-256 encryption algorithm.
• The encryption key, also known as the 'master key', is a 256-bit key that is stored (and protected as described later) in the master key file (mk.dat).
• The encryption process uses a salt that depends on the name and type (or 'context') of the protected entity. This prevents reuse attacks.
• The encryption process uses a nonce so that the same secret is always encrypted differently. This prevents dictionary attacks.

When exported to a JSON file, secrets appear in the following format:

"{Zxhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh}"

Where:

• 'Z' is a fixed character
• 'x' a protection method indicator
• 'hhhhhh…' is a hexadecimal representation of the encrypted secret

Upon import, if a secret is provided in clear text (i.e. not in the format shown above), it is automatically encrypted and stored in the protected format.

The master key used in encrypting secrets is stored in the master key file (mk.dat) described below.

The master key file

The master key file is a binary file that stores the root secrets of Replicate. These root secrets currently include:

• The AES 256-bit master key, which is used for encrypting passwords and other secrets in Replicate settings.

• The server admin password, which is used (by the UI server) to access the replication server remotely.

  Information note

  When running multiple Replicate Linux instances, you need to enter the server admin password for each instance. For details, see Installing an additional instance of Replicate as a service.

• The default location for the master key file is the product data folder - by default <product-dir>/data - or data directories when multiple instances of Replicate are installed on Linux. If the server admin password does not exist, it is automatically created with randomly generated passwords that are safe but unknown. The user can change the server admin password as well as the master key to known values, if needed (e.g to connect to the replication server remotely).

  For more information, see Changing the server password.

When Replicate is set to run on a cluster using the same settings and storage, the mk.dat file should also be the same (or shared). Similarly, if the Replicate settings are exported in order to be moved to a different machine, the mk.dat file should also be moved in order to avoid the need to reenter all secrets.

The procedure for changing the master key as well as measures that can be taken to protect the file containing the master key are described in Changing and protecting the master key.

Changing and protecting the master key

This section describes how to change the master key as well as how to prevent unauthorized access to the master key file.

Changing the master key replacement

The master key is originally a randomly generated 256-bit key. It can be changed as described below immediately after installation with no consequences. However, if you change the master key after endpoints or tasks are already configured, all stored secrets will no longer be valid (as they were encrypted using the old key). Therefore, after changing the master key, you need to reenter the passwords in all the relevant places.

To change the Master Key:

1. Stop any running tasks.
2. Stop the Replicate services or instance(s) on Linux.
3. Open a command prompt as an administrator/root.

4. Change the working directory to the product "bin" directory and then run the following command (on Linux run ./repctl …):

   repctl [-d data-directory] setmasterkey your_new_master_key [master_key_scope=scope]

   where data-directory is the name of the Replicate data directory (by default: data).

   Information note

   You only need to include -d data-directory in the command if you changed the default data directory (e.g. when installing multiple Replicate Linux instances).

   Information note

   In order not to have the key appear in the shell history, you can use the command interface (on Linux run ./repctl):

   repctl {enter}

   setmasterkey your_new_master_key [master_key_scope=scope] {enter}

   quit {enter}

   See Protecting the master key file from misuse for the master_key_scope options.

   Example:

   repctl -d opt/mydatadir setmasterkey 78543043vuiyfyrf64454555jy65 master_key_scope=1

5. Start the Qlik Replicate Server or instance(s) on Linux.
6. Reenter the access passwords in all endpoints.
7. Start the tasks.

Protecting the master key file from misuse

Access to the mk.dat file is restricted to administrators (Windows) or to the users in the group under which the product was installed (Linux). Care should be taken to maintain this file protection.

The mk.dat file is always encrypted by some built-in, fixed key. On Windows, there are two additional options for preventing unauthorized access and use of the mk.dat file. These are as follows:

• Tying the mk.dat to the Machine Profile - With this option, the mk.dat file can only be used on the Windows machine where it was created. Using the file from a different machine will generate an error. This option is not appropriate for clusters that share the mk.dat file as only one machine will be able to read it.
• Tying the mk.dat to a User Profile - With this option, the mk.dat file can only be used under the identity that was used while creating it (typically the user that installed the product). For this option to work, the Replicate services must be modified to run under the same user. If the user has a roaming profile, then the mk.dat file can be used on multiple machines (e.g. in a cluster).

These options are specified in the master_key_scope option of the setmasterkey command. They should be set at the same time that the master key is set since any such change invalidates the stored passwords.

The master key scopes are:

• 1 (Constant) - The default. The mk.dat file can be used wherever it is copied to.
• 2 (User) - The mk.dat file can only be used under the same account as the one that was used when creating it.
• 3 (Machine) - The mk.dat file can only be used on the same machine where it was created.

Master key considerations when exporting and importing tasks

To be able to export tasks from one machine and then import them to another, the same master key must exist on both machines. This means that if you change the master key on one machine, you must also change it on the other machine.

For more information on importing and exporting Replicate tasks, see Exporting tasks.