[MD] Research on encryption / decryption strategies

This is the sub issue for https://github.com/opensearch-project/OpenSearch-Dashboards/issues/1720

Research outline

1. Research on Encryption / Decryption Strategies
   1. Hash? - No Hash
   2. symmetric VS asymmetric - symmetric
   3. which symmetric algorithm - AES-GCM
   4. Recommendation / Best practices
2. Research on NodeJs Cryptography packages
   1. Bcrypt
   2. NodeJS native crypto library
   3. AWS Encryption SDK for Javascript
   4. Bcrypt VS Crypto VS AWS Encryption SDK - Recommand the last one

Research on Encryption / Decryption Strategies

Hash? - No Hash

The use case for our credential encryption: on OpenSearch Dashboard, the data is encrypted using a standard encryption algorithm that ensures data integrity, and the key must be decrypted and passed to OpenSearch for further authentication. As a result, we cannot use irreversible hashing functions.

Symmetric VS Asymmetric? - Symmetric

The use case of our credential encryption (as mentioned above) is categorized as encryption of data at rest.

Symmetric encryption, which employs a single key, is preferable for data-at-rest. Data stored in databases must be encrypted to prevent it from being compromised or stolen. Because this data only needs to be safe until it needs to be accessed in the future, it does not require two keys, only the one provided by symmetric encryption.

Asymmetric encryption, on the other hand, has typical use case such as data sent to other people via email. If only symmetric encryption was used on data in emails, an attacker could steal or compromise the data by obtaining the key used for encryption and decryption. Because their public key was used to encrypt the data, the sender and recipient ensure that only the recipient can decrypt the data using asymmetric encryption. Both types of encryption are used in conjunction with other processes, such as digital signing or compression, to provide additional data security.

Which symmetric Algorithm?

AES-GCM, as recommended by NIST (National Institute of Standards and Technology) - Ref https://csrc.nist.gov/publications/detail/sp/800-38d/final

Other Recommendation / Best practices

• It is recommended that symmetric keys be generated from the text to be encrypted rather than provided by the API caller, which the later adds risks to data security. Ref - key derivation function https://en.wikipedia.org/wiki/Key_derivation_function
• Good to have - For the implementation, define a key-management or lifecycle that includes secure key generation or establishment, key storage, key destruction, and key rotation.
• Use an AES key in preference order of 256 or 128 bits.
• Use a mac_tag size of 128 bits.
• Never encrypt more than 236 – 32 bytes, or protect more than 261 – 1 bytes of additional data in a single invocation (a call to the AES_GCM_Encrypt( ) primitive).
• Do not use decrypted plaintext if message authenticity fails.

Research on NodeJs Cryptography packages

Bcrypt

Bcrypt is a npm repository that provides robust hashing functions for password encryption that is also computationally expensive and irreversible. Examples of usage include:

const bcrypt = require('bcrypt');
const saltRounds = 10;
const myPlaintextPassword = 's0/\/\P4$$w0rD';
const someOtherPlaintextPassword = 'not_bacon';

NodeJS native crypto library

Crypto is a native NodeJS library that provides hashing as well as encryption and decryption. Examples of usage include as documented on :

const {
  scrypt,
  randomFill,
  createCipheriv
} = await import('node:crypto');

const algorithm = 'aes-192-cbc';
const password = 'Password used to generate key';

// First, we'll generate the key. The key length is dependent on the algorithm.
// In this case for aes192, it is 24 bytes (192 bits).
scrypt(password, 'salt', 24, (err, key) => {
  if (err) throw err;
  // Then, we'll generate a random initialization vector
  randomFill(new Uint8Array(16), (err, iv) => {
    if (err) throw err;

    // Once we have the key and iv, we can create and use the cipher...
    const cipher = createCipheriv(algorithm, key, iv);

    let encrypted = '';
    cipher.setEncoding('hex');

    cipher.on('data', (chunk) => encrypted += chunk);
    cipher.on('end', () => console.log(encrypted));

    cipher.write('some clear text data');
    cipher.end();
  });
});

AWS Encryption SDK for Javascript

The AWS Encryption SDK for Javascript provides a fully compliant, open-sourced, native Javascript implementation of the AWS Encryption SDK. Several core packages including in AWS Encryption SDK for JavaScript client for Node.js:

• decrypt-node
• encrypt-node
• raw-aes-keyring-node
• material-management-node

PoC https://github.com/noCharger/aws-encryption-sdk-poc

Quick hit - encrypt take the wrapping key (key to encrypt data key) as input, return the encrypted buffer with (plaintext + data key + salts) as well as metadata (data key from kdf). decrypt take the same wrapping key, and encrypted buffer, return the plaintext and metadata

Bcrypt VS Crypto VS AWS Encryption SDK

Although Bcrypt provides strong hash functionalities, it does not meet the use case in the context of multiple data sources. #1388

Crypto VS AWS Encryption SDK

The main advantage of using a native Crypto library is that it avoids introducing external modules, which prevents malicious code from being injected. However, the final decision favored the AWS Encryption SDK (the SDK) because of the following benefits:

1. Security best practice

The SDK provides hands on symmetric encryption mechinism following best practice. For example, when using RequireEncryptRequireDecrypt commitment policy, it provides the following benefits with the ALG_AES256_GCM_IV12_TAG16_HKDF_SHA384_ECDSA_P384 algorithm suite:

a. Envelope encryption has multiple befinits including strong protection on data keys, encryption the same data with multiple wrappign keys, etc

b. Key derivation algorithm: HKDF with SHA-384, which “helps you avoid accidental reuse of a data encryption key and reduces the risk of overusing a data key.” Ref: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/supported-algorithms.html

c. Signature algorithm: ECDSA with P-384 and SHA-384. Under multiple data source case, data source indices stored on OpenSearch can be modified / replaced by attacker, as @dblock pointed out. With ECDSA signature, ciphertext decryption will fail if it’s getting pullted. No one will be able to create another signature that verifies with the public key because the private key has been dropped.

2. Extensibility

The SDK includes the Keyring interface, which supports AWS Key Management Service (AWS KMS) symmetric encryption AWS KMS keys. It would save at least months of effort for any developer who wants to have extreme safety and durability of master key / wrapping key of the envelope encryption.

[AMoo-Miki]

@noCharger, I found the updated text very informative and helpful in my understanding of the reasoning behind the choice. Thanks a ton for your research and the update.

[noCharger]

I am not convinced that we needed an AWS package to do a simple AES encryption which is built into Node.

Hi @AMoo-Miki, updated the issue with more context. Please review~