Overview
The Oracle Cloud Infrastructure (OCI) Vault is a service that provides management of encryption keys and secret credentials. A vault is a storage container that holds keys and secrets. The Vault service not only secures your secrets it provides a central repository that allows them to be used in different notebooks and shared with only those that need access. No longer will your secrets be stored in code that can accidentally be checked into git repositories.
This blog demonstrates how to create a vault, a key, and store a secret that is encrypted with that key. It also demonstrates how to retrieve the secret so that it can be used in a notebook. The notebook explains how to update that secret and basic operations, such as listing deleting vaults, keys, and secrets.
Prerequisites
The blog shows you how to perform CRUD (create, read, update, delete) operations on vaults, keys, and secrets. These are all part of the Vault service. Your notebook session requires permissions to these resources. The account administrator needs to grant privileges to perform these actions. How the permissions are configured can depend on your tenancy configuration, see the Vault Service’s permissions documentation for details. The Vault Service’s common policies are:
allow group <group_name> to manage vaults in compartment <compartment_name>
allow group <group_name> to manage keys in compartment <compartment_name>
allow group <group_name> to manage secret-family in compartment <compartment_name>
</compartment_name></group_name></compartment_name></group_name></compartment_name></group_name>
The examples in this blog use use the OCI configuration file (~/.oci/config). If that has not been set up refer to the section in on setting up the oci configuration in the OCI CLI documentation. In a Data Science notebook session, the OCI CLI is already installed.
Introduction to the Vault Service
The Oracle Cloud Infrastructure Vault lets you centrally manage the encryption keys that protect your data and the secret credentials that you use to securely access resources.
Vaults securely store master encryption keys and secrets that you might otherwise store in configuration files or in code.
Use the Vault service to exercise control over the life cycle of keys and secrets. Integration with OCI Identity and Access Management (IAM) lets you control who and what services can access which keys and secrets and what they can do with those resources. The OCI Audit integration gives you a way to monitor key and secret use. Audit tracks administrative actions on vaults, keys, and secrets.
Keys are stored on highly available and durable hardware security modules (HSM) that meet Federal Information Processing Standards (FIPS) The main use case for a data scientist is to store a secret, such as an SSH key, database password, or some other credential. To do this, a vault and key are required.
Key and Secret Management Concepts
The following concepts are integral to understanding the Vault service.
Vaults
Vaults are logical entities where the Vault service stores keys and secrets. The Vault service offers different vault types. A virtual private vault is an isolated partition on an HSM. Vaults can share partitions on the HSM with other vaults.
Keys
Keys are logical entities that represent one or more key versions that contain the cryptographic material used to encrypt and decrypt data. The Vault service recognizes master encryption keys, wrapping keys, and data encryption keys.
Master encryption keys can be generated internally by the Vault service or imported to the service from an external source. Once a master encryption key has been created, the OCI API can be used to generate data encryption keys that the Vault service returns to you. By default, a wrapping key is included with each vault. A wrapping key is a 4096-bit asymmetric encryption key pair based on the RSA algorithm.
Key Version
Each master encryption key is assigned a version number. When a key is rotated, a new key version is created by the Vault service or it can be imported. Periodically rotating keys reduces the risk if a key is ever compromised. A key’s unique OCID remains the same across rotations, but the key version enables the Vault service to seamlessly rotate keys to meet any compliance requirements. Older key versions cannot be used for encryption. However, they remain available to decrypt data.
Hardware Security Modules
Keys and secrets are stored within an HSM. This provides a layer of physical security. Keys and secrets are only stored on HSM and cannot be exported from the HSM. HSMs meet the FIPS 140-2 Security Level 3 security certification. This means that the HSM hardware is tamper-evident, has physical safeguards for tamper-resistance, requires identity-based authentication, and deletes keys from the device when it detects tampering.
Envelope Encryption
The data encryption key used to encrypt your data is, itself, encrypted with a master encryption key. This concept is known as envelope encryption. OCI services do not have access to the plain text data without interacting with the Vault service and without access to the master encryption key that is protected by IAM.
Secrets
Secrets are credentials, such as passwords, certificates, SSH keys, or authentication tokens. You can retrieve secrets from the Vault service when you need them to access resources or other services.
Secret Versions
Each secret is automatically assigned a version number. When secrets are rotated and updated, the new secret has a new version number. A secret’s unique OCID remains the same across rotations and updates. It is possible to configure a rule that prevents a secret from being reused after rotation and updating. However, the older secret remains available.
Secret Bundles
A secret bundle consists of the secret contents, properties of the secret, the secret version (version number or rotation state), and user-provided contextual metadata for the secret.
Setting Up a Use Case
For the purposes of this blog, a secret is stored with the credentials needed to access a database. The sample code is designed so that any secret can be stored as long as it is in the form of a dictionary. To store your secret, just modify the dictionary.
# Sample credentials that are going to be stored.
credential = {'database_name': 'databaseName_high',
'username': 'admin',
'password': 'MySecretPassword',
'database_type': 'oracle'}
Note, to connect to an Oracle database the database_name value should be its connection identifier. You can find the connection identifier by extracting the credential wallet zip file and opening the tnsnames.ora file (connection_identifier = (…)). Usually the connection identifier will end with _high, _medium, or _low i.e.‘MyDatabaseName_high’.
Define the modules that are needed.
import base64
import json
import oci
import os
import random
import string
from oci.config import from_file
from oci.key_management import KmsManagementClient
from oci.key_management import KmsManagementClientCompositeOperations
from oci.key_management import KmsVaultClient
from oci.key_management import KmsVaultClientCompositeOperations
from oci.key_management.models import CreateVaultDetails
from oci.key_management.models import KeyShape
from oci.key_management.models import CreateKeyDetails
from oci.key_management.models import ScheduleKeyDeletionDetails
from oci.key_management.models import ScheduleVaultDeletionDetails
from oci.secrets import SecretsClient
from oci.vault import VaultsClient
from oci.vault.models import Base64SecretContentDetails
from oci.vault.models import CreateSecretDetails
from oci.vault.models import ScheduleSecretDeletionDetails
from oci.vault.models import UpdateSecretDetails
from oci.vault import VaultsClientCompositeOperations
from os import path
This blog uses the OCI Configuration file (~/.oci/config) to work with the Vault service. The following code obtains information about your OCI configuration.
# Select the configuration file to connect to OCI resources
config = from_file(path.join(path.expanduser("~"), ".oci", "config"), "DEFAULT")
# Select the compartment to create the secrets in.
# Use the notebook compartment by default
compartment_id = os.environ['NB_SESSION_COMPARTMENT_OCID']
Create a Vault
To store a secret, a key is needed to encrypt and decrypt the secret. This key and secret are stored in a vault. The following code creates a vault and stores the Vault OCID in the vault_id variable. The KmsVaultClient class takes a configuration object and establishes a connection to the key management service (KMS). Communication with KmsVaultClient is asynchronous. In most data science workflows, you will want to wait for this operation to complete. To do have synchronous communication, the KmsVaultClient is wrapped in a KmsVaultClientCompositeOperations object.
The details of the vault are specified using an object of the CreateVaultDetails type. A compartment ID must be provided along with the properties of the vault. Replace <vault_name> with a unique vault name.
# Create a VaultClientCompositeOperations for composite operations.
vault_client = KmsVaultClientCompositeOperations(KmsVaultClient(config))
# Create vault_details object for use in creating the vault.
vault_details = CreateVaultDetails(compartment_id=compartment_id,
vault_type=oci.key_management.models.Vault.VAULT_TYPE_DEFAULT,
display_name="<vault_name>"
# Vault creation is asynchronous; Create the vault and wait until it becomes active.
vault = vault_client.create_vault_and_wait_for_state(vault_details,
wait_for_states=[oci.vault.models.Secret.LIFECYCLE_STATE_ACTIVE]).data
vault_id = vault.id
</vault_name>
Create a Key
The secret is encrypted and decrypted using an AES key. The following code creates a key. The KmsManagementClient class takes a configuration object and the endpoint for the vault that is going to be used to store the key. It establishes a connection to the KMS. Communication with KmsManagementClient is asynchronous. As in the case of creating a vault, most data science workflows would want a synchronous behavior. So, the KmsManagementClient is wrapped in a KmsManagementClientCompositeOperations object.
The details of the key are specified using an object of type CreateKeyDetails. A compartment OCID must be provided along with the properties of the key. The KeyShape class defines the properties of the key. In this example, it is a 32-bit AES key.
Replace <vault_key_name> with a unique key name.
# Create a vault management client using the endpoint in the vault object.
vault_management_client = KmsManagementClientCompositeOperations(
KmsManagementClient(config, service_endpoint=vault.management_endpoint))
# Create key_details object that needs to be passed when creating key.
key_details = CreateKeyDetails(compartment_id=compartment_id,
display_name="<vault_key_name>",
key_shape=KeyShape(algorithm="AES", length=32))
# Vault creation is asynchronous; Create the vault and wait until it becomes active.
key = vault_management_client.create_key_and_wait_for_state(key_details,
wait_for_states=[oci.key_management.models.Key.LIFECYCLE_STATE_ENABLED]).data
key_id = key.id
</vault_key_name>
Secret
Store a Secret
The following code creates a secret that is to be stored. The variable credential is a dictionary and contains the information that is to be stored. The UDF dict_to_secret() takes a Python dictionary, converts it to a JSON string, and then Base64 encodes it. This string is what is to be stored as a secret.
The VaultsClient class takes a configuration object and establishes a connection to the Vault service. Communication with VaultsClient asynchronous. To make synchronous operation,VaultsClient is wrapped in a VaultsClientCompositeOperations object.
The contents of the secret are stored in a Base64SecretContentDetails object. This object contains information about the encoding being used, the stage to be used,and most importantly the payload (the secret). The CreateSecretDetails class is used to wrap the Base64SecretContentDetails object and also specify other properties about the secret. It requires the compartment OCID, the vault that is to store the secret, and the key to use to encrypt the secret. Replace <vault_secret_name> with a unique name of the secret..
# Encode the secret.
secret_content_details = Base64SecretContentDetails(
content_type=oci.vault.models.SecretContentDetails.CONTENT_TYPE_BASE64,
stage=oci.vault.models.SecretContentDetails.STAGE_CURRENT,
content=dict_to_secret(credential))
# Bundle the secret and metadata about it.
secrets_details = CreateSecretDetails(
compartment_id=compartment_id,
description = "Data Science service test secret",
secret_content=secret_content_details,
secret_name="<vault_secret_name>",
vault_id=vault_id,
key_id=key_id)
# Store secret and wait for the secret to become active.
vaults_client_composite = VaultsClientCompositeOperations(VaultsClient(config))
secret = vaults_client_composite.create_secret_and_wait_for_state(
create_secret_details=secrets_details,
wait_for_states=[oci.vault.models.Secret.LIFECYCLE_STATE_ACTIVE]).data
secret_id = secret.id
</vault_secret_name>
Retrieve a Secret
The SecretsClient class takes a configuration object. The get_secret_budle method takes the secret’s OCID and returns a Response object. Its data attribute returns SecretBundle object. This has an attribute secret_bundle_content that has the object Base64SecretBundleContentDetails and the content attribute of this object has the actual secret. This returns the Base64 encoded JSON string that was created with the dict_to_secret() function. The process can be reversed with the secret_to_dict() function. This will return a dictionary with the secrets.
secret_bundle = SecretsClient(config).get_secret_bundle(secret_id)
secret_content = secret_to_dict(secret_bundle.data.secret_bundle_content.content)
Update a Secret
Secrets are immutable but it is possible to update them by creating new versions. In the following code, the credential object updates the password key. To update the secret, a Base64SecretContentDetails object must be created. The process is the same as previously described in the Store a Secret section. However, instead of using a CreateSecretDetails object, an UpdateSecretDetails object is used and only the information that is being changed is passed in.
Note that the OCID of the secret does not change. A new secret version is created and the old secret is rotated out of use, but it may still be available depending on the tenancy configuration.
The following code updates the secret.
# Update the password in the secret.
credential['password'] = 'UpdatedPassword'
# Encode the secret.
secret_content_details = Base64SecretContentDetails(
content_type=oci.vault.models.SecretContentDetails.CONTENT_TYPE_BASE64,
stage=oci.vault.models.SecretContentDetails.STAGE_CURRENT,
content=dict_to_secret(credential))
# Store the details to update.
secrets_details = UpdateSecretDetails(secret_content=secret_content_details)
# Create new secret version and wait for the new version to become active.
secret_update = vaults_client_composite.update_secret_and_wait_for_state(
secret_id,
secrets_details,
wait_for_states=[oci.vault.models.Secret.LIFECYCLE_STATE_ACTIVE]).data
List Resources
This section demonstrates how to obtain a list of resources from the vault, key, and secrets
List Secrets
The list_secrets method of the VaultsClient provides access to all secrets in a compartment. It returns a Response object and the data attribute in that object is a list of SecretSummary objects.
The SecretSummary class has the following attributes:
◉ compartment_id: Compartment OCID.
◉ defined_tags: Oracle defined tags.
◉ description: Secret description.
◉ freeform_tags: User-defined tags.
◉ id: OCID of the secret.
◉ key_id: OCID of the key used to encrypt and decrypt the secret.
◉ lifecycle_details: Details about the lifecycle.
◉ lifecycle_state: The current lifecycle state, such as ACTIVE and PENDING_DELETION.
◉ secret_name: Name of the secret.
◉ time_created: Timestamp of when the secret was created
◉ time_of_current_version_expiry: Timestamp of when the secret expires if it is set to expire.
◉ time_of_deletion: Timestamp of when the secret is deleted if it is pending deletion.
◉ vault_id: Vault OCID that the secret is in.
Note that the SecretSummary object does not contain the actual secret. It provides the secret’s OCID and that can be used to obtain the secret bundle, which has the secret.
The following code uses attributes about a secret to display basic information about all the secrets.
secrets = VaultsClient(config).list_secrets(compartment_id)
for secret in secrets.data:
print("Name: {}\nLifecycle State: {}\nOCID: {}\n---".format(
secret.secret_name, secret.lifecycle_state,secret.id))
Name: secret_foo
Lifecycle State: ACTIVE
OCID: ocid1.vaultsecret..<unique_id>
---
Name: secret_bar
Lifecycle State: ACTIVE
OCID: ocid1.vaultsecret..<unique_id>
---
</unique_id></unique_id>
List Keys
The list_keys method of the KmsManagementClient object provide access returns a list of keys in a specific vault. It returns a Response object and the data attribute in that object is a list of KeySummary objects.
The KeySummary class has the following attributes:
◉ compartment_id: OCID of the compartment that the key belongs to.
◉ defined_tags: Oracle defined tags.
◉ display_name: Name of the key.
◉ freeform_tags: User-defined tags.
◉ id: OCID of the key.
◉ lifecycle_state: The lifecycle state such as ENABLED.
◉ time_created: Timestamp of when the key was created.
◉ vault_id: OCID of the vault that holds the key.
Note, the KeySummary object does not contain the AES key. When a encrypted secret is returned, it will automatically be decrypted. The common use cases, for a data scientist, is to list keys to get the OCID of a desired key but not to interact directly with the key.
The following code uses some of the above attributes to provide details on the keys in a given vault.
# Get a list of keys and print some information about each one
key_list = KmsManagementClient(config, service_endpoint=vault.management_endpoint).list_keys(
compartment_id=compartment_id).data
for key in key_list:
print("Name: {}\nLifecycle State: {}\nOCID: {}\n---".format(
key.display_name, key.lifecycle_state,key.id))
Name: key_foo
Lifecycle State: ENABLED
OCID: ocid1.key..<unique_id>
---
</unique_id>
List Vaults
The list_vaults method of the KmsVaultClient object returns a list of all the vaults in a specific compartment. It returns a Response object and the data attribute in that object is a list of VaultSummary objects.
The VaultSummary class has the following attributes:
◉ compartment_id: OCID of the compartment that the key belongs to.
◉ crypto_endpoint: The end-point for encryption and decryption.
◉ defined_tags: Oracle defined tags.
◉ display_name: Name of the key.
◉ freeform_tags: User-defined tags.
◉ id: OCID of the vault.
◉ lifecycle_state: The lifecycle state, such as ACTIVE.
◉ time_created: Timestamp of when the key was created.
◉ management_endpoint: Endpoint for managing the vault.
◉ vault_type: The oci.key_management.models.Vault type. For example, DEFAULT.
The following code uses some of the above attributes to provide details on the vaults in a given compartment.
# Get a list of vaults and print some information about each one.
vault_list = KmsVaultClient(config).list_vaults(compartment_id=compartment_id).data
for vault_key in vault_list:
print("Name: {}\nLifecycle State: {}\nOCID: {}\n---".format(
vault_key.display_name, vault_key.lifecycle_state,vault_key.id))
Name: vault_foo
Lifecycle State: ACTIVE
OCID: ocid1.vault..<unique_id>
---
Name: vault_bar
Lifecycle State: DELETED
OCID: ocid1.vault..<unique_id>
</unique_id></unique_id>
Deletion Resources
Vaults, keys, and secrets cannot be deleted immediately. They are marked as pending deletion. By default, they are deleted 30 days after they request for deletion. The length of time before deletion is configurable.
Delete a Secret
The schedule_secret_deletion method of the VaultsClient class is used to delete a secret. It requires the secret’s OCID and a ScheduleSecretDeletionDetails object. The ScheduleSecretDeletionDetails provides details about when the secret is deleted.
The schedule_secret_deletion method returns a Response object that has information about the deletion process. If the key has already been marked for deletion, a ServiceError occurs with information about the key.
VaultsClient(config).schedule_secret_deletion(secret_id, ScheduleSecretDeletionDetails())
Delete a Key
The schedule_key_deletion method of the KmsManagementClient class is used to delete a key. It requires the key’s OCID and a ScheduleKeyDeletionDetails object. The ScheduleKeyDeletionDetails provides details about when the key is deleted.
The schedule_key_deletion method returns a Response object that has information about the deletion process. If the key has already been marked for deletion, a ServiceError occurs.
Note that secrets are encrypted with a key. If that key is deleted, then the secret cannot be decrypted.
KmsManagementClient(config, service_endpoint=vault.management_endpoint)\
.schedule_key_deletion(key_id, ScheduleKeyDeletionDetails())
Delete a Vault
The schedule_vault_deletion method of the KmsVaultClient class is used to delete a vault. It requires the vault’s OCID and a ScheduleVaultDeletionDetails object. The ScheduleVaultDeletionDetails provides details about when the vault is deleted.
The schedule_vault_deletion method returns a Response object that has information about the deletion process. If the vault has already been marked for deletion, then a ServiceError occurs.
Note that keys and secrets are associated with vaults. If a vault is deleted, then all the keys and secrets in that vault are deleted.
KmsVaultClient(config).schedule_vault_deletion(vault_id, ScheduleVaultDeletionDetails())
Source: oracle.com
Posts
0 comments:
Post a Comment