Why move Oracle Database Backups to OCI?
The OSB Cloud Module for AWS S3 does not support Glacier (archive storage), your backups can only be stored in S3 with the associated costs, which can become very high, especially over the long term.
The Database Backup Service in OCI does support Object Lifecycle Policies. Backups can be stored in standard Object Storage for a defined period of time for quick restore, and then automatically tiered to Archive Storage for long-term, low-cost retention. RMAN automatically manages recalling and restoring backups from Archive Storage as needed.
The Backup Service in OCI does not require per-stream licenses; they are required with the OSB Cloud Module for AWS S3. The Database Backup Cloud Module for OCI is included as part of the service and can leverave as many streams as necessary to achieve your performance goals, across any number of databases.
Because the OSB Cloud Module for AWS S3 and the DB Backup Cloud Module for OCI use the same format when storing backup data in cloud buckets, migrating backups from S3 to OCI is a straightforward process as shown below.
How RMAN backup sets map to OCI objects
RMAN uploads backup sets to cloud buckets as a number of backup pieces, and each backup piece maps to one or more objects in the cloud bucket – these associations are recorded in the metadata.xml manifest file. The number of objects depends on the size of the backup piece. By default, a backup piece up to 10GB in size is stored in OCI as a single object, larger backup pieces will use multiple objects.
The backup piece name is controlled by the FORMAT parameter in RMAN. The recommendation is to include at least ‘%d_%U’ to make sure each object name is unique.
%d -> DB_NAME
%U -> system generated unique identifier
For example: ORCL_ctua720h_1_1 is a backup piece handle created using the ‘%d_%U’ format.
The objects in the cloud bucket for such a backup piece will look like this.
sbt_catalog/ORCL_ctua720h_1_1/metadata.xml (metadata manifest file)
file_chunk/<DBID>/<DBNAME>/backuppiece/<DATE>/ORCL_ctua720h_1_1/<INCARNATION>/<CHUNK#>
file_chunk/<DBID>/<DBNAME>/backuppiece/<DATE>/ORCL_ctua720h_1_1/<INCARNATION>/metadata.xml (copy of metadata.xml file)
1. Copying data to OCI
All the objects pertaining to a specific backup piece must be copied for RMAN to be able to restore it. The chunks where the actual backup data are and the two metadata.xml files.
To copy objects from the AWS S3 bucket to OCI you can use rclone available via http://rclone.org.
It can be installed and used on any system that has access to both the AWS S3 bucket and the OCI Object Storage bucket.
First of all, you need to obtain:
◉ S3 ACCESS KEY and SECRET ACCESS KEY for the AWS S3 user
◉ If you don’t have them, go to the AWS console ->IAM->Users->Security Credentials and click on Create Keys
◉ S3 REGION where the bucket is located
◉ S3 BUCKET name for the bucket where the backups are located
◉ OCI STORAGE NAMESPACE
◉ Go to Profile on the top right corner of the OCI console and select Tenancy:<your tenancy name> in the Tenacy Information section. Under Object Storage Settings, you will see the Object Storage Namespace. The namespace value can be either the tenacy name or a randomly generated string. The name cannot be changed.
◉ OCI Amazon S3 Compatibility API Designated Compartment
◉ In the same Object Storage Settings of the Tenancy Information section you will also find the compartment where your OCI destination bucket must be located. If you want to change the designated Compartment click on Edit Object Storage Settings at the top of the page.
◉ OCI CUSTOMER ACCESS KEY and SECRET KEY
◉ If you don’t have them, go to Profile on the top right corner of the OCI console and select User Settings, then on the bottom left of the screen, under Resources click on Customer Secret Keys and then click on Generate Secret Key. Make a note of the Secret Key as it’s displayed only once.
◉ OCI REGION where you want your target bucket to be located
Before starting the copy with rclone, we have to set some environment variables to define the source and target locations based on the information above:
# export RCLONE_CONFIG_S3_TYPE=s3
# export RCLONE_CONFIG_S3_ACCESS_KEY_ID=AKIRGGSJRV23S5AG4N
# export
RCLONE_CONFIG_S3_SECRET_ACCESS_KEY=TLJkltRDASlSlhVRPsRuJse2FtWLnFD5
# export RCLONE_CONFIG_S3_REGION=<region> for example: us-east-1
# export SOURCE=s3:<bucket name>
# export RCLONE_CONFIG_OCI_TYPE=s3 rclone uses S3 compatibility APIs to access OCI Object Storge
# export RCLONE_CONFIG_OCI_ACCESS_KEY_ID=b8d65742ca7385eac091f1c0e86376d1e30eb4
# export RCLONE_CONFIG_OCI_SECRET_ACCESS_KEY=26TtH1CSSFgddsEPwDoBqPCsLVrapmerolAsDg=
# export RCLONE_CONFIG_OCI_REGION=<region> for example: us-ashburn-1
# export RCLONE_CONFIG_OCI_ENDPOINT=https://<namespace>.compat.objectstorage.<region>.oraclecloud.com
Once everything is set up you can start the rclone copy process, all the content of the AWS S3 bucket will be copied to the OCI target. If the bucket does not exist it is created.
# rclone --verbose --cache-workers 64 --transfers 64 --retries 32 copy $SOURCE oci:<bucketname>
2. Installing the DB Backup Cloud Module
Once the copy is complete, you will find all the backup objects in the bucket you designated. In order to be able to restore from OCI you have to install the DB Backup Cloud Module and configure it to point to the new OCI bucket.
You also have the option to set a lifecycle policy on the bucket in order to move the backups to archive storage after a period of time that can be between 0 days (data will be archived within 24 hours) and number of years. This can be done using –enableArchiving, –archiveAfterBackup, and –retainAfterRestore parameters.
Note: Do not create the policy manually using the OCI console. Instead, use the appropriate archiving options mentioned above for the DB Backup Cloud Module installer so that the appropriate exclusion for .xml files is also added.
The DB Backup Cloud Module for OCI can co-exist with the OSB Cloud Module for AWS S3 if you want to be able to access both buckets for a period of time.
RMAN can now be configured to use the newly installed module, referencing the new libopc.so SBT library and using the new configuration file created by the installer.
The backups in the OCI bucket are now accessible and an RMAN restore validate database command can be executed to verify the process was successful.
3. Restoring from archive storage
Since data residing in the archive storage tier cannot be immediately accessed by RMAN clients, it must first be restored (or recalled, in RMAN terminology) to standard object storage. This process can take 1 hour or more depending on the volume of data. The objects go from an "Archived" state to "Restoring" and then "Restored". When an object is "Restored" it means it's in standard object storage and RMAN can access it.
RMAN can initiate this recall process through the ‘restore database preview recall’ command.
Allternatively, the RMAN restore database preview command can be used to just determine if the backup pieces required for a database restore operation are immediately available in standard object storage, or if recalling backups from archive storage tier is required. In this example, an RMAN restore database preview command output shows that a certain backup piece is archived and needs to be recalled.
List of Backup Sets
===================
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
179 Full 256.00K SBT_TAPE 00:00:02 14-SEP-19
BP Key: 179 Status: AVAILABLE Compressed: NO Tag: TAG20190914T100406
Handle: 89ubntom_1_1 Media: objectstorage.us-ashburn-..ecloud.com/n/oradbclouducm/tdemoaug22
List of Datafiles in backup set 179
File LV Type Ckp SCN Ckp Time Abs Fuz SCN Sparse Name
---- -- ---- ---------- --------- ----------- ------ ----
21 Full 4715399 14-SEP-19 NO /ade/b/3380669573/oracle/dbs/tbs_25.f
using channel ORA_SBT_TAPE_1
using channel ORA_DISK_1
archived logs generated after SCN 4715399 not found in repository
recovery will be done up to SCN 4715399
Media recovery start SCN is 4715399
Recovery must be done beyond SCN 4715399 to clear datafile fuzziness
List of remote backup files
============================
Handle: 89ubntom_1_1 Media: objectstorage.us-ashburn-..ecloud.com/n/oradbclouducm/tdemoaug22
validation succeeded for backup piece
Finished restore at 14-SEP-19
If one or more files are listed as “remote” you can either issue a RMAN restore database preview recall command or just RMAN restore database command. While the restore database preview recall command will initiate the recall from archive storage and exit, the restore database command will also initiate the recall from archive process and then wait for it to be completed before proceeding with the actual database restore.
The following example shows the output of a restore database preview recall command:
List of Backup Sets
===================
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
179 Full 256.00K SBT_TAPE 00:00:02 14-SEP-19
BP Key: 179 Status: AVAILABLE Compressed: NO Tag: TAG20190914T100406
Handle: 89ubntom_1_1 Media: objectstorage.us-ashburn-..ecloud.com/n/oradbclouducm/tdemoaug22
List of Datafiles in backup set 179
File LV Type Ckp SCN Ckp Time Abs Fuz SCN Sparse Name
---- -- ---- ---------- --------- ----------- ------ ----
21 Full 4715399 14-SEP-19 NO /ade/b/3380669573/oracle/dbs/tbs_25.f
using channel ORA_SBT_TAPE_1
using channel ORA_DISK_1
archived logs generated after SCN 4715399 not found in repository
recovery will be done up to SCN 4715399
Media recovery start SCN is 4715399
Recovery must be done beyond SCN 4715399 to clear datafile fuzziness
Initiated recall for the following list of remote backup files
==========================================================
Handle: 89ubntom_1_1 Media: objectstorage.us-ashburn ..ecloud.com/n/oradbclouducm/tdemoaug22
validation succeeded for backup piece
Finished restore at 14-SEP-19
In this blog we showed how to move you backups taken using the Oracle Secure Backup Cloud Module for AWS S3 from an S3 bucket to an OCI Object Storage bucket in order to be restored using the Oracle Database Backup Cloud Module. These backups could be coming from a database running in an AWS EC2 compute instance or from and on-premises database and can be restored back to on-premises or to an OCI instance (Compute, DBaaS or ExaCS). For restoring into DBaaS or ExaCS some limitations apply on versions, DB architecture etc.
Source: oracle.com
Posts
0 comments:
Post a Comment