MAA with NetApp Filer & Oracle 11g

This article gives an overview on how to implement maximum availability architecture for an Oracle environment comprising E-Business Suite (R12), Service Oriented Architecture (SOA 10g), Oracle Business Intelligence Enterprise Edition (OBIEE) and Discoverer 10g. This architecture has been successfully implemented for an automotive leasing company in UK.

In the below architecture, the primary site has 3 nodes Real Application Cluster (RAC) for databases while the middle tier is clustered with Oracle technologies. The SOA, OBIEE and Discoverer have the middle tier configured with two nodes cluster while the E-Business Suite middle tier is configured with 3 nodes cluster. The RAC databases are configured on the NetApp volumes where the NFS file systems are used for shared and non-shared files. The Oracle Homes and binaries are deployed on the NetApp non-shared volumes while the database files are configured on the NetApp shared volumes. With 3 nodes on database tier and application tier this architecture provides redundancy, stability, high availability, load balancing and better performance on the primary site. NetApp snapshots are used for backup. This completes the backup process in a few seconds.

The IT team started planning an alternate disaster recovery (DR) site. Several technologies like Oracle Data Guard, Oracle Application Server Guard, SnapMirror, physical copy etc were discussed before the team narrowed down a DR solution that was a combination of SnapMirror and Data Guard.

For the database tier, Data Guard was implemented to replicate the database on the DR site for E-Business Suite, SOA and OID databases. For the middle tier E-Business Suite, SOA, OBIEE and Discoverer - NetApp SnapMirror was implemented. The following section describes the process that was followed to setup the Data Guard and the SnapMirror.

Once the Data Guard is configured for a database, the Data Guard broker has to be configured and interfaced with OEM grid control. The Data Guard broker will simplify the DBA effort in daily monitoring and maintenance.

The Data Guard broker has to be configures in both the primary and in the DR environment. Please note that the metadata for the broker should be available in a shared disk that is accessible by all the instances in the RAC environment

SnapMirror Configuration
For the middle tier application binaries we use SnapMirror technology to replicate the binaries from the primary environment to the DR environment. SnapMirror will mirror the volumes of the primary storage array to the DR storage array. For SnapMirror the complete volume will be mirrored. It is not possible to mirror selected files or folders to snap mirror. Hence all the application binaries should be deployed on NFS volumes. In the present architecture all the application binaries were deployed on the NFS volumes. The very first time, SnapMirror takes a bit of time but the subsequent mirroring will be faster since only the changed blocks are copied from the primary volumes to DR volumes. For a typical SnapMirror operation, the mirroring is scheduled every 15 minutes. This will minimize the loss of data during disaster to 15 minutes.

Mirror volumes are read only volumes. To enable it as read-write volume you need to break the mirror. For a failover/switchover scenario, the Administrator will break the mirror. As the volume becomes read-write mode, we can configure application tier. As the primary site is ready, we can re-enable SnapMirror. For a volume that is subject to SnapMirror will have state as ‘snap mirrored’. For any volume with SnapMirror option will have following ‘Status’ and ‘State’.

Status:
Pending – Subject to SnapMirror
Transferring – SnapMirror in progress
Idle – SnapMirror is completed
State:
Snap mirrored –SnapMirror operation is pending, transferring or idle
Broken-Off – SnapMirror operation is manually stopped

Performing Switchover/Failover
As mentioned in above section, during the switchover/failover you can break snap mirror for the application tier. While for the database you need to perform the Data Guard switchover/failover operation.

The switchover operation with Data Guard broker is as given below.

• Stop Application Services in Primary Environment
• Connect to DG Broker command line on Primary Environment
• Show Configuration/Database/Instance. The value to return for all the command should be ‘SUCCESS’. If any error, solve it then only proceed for SwitchOver.
• To modify any DG Broker parameter. For e.g.
• The above given to be executed from both the primary and DR sites
• Create a restore point in primary and standby databases
• Execute Switchover command from DG Broker
• If the switchover to perform for E-business Suite database, perform the following
• Clean FND Concurrent Nodes
• Execute autoconfig in new primary environment (On all cluster nodes)
• Changing the Custom Concurrent Managers

If the switchover to perform for E-business Suite database, perform the following If the switchover to perform for E-business Suite database, perform the following

• Clean FND Concurrent Nodes
• Execute autoconfig in new primary environment (On all cluster nodes)
• Changing the Custom Concurrent Managers
• Reconfigure workflow mailer, ICSM, fulfilment, printers

Conclusion
This article hopes to provide a better understanding on how to implement maximum availability architecture with NetApp and Oracle technologies.