BLOG DBA TIPS AND TRICKS

This document describes how to install Oracle RAC cluster 12cR1 on Redhat. We’ll install the grid, configure the ASM diskgroups and then install the cluster database.

Below the summary of study case :

2.Create oracle user and softwares directories

User ‘oracle’ will be used as owner of grid and database binaries. As root user, create oracle user and groups. Change also oracle password like below:

# groupadd -g 502 asmadmin
# groupadd -g 503 dba
#useradd -u 502 -g dba -G asmadmin -d /home/oracle -c “Oracle user” oracle
#chown oracle:dba /u01
# passwd oracle
#su – oracle –> to connect as oracle user
$mkdir -p /u01/app/grid
$mkdir -p /u01/app/12.1

3.Configure system requirements

Edit file /etc/sysctl.conf and add/modify parameters like below:
#—- Params for oracle ——–
net.ipv4.ip_local_port_range = 9000 65500
kernel.shmmax = 4398046511104
kernel.shmall = 1073741824
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 3145728
kernel.panic_on_oops = 1
fs.file-max = 6815744

Edit file /etc/security/limits.conf
#— Oracle params ———
oracle soft nproc 16384
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
oracle hard stack 32768

Apply modifications with command below:
# /sbin/sysctl –p

4.Disable NTP server

As root user, execute :
# /etc/init.d/ntpd status
ntpd (pid 5032) is running…
# service ntpd stop
Shutting down ntpd: [ OK ]
# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig
# rm /var/run/ntpd.pid
rm: cannot remove `/var/run/ntpd.pid’: No such file or directory
#

5.Configure /etc/hosts files:

Edit and configure file /etc/hosts on both nodes as root user :
# Private Interconnect – (eth2)
192.168.102.1 svr-lcmsdb-01-priv.mtn.ci svr-lcmsdb-01-priv
192.168.102.2 svr-lcmsdb-02-priv.mtn.ci svr-lcmsdb-02-priv

# Public Network – (bond0)
192.168.0.135 svr-lcmsdb-01.mtn.ci svr-lcmsdb-01
192.168.0.136 svr-lcmsdb-02.mtn.ci svr-lcmsdb-02

# Public Virtual IP (VIP) addresses – (eth)
192.168.0.139 svr-lcmsdb-01-vip.mtn.ci svr-lcmsdb-01-vip
192.168.0.140 svr-lcmsdb-02-vip.mtn.ci svr-lcmsdb-02-vip

6.Configure SCAN:

Choose a scan name and add 3 public ip adressesses for SCAN. Add 3 entries in DNS with scan name and the ips: each entry with the scan name and one of the 3 ips

Edit file /etc/resolv.conf and add the domain and the ips of the different DNS used in the network
# cat /etc/resolv.conf
search mydomain.ci
nameserver 192.168.0.138
nameserver 192.168.0.139

Check scan name resolution like below:

# nslookup LCMS-cluster-scan
Server: 192.168.0.138
Address: 192.168.0.138#53

Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.143
Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.142
Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.141

# nslookup LCMS-cluster-scan
Server: 192.168.0.138
Address: 192.168.0.138#53

Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.142
Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.141
Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.143

# nslookup LCMS-cluster-scan
Server: 192.168.0.138
Address: 192.168.0.138#53

Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.141
Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.143
Name: LCMS-cluster-scan.mydomain.ci
Address: 192.168.0.142

7.Packages installation

Use the yum package manager to install the packages and any of their dependencies. Create a file called pkg.txt containing all the packages to be installed. As root user, use the following command to install the packages:
# yum install `awk ‘{print $1}’ ./pkg.txt`
—–Content of pkg.txt file:
cloog-ppl
compat-libcap1
compat-libstdc++-33
cpp
gcc
gcc-c++
glibc-devel
glibc-headers
kernel-headers
ksh
libXmu
libXt
libXv
libXxf86dga
libXxf86misc
libXxf86vm
libaio-devel
libdmx
libstdc++-devel
mpfr
make
ppl
xorg-x11-utils
xorg-x11-xauth
oracleasm
unixODBC
unixODBC-devel
compat-libstdc++-33.i686
glibc-devel.i686
libstdc++.i686
libstdc++-devel.i686
libX11.i686
libXext.i686
libXi.i686
libXtst.i686
libaio.i686
libaio-devel.i686

Install package cvuqdisk*

cd /home/oracle/software/grid/rpm
# CVUQDISK_GRP=dba; export CVUQDISK_GRP
#rpm -Uvh cvuqdisk*

Install package oracleasm-support-2.1.8-1.el6.x86_64.rpm and oracleasmlib-2.0.4-1.el6.x86_64.rpm for asm

# rpm -i oracleasm-support-2.1.8-1.el6.x86_64.rpm
warning: oracleasm-support-2.1.8-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY

# rpm -i oracleasmlib-2.0.4-1.el6.x86_64.rpm
warning: oracleasmlib-2.0.4-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY

Check oracleasm successfully installed :
# which oracleasm
/usr/sbin/oracleasm

8.Exchange keys between servers

In order to allow oracle user to connect from one node to another without using password, we need to exchange keys between servers

[oracle@svr-lcmsdb-01 ]$cd /home/oracle/.ssh/
[oracle@svr-lcmsdb-01 .ssh]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
56:21:06:60:bb:3c:fa:32:82:ef:a3:63:ae:26:4c:ec oracle@svr-lcmsdb-01.mydomain.ci
The key’s randomart image is:
+–[ DSA 1024]—-+
| o…o . |
| . . . . . |
| . . |
| . . . |
|. + S |
| o . . . |
|= . |
|=E+. |
|OBo+. |
+—————–+

The public key should be added in the file called authorized_keys in the second node. The file should be created if not existing. Here the file doesn’t exist on second node. So we create it as below:
[oracle@svr-lcmsdb-01 .ssh]$ scp id_dsa.pub oracle@192.168.136:/home/oracle/.ssh/authorized_keys

Do the same on node 2: generate the public key and copy it in the file called authorized_keys on node 1:
[oracle@svr-lcmsdb-02 .ssh]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
7d:d4:69:ab:f0:f7:82:86:0c:92:df:9c:65:cc:a6:73 oracle@svr-lcmsdb-02.mydomain.ci
The key’s randomart image is:
+–[ DSA 1024]—-+
| |
| . . |
| . + |
| . . . . |
| .S oo. . |
| o . +*. |
| o = Bo.. |
| . O E… |
| + ..|
+—————–+

[oracle@svr-lcmsdb-02 .ssh]$ ls
id_dsa id_dsa.pub
$scp id_dsa.pub oracle@192.168.0.135:/home/oracle/.ssh/authorized_keys


9.Configuring ASM driver servers

Configure ASM driver like below:
[root@svr-lcmsdb-01]# /usr/sbin/oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets (‘[]’). Hitting without typing an answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
[root@svr-lcmsdb-01 mapper]#
# vi /etc/sysconfig/oracleasm
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER=”dm“
……
# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan ORACLEASM_SCANEXCLUDE=”sda“
# /etc/init.d/oracleasm enable
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]


10.Configuring disks for ASM

Check disks provided to be configured. Here disks are in /dev :

[root@svr-lcmsdb-02 oracle]# ls -ltrh /dev/dm*
brw-rw—- 1 root disk 253, 0 Nov 19 01:44 /dev/dm-0
brw-rw—- 1 root disk 253, 2 Nov 19 01:44 /dev/dm-2
……
brw-rw—- 1 root disk 253, 20 Nov 19 01:44 /dev/dm-20
brw-rw—- 1 root disk 253, 19 Nov 19 01:44 /dev/dm-19
[root@svr-lcmsdb-02 oracle]#

Use oracleasm to configure disks :
# /usr/sbin/oracleasm createdisk DATA1 /dev/dm-3
Writing disk header: done
Instantiating disk: done
# /usr/sbin/oracleasm createdisk DATA2 /dev/dm-4
Writing disk header: done
Instantiating disk: done
…… # /usr/sbin/oracleasm createdisk DATA15 /dev/dm-17
Writing disk header: done
Instantiating disk: done
# /usr/sbin/oracleasm createdisk DATA16 /dev/dm-18
Writing disk header: done
Instantiating disk: done

List all disks configured:
# /usr/sbin/oracleasm listdisks
DATA1
DATA10
DATA11
DATA12
DATA13
DATA14
DATA15
DATA16
DATA2
DATA3
DATA4
DATA5
DATA6
DATA7
DATA8
DATA9
#
NB : On the other node, perform a scandisk to update disks table:
# oracleasm scandisks

11.Installing Grid Infrastructure

Unzip grid software by extracting files p17694377_121020_Linux-x86-64_3of8.zip and p17694377_121020_Linux-x86-64_4of8.zip

[oracle@svr-lcmsdb-01 software]$ unzip p17694377_121020_Linux-x86-64_3of8.zip
[oracle@svr-lcmsdb-01 software]$ unzip p17694377_121020_Linux-x86-64_4of8.zip

[oracle@svr-lcmsdb-01 software]$cd grid
[oracle@svr-lcmsdb-01 grid]$ ls
install readme.html response rpm runcluvfy.sh runInstaller sshsetup stage welcome.html
[oracle@svr-lcmsdb-01 grid]$ id -a
uid=502(oracle) gid=503(dba) groups=503(dba),502(asmadmin)
[oracle@svr-lcmsdb-01 grid]$ export DISPLAY=192.xx.xx.xx:0.0
[oracle@svr-lcmsdb-01 grid]$./runInstaller
Starting Oracle Universal Installer…

Checking Temp space: must be greater than 415 MB. Actual 18504 MB Passed
Checking swap space: must be greater than 150 MB. Actual 63999 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2015-11-27_11-35-14AM. Please wait …

Choose first option to install grid for a cluster and Next

Choose “Configure a standalone cluster” and Next

Choose “Advanced installation” and Next

Choose the language and Next

Enter scan informations and Next

Node 1 is added in the cluster. Use this screen to add all nodes to be added in the cluster. Click Add to add seconde node

Enter informations concerning the node 2, adding the .mydomain.ci at the end of host or vip name. Click OK

Check node 2 added to the cluster. Click Next to continue

Set up the ssh connectivity by clicking “SSH connectivity…” and providing user oracle password . Click Next to continue

When connectivity is OK, you get the screen below . Click OK to continue

You can click on “Test” button to check again connectivity with parameters set . Click Next to continue

System checks for installer validation…

You should define which network interface should be used as private interconnect between nodes and which should be used for public interface. Click Next once done

Choose “Use Standard ASM for storage”. Click Next once done

Before continuing, you’ll need to create an ASM diskgroup. This will be used to store cluster configuration.
Here we create a diskgroup called “DATA”. Be careful to choose the correct redundancy mode. We chose “External” to be sure to use all disks size for the db without mirroring in ASM. We’ll rely on the storage mirroring
To list all disks configured with oracleasm, specify “/dev/oracleasm/disks/*” in the discovery path.
We keep also default “Allocation Unit Size” to 1MB, appropriate for transactional databases

Check all disks candidates to be added in diskgroup “DATA”. You can later user asmca GUI tool to resize the diskgroup

Specify the password for user sys to connect as SYSDBA for ASM administration. Oracle doesn’t force but recommends to use a strong password

Select “Do not use IPMI” and Next

Do not check the box if you not ready to configure the db to be monitored by Entreprise Manager and click Next to continue

Choose the ASM group for user authentication and Next

Specify the oracle base and grid infra home which should not be under the oracle base and click Next

Specify the oracle inventory for installation log directory and click Next

Check the box and specify the root password to allow installation to automatically connect on the other node(s) and run some scripts as root. Otherwise, the installation will ask you to connect to the other node(s) and run some scripts as root user.
NB: You should allow remote connexion with root user between nodes during the installation.
Click Next to continue

Installation checks for requirements …

You can have to fix some errors or warnings. Installation will show you nodes concerned and provide sometimes scripts to run to fixe these errors.
Correct the errors and click Next to continue

You’ll get a summary like this one. Click install to proceed with grid installation

When the installation completes, you get the screen below :

You can check ASM instance is running by using command below:
[oracle@svr-lcmsdb-01 12.1]$ ps -ef|grep pmon
oracle 8440 7912 0 22:40 pts/1 00:00:00 grep pmon
oracle 20211 1 0 21:50 ? 00:00:00 mdb_pmon_-MGMTDB
oracle 27035 1 0 20:31 ? 00:00:00 asm_pmon_+ASM1
[oracle@svr-lcmsdb-01 12.1]$

We can continue with oracle software and database installation.

12.Installing Oracle Database Software

Unzip database software by extracting files p17694377_121020_Linux-x86-64_1of8.zip and p17694377_121020_Linux-x86-64_2of8.zip and run the runInstaller

Uncheck the box and click Next

Click OK and Next

Choose “Install database software only”. We’ll install database after. Click OK and Next

Choose option “Oracle Real Application …” to install software on the cluster. Click Next

Check the nodes to be added in the cluster. Click “SSH connectivity…”

System check for SSH connectivity …

When connectivity is established, you get below message

Choose software language(s)

Select “Enterprise Edition” and Next

Specify Oracle base and Software location which will be considered as ORACLE HOME. Click Next

Choose the oracle user group to connect as SYSDBA. Click Next

System checks for requirements. You have to fix errors before continuing but you can choose to check ignore all to ignore warnings.

If you choose to fix errors, the system will show you nodes concerned. Check recommendations to fix issues.

System checks again if error or warning is fixed.

You can choose to ignore all warnings by checking “ignore all” box and confirm.

Installation shows a summary of options chose.

Click Install to begin installation

At the end of the installation, the system prompts to connect to each node and run specified script as root user before returning to this screen to click OK

Open a shell for each node and run the root.sh script in the $ORACLE_HOME/bin directory like below:

When you return to the screen and click OK, you get below screen which confirms installation completed

13.Create diskgroups in ASM
Complete the configuration of ASM diskgroups by running asmca in $GRID_HOME/bin directory. Via the interface, you can create or alter diskgroups by adding/removing disks

Create relevant diskgroups within disks like below

14.Create RAC database

After Grid infra and ASM diskgroups configuration, we can proceed now with the database installation.
Run dbca at the command line to launch database installation tool

Choose “Create Database” and click Next

You can choose to create with “default configuration” or “Advanced Mode”. Here we chose “default configuration”. Specify required parameters like below and click Next

System check for installation requirements…

Check prerequisites to fix errors and/or warnings. Click Next if no error like below

Check installation summary. Click Finish to start installation

Installation in progress…

Installation completed

Use command like below to check database configuration:

$ srvctl config database -d lcmsdb
Database unique name: LCMSDB
Database name: LCMSDB
Oracle home: /u01/app/oracle/product/12.1.0/dbhome_1
Oracle user: oracle
Spfile: +DATA01/LCMSDB/PARAMETERFILE/spfile.288.896968523
Password file: +DATA01/LCMSDB/PASSWORD/pwdlcmsdb.276.896968239
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: FRA,DATA01
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: asmadmin
OSOPER group: asmadmin
Database instances: LCMSDB1,LCMSDB2
Configured nodes: svr-lcmsdb-01,svr-lcmsdb-02
Database is administrator managed
$

This completes the Grid Infrastructure and RAC database installation!

Share this:

• X
• Facebook

Like Loading…