LINUX System Prep for Oracle 12.1.0.2

<- Legacy

TOC

• Create Groups and Users
• Create Directories
• OS Configuration Changes: Firewall | Hostname | Kernel | ntp | Packages | SELINUX
• Configure Disks
• Storage: Configure ASMFD | Configure UDev
• Cluster Verification Script
• APPENDIX

Overview

What follows are all the common steps to prep a Linux 7.x system (RHEL, CentOS, Oracle Linux) to use Oracle 12c Grid Infrastructure or Database. I wrote this script to automate all this.

Prerequisites

• You have downloaded required install files:
  • Oracle Database 12c Database for Linux x86-64.
  • Oracle Database 12c Infrastructure for Linux x86-64 (if RAC).
• If RAC, all systems must be running the exact same Linux kernel version.
• RAM 4gb (test) or more. 64 gb+ (production).
• Required disks provided by your SA so they can be seen by the LINUX OS.

/      = OS and SWAP        (75 gb+)
/u01   = Oracle Data        (size as needed for your database)  
/u02   = FRA, RMAN, Exports (size as needed for your database)

Create Groups and Users

The following is derived from the Oracle docs. Perform as the root user unless indicated otherwise.

Oracle recommends that you create one software owner to own each Oracle software product:

• oracle for the database software owner user and
• grid for Oracle Grid Infrastructure.

Groups
 groupadd --gid 54321 oinstall
 groupadd --gid 54322 dba
 groupadd --gid 54323 asmdba
 groupadd --gid 54324 asmoper
 groupadd --gid 54325 asmadmin
 groupadd --gid 54326 oper
 groupadd --gid 54327 backupdba
 groupadd --gid 54328 dgdba
 groupadd --gid 54329 kmdba

oracle User
 useradd --uid 54321 --gid oinstall --groups dba,oper,asmdba,asmoper,backupdba,dgdba,kmdba oracle
 passwd oracle

 If oracle user already exists:
   usermod -a -G dba,oper,asmdba,asmoper,backupdba,dgdba,kmdba oracle
   usermod -g oinstall oracle

grid User
 useradd --uid 54322 --gid oinstall --groups dba,asmadmin,asmdba,asmoper grid
 passwd grid

Create Directories

• Perform as the root user unless indicated otherwise.
• mkdir -p option makes parent directories as needed.

Create Directories for Oracle Software

1. Create software directory.
   mkdir -p /u01/orasw/rpm
   chown -R oracle:oinstall /u01
   chmod -R 775 /u01
2. Extract files.
   a. Copy installation files to new orasw directory.
   b. Set permissions
      chown -R oracle:oinstall /u01/orasw
   c. Unzip Oracle Database Files
      su - oracle 
      cd /u01/orasw
      unzip linuxamd64_12102_database_1of2.zip
      unzip linuxamd64_12102_database_2of2.zip
      A /u01/orasw/database subdirectory will be created.
   d. Unzip grid files.
      su - grid
      cd /u01/orasw
      unzip linuxamd64_12102_grid_1of2.zip
      unzip linuxamd64_12102_grid_2of2.zip
      A /u01/orasw/grid subdirectory will be created.

Create Directories for Oracle Product

Software

mkdir -p /u01/app/oraInventory
mkdir -p /u01/orasw/rpm
mkdir -p /u01/orasw/staging
chown -R grid:oinstall /u01/app/oraInventory
chown -R oracle:oinstall /u01/orasw
chown -R oracle:oinstall /u01/orasw/staging

Base Dirs

mkdir -p /u01/app/grid
mkdir -p /u01/app/oracle
mkdir -p /u01/app/cfgtoollogs
mkdir -p /u01/app/oraInventory
chown oracle:oinstall /u01/app
chown oracle:oinstall /u01/app/cfgtoollogs
chown oracle:oinstall /u01/app/oracle
chown grid:oinstall /u01/app/grid
chown grid:oinstall /u01/app/oraInventory

RDBMS HOME

mkdir -p /u01/app/oracle/product/$RDBS_VERSION/db_1
chown -R oracle:oinstall /u01/app/oracle/product
chown oracle:oinstall /u01/app/oracle/product/$RDBS_VERSION
chown oracle:oinstall /u01/app/oracle/product/$RDBS_VERSION/db_1

Example RDBS_VERSION="12.1.0.2"

GI HOME

mkdir -p /u01/app/$GRID_VERSION/grid
chown grid:oinstall /u01/app/$GRID_VERSION
chown grid:oinstall /u01/app/$GRID_VERSION/grid
chmod -R 775 /u01/

Example GRID_VERSION="12.1.0.2"

OS Configuration Changes

Set Hostname

1. Replace the old hostname with the desired one if present:
   • vi /etc/hosts
   • vi /etc/sysconfig/network
2. Run: hostnamectl set-hostname <YourHostName>
3. Restart System: shutdown -r now

Confirm:

1. echo $HOSTNAME
2. Run: hostname

Stop Firewall

systemctl disable firewalld
systemctl stop firewalld

service iptables stop
chkconfig iptables off

Edit Host Files: /etc/hosts

127.0.0.1 localhost.localdomain localhost
<ThisSystemsIP> <ThisSystemsFQDN> <ThisSystemsHostName>

Edit tmpfs: /dev/shm <=DEPRECATED

"The use of AMM is absolutely incompatible with [Linux] HugePages" -Oracle. You dont need to be concerned with allocating explicit resources to tmpfs (/dev/shm) when using HugePages and ASMM.
The value of /dev/shm must be larger than you MEMORY_TARGET value. For instance, if you want your database to use 2048m you could set /dev/shm to 2304m

• Short Term: mount -t tmpfs shmfs -o size=2304m /dev/shm
• Permanently: vi /etc/fstab and add this entry:
• shmfs /dev/shm tmpfs size=2304m 0
• Display Current Value: df -h

Edit Kernel Parameters: /etc/sysctl.conf

Add any values to the end if they dont exist.

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 29670075
kernel.shmmax = 67515904000
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
vm.swappiness = 1
vm.dirty_background_ratio = 3
vm.dirty_ratio = 80
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
kernel.panic_on_oops = 1
# HUGEPAGES SETTING FOR ORACLE
vm.nr_hugepages = 39953

Apply changes: /sbin/sysctl -p

Disable Reverse Path Filtering Change eth0 and eth1 with your private interface names.

net.ipv4.conf.eth0.rp_filter = 2
net.ipv4.conf.eth1.rp_filter = 2

Edit Limits File: /etc/security/limits.conf

Add any values to the end if they dont exist.

grid          soft    nproc   2047
grid          hard    nproc   16384
grid          soft    nofile  1024
grid          hard    nofile  65536
grid          soft    stack   10240
grid          hard    stack   32768
oracle        soft    nproc   2047
oracle        hard    nproc   16384
oracle        soft    nofile  1024
oracle        hard    nofile  65536
oracle        soft    stack   10240
oracle        hard    stack   32768
oracle        soft    memlock 3145728
oracle        hard    memlock 3145728

Disable SELINUX

Depending on system edit:

• /etc/selinux/config OR
• /etc/sysconfig/selinux

SELINUX=disabled

The modification of the /etc/selinux/config file takes effect after a reboot.

Confirm SELinux Status

/usr/sbin/getenforce
Disabled

Install Dependencies\Packages

• When using yum if a package is already installed it will display Nothing to do.
• If you wish to check if a package exists you can use this:
  • rpm -qa |grep -i <package name>
• If you dont have access to the repositories you can do this check this way:
  • rpm -ivh <package name>

yum install binutils -y
yum install compat-libcap1 -y
yum install compat-libstdc++-33 -y
yum install compat-libstdc++-33.i686 -y
yum install gcc -y
yum install gcc-c++ -y
yum install glibc -y
yum install glibc.i686 -y
yum install glibc-devel -y
yum install glibc-devel.i686 -y
yum install ksh -y
yum install libgcc -y
yum install libgcc.i686 -y
yum install libstdc++ -y
yum install libstdc++.i686 -y
yum install libstdc++-devel -y
yum install libstdc++-devel.i686 -y
yum install libaio -y
yum install libaio.i686 -y
yum install libaio-devel -y
yum install libaio-devel.i686 -y
yum install libXext -y
yum install libXext.i686 -y
yum install libXtst -y
yum install libXtst.i686 -y
yum install libX11 -y
yum install libX11.i686 -y
yum install libXau -y
yum install libXau.i686 -y
yum install libxcb -y
yum install libxcb.i686 -y
yum install libXi -y
yum install libXi.i686 -y
yum install make -y
yum install sysstat -y
yum install unixODBC -y
yum install unixODBC-devel -y

yum install libXmu -y
yum install libXxf86dga -y
yum install nfs-utils -y
yum install libXau -y
yum install libdmx -y
yum install xorg-x11-utils -y
yum install xorg-x11-xauth -y
yum install libXv -y
yum install libXi -y
yum install libXt -y
yum install libXxf86misc -y
yum install LibXxf86vm -y

yum install device-mapper-multipath -y

If running RHEL 7 and Oracle 12.1.02 you will need enable the optional rpms and get this required package: compat-libstdc++-33:

yum-config-manager --enable rhel-7-server-optional-rpms
yum install compat-libstdc++-33 -y

This is said to have been fixed in 12.2.

Edit nproc: /etc/security/limits.d/90-nproc.conf

# Change this
* soft nproc 1024

# To this
* - nproc 16384

Configure NTP

It is critical that the time of all your nodes is kept in sync for an Oracle RAC environment. Swift changes in time can lead to Oracle shutting down a node(s) due to inconsistent timers. Use the following line\values for the /etc/sysconfig/ntpd file.

OPTIONS="-g -x -u ntp:ntp -p /var/run/ntpd.pid"

• -g = panicgate: allows the first adjustment to exceed the panic limit (1000s by default).
• -p = pidfile: name and path of the file used to record ntpd process ID.
• -u = Linux user account to own process.
• -x = slew: make micro time adjustments (rather than in one large adjustment).

-- If CentOS Disable Chrony (the default)

• systemctl disable chronyd.service
• systemctl disable chronyd

-- System Date & Time: View
date
Tue Nov 22 08:46:20 EST 2016

date 112208362016
Tue Nov 22 08:36:00 EST 2016

-- ntpd Service: Installation

1. yum install -y ntp
2. systemctl enable ntpd
3. systemctl start ntpd
4. systemctl status ntpd
5. ntpq -p
   If you don't see a * next to a server your ntpd is not synced to any time source.

-- Sync NTP To Time Source
Add server entry to /etc/ntp.conf with prefer clause.

1. vi /etc/ntp.conf
2. server <NTP_Auth_Server_IP> prefer
3. systemctl restart ntpd
4. ntpq -p
   You should now see a * next to a server.

-- Stop|Disable

• systemctl stop ntpd
• systemctl disable ntpdate

-- Status Check

• ntpq -p
• ntpdc -c loopinfo
• ntpstat
• timedatectl

NTP Troubleshooting

APPENDIX

Cluster Verification Script

Usage: runcluvfy.sh stage -pre crsinst -n <node1,node2> -fixup -verbose
Example: /u01/orasw/grid/runcluvfy.sh stage -pre crsinst -n rac01,rac02 -fixup -verbose

Useful BASH Aliases (place in .bashrc)

Change db1 to your instnace name.

# Aliases - Common
alias cl='crontab -l'
alias l9='ls -alt | head -9'
alias l20='ls -alt | head -20'
alias l50='ls -alt | head -50'
alias tf='date;ls -l|wc -l'

# Aliases - Oracle
alias alog='tail -200 /u01/app/oracle/diag/rdbms/db1/db1/trace/alert_db1.log'
alias aloge='tail -200 /u01/app/oracle/diag/rdbms/db1/db1/trace/alert_db1.log|grep ORA-'
alias cdbin='cd /u01/app/oracle/product/12.1.0.2/db_1/bin;pwd'
alias cdtrace='cd /u01/app/oracle/diag/rdbms/db1/db1/trace;pwd'
alias cdnet='cd /u01/app/oracle/product/12.1.0.2/db_1/network/admin;pwd'

alias orae='ORACLE_SID=DB1;ORACLE_BASE=/u01/app/oracle;ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/db_1'
alias sqp='/u01/app/oracle/product/12.1.0.2/db_1/bin/sqlplus / as sysdba'
alias rmn='/u01/app/oracle/product/12.1.0.2/db_1/bin/rman target / nocatalog'

<- Legacy