oracledba.help works great on your mobile device too!

oracledba.help

Linux Quick Reference for DBAs

<- SpecialTopics

TOC

Linux OS install here.

Boot, Last

 who -b
 system boot  2017-10-20 07:26

Boot Log

 last reboot
 reboot   system boot  4.1.12-61.1.18.e Wed Jul 19 08:55 - 09:08  (00:12)    
 reboot   system boot  4.1.12-61.1.18.e Wed Jul 19 08:53 - 09:08  (00:15)    
 reboot   system boot  4.1.12-61.1.18.e Wed Jul 19 08:43 - 09:08  (00:24)    

 wtmp begins Wed Jul 19 08:43:36 2017

crontab

  • List: crontab -l
  • Edit: crontab -e
# Min.  Hour  Day   Month  Weekday
# 0-60  0-23  1-31  1-12   0-6 (0 is Sunday)
# Every n minutes = */n
OUT=/u01/app/scripts/out
SCRIPTS=/u01/app/scripts

# Every 5 Minutes
*/5 * * * * $SCRIPTS/myscript.sh 2>&1

# Output Job to File
0 13 * * 0 $SCRIPTS/myscript.sh > $OUT/myscript.sh.out

cron Output

By default cron outputs to /var/spool/mail/<username>

  • Delete all mail
    • -N Inhibits the initial display of message headers when reading mail or editing a mail folder.
    • d * delete all mails
mail -N
d *
quit

DHCP, Release and Renew Lease

Interface name in example: enp0s8

dhclient -r enp0s8
dhclient enp0s8
ifconfig enp0s8

In VirtualBox get a new IP by: Network-> Adpater -> Advanced -> Refresh MAC address button.

hostname, Changing

Change a systems hostname (to lnx02 example).

  • Run hostnamectl.
    hostnamectl set-hostname lnx02
  • vi /etc/sysconfig/network
    HOSTNAME=lnx02

VBox: resolv.conf

Set IP to current system.

  1. chattr -i /etc/resolv.conf
  2. vi /etc/resolv.conf
    Example entry line: nameserver 192.168.56.71
  3. chattr +i /etc/resolv.conf

VBox: dnsmasq

Set IP (listen-address) to current system.

  1. vi /etc/dnsmasq.conf
expand-hosts
local=/localdomain/
listen-address=127.0.0.1
listen-address=192.168.56.71
bind-interfaces
  1. service dnsmasq restart
  2. If needed: shutdown -r now

fdisk

fdisk can be used to create LINUX disk partitions.

  • Replace /dev/sdb with your disk name.
  • Ensure your OS can see the disk add partition to using: lsblk

Procedure

  1. Run: fdisk /dev/sdb
  2. Command (m for help): new
  3. Partition type: primary
  4. Partition Number: 1
  5. First sector: <Enter> taking default (first).
  6. Last sector: <Enter> taking default (last).
  7. Command (m for help): w to write changes to disk.

lsblk now shows the attached partition (sdb1)

sdb               8:16   0    8G  0 disk 
└─sdb1            8:17   0    8G  0 part

Repeat above steps for each disk.

Run partprob

partprobe is a program that informs the operating system kernel of partition table changes, by requesting that the operating system re-read the partition table.

/sbin/partprobe

Disable Linux Disk IO Scheduler (if to be shared disk for ASM)

Confirm Scheduler Exists

 cat /sys/block/sdb/queue/scheduler
 cat /sys/block/sdc/queue/scheduler
 ...

Change

 echo deadline > /sys/block/sdb/queue/scheduler
 echo deadline > /sys/block/sdc/queue/scheduler
 ...

Files

Delete Files Older Than x Days on Linux

find /u01/app/oracle/diag/rdbms/dnaprod/dnaprod4/trace/* -mtime +30 -exec rm {} \;

Delete Directory and All Files in It

cd /my_parent_dir
rm -r -f my_sub_dir_to_delete

find

  • find -name "*.aud" -printf '%h\n' > /tmp/aud.txt
  • find / -name "myscript" -printf '%h\n'
  • find / -iname "myscript"
  • find /u01/orasw -iname "myscript.sh"
  • find $ORACLE_HOME -iname "myscript.sh"

Folder and File Sizes

Find folder\files taking up the most space.
Folders

  • cd /parent_folder_with_subfolders_to_check
  • du -shx */

Files

  • cd /folder_to_check
  • du -sh *

Permissions (per chmod)

 # Permission              File Attributes
 7 read, write and execute rwx
 6 read and write          rw-
 5 read and execute        r-x
 4 read only               r--
 3 write and execute       -wx
 2 write only              -w-
 1 execute only            --x
 0 none                    ---

 mkdir /u01/rman
 chown -R oracle:oinstall /u01/rman
 chmod -R 765 /u01/rman

fstab and mount

The following scenario creates /u01 from the disk partition /dev/sdb.
Note: Disks for ASM only need to be partitioned, i.e. not formatted and mounted.

Create Disk Partitions

 Display existing: lsblk
 Create partition: fdisk /dev/sdb

Ensure Not mounted

 umount -l /dev/sdb1

Create xfs Filesystem
mkfs.xfs -f /dev/sdb1

 You can only mount file systems, not partitions. An empty partition (one   
 created with fdisk but not initialized with mkfs) cannot be mounted. If
 file -s can’t find a file system inside a disk partition, you cannot mount
 it – you probably forgot to make one using mkfs.

 To check use this: file -s /dev/sdb1

Create Mount Points

 cd /
 mkdir /u01
 mount /dev/sdb1 /u01

Check

 mount | grep /dev/sdb1
 df -h

Make Mount Point Permanent

 vi /etc/fstab
 /dev/sdb1  /u01 xfs  defaults  0  0

 Test /etc/fstab via command: mount -a

 This command mounts everything in the /etc/fstab except for those
 lines that contain the noauto keyword.  In this way you can test 
 without rebooting.

Reboot Check

 shutdown -r now
 mount | grep /dev/sdb1
 df -h

Creating Partitions Larger than 2tb

The following scenario creates a partition larger than 2tb (2.3tb) via ext4 filesystem.

 -- Show Existing Partitions
 root> lsblk

 If need be unmount
  umount -l /u02

 -- Create Partition
 root> parted /dev/sdb
   mklabel gpt
   Warning: The existing disk label on /dev/sdb will be destroyed 
            and all data on this disk will be lost. 
            Do you want to continue? Yes/No? yes
   unit TB
   mkpart primary 0.00TB 2.30TB
   print
   quit

 root> lsblk

 -- Format New Partition as ext4
 root> mkfs.ext4 /dev/sdb1

 -- Create Mount Point
 root> cd /
 root> mkdir /u02
 root> mount /dev/sdb1 /u02
 root> df -h

 -- Make Permanent
 root> vi /etc/fstab
 /dev/sdb1  /u02 ext4  defaults  0  0

 root> mount -a

The max size for EXT2 type file systems is 2tb.

gnome, Change default app settings

You must install dconf-editor to change settings via GUI.

  • yum install dconf-editor -y
  • Run: dconf-editor (Run for each user account as desired.)

nautilus

org -> gnome -> nautilus ->
 list-view
    default-visible-columns: ['name', 'size', 'type', 'date_modified', 'permissions']
    user-tree-view [x]
 preferences
    always-use-location-entry  (enables access to pwd top text)
    show-hidden-files [x]

gedit

org -> gnome -> gedit-> preferences
 editor 
    insert-spaces [x]
    tabs-size: 3
 ui
    status-bar-visible: [x]
    side-panel-visible: [x] 
    toolbar-visible: [x] 

Interface, Bounce

Format: if[up|down] <InterfaceName>

ifdown ens192
ifup ens192

Use ifconfig to show status.

Set Default Kernel to Use at Boot Menu

The IDs are assigned in order the menu entries appear in the /etc/grub2.cfg file starting with 0. To specify which kernel should be loaded first, pass its number to the grub2-set-default command.

  Print List of Boot Options
  awk -F\' /^menuentry/{print\$2} /etc/grub2.cfg

  Set kernel to Boot
  Example to set it to 2nd line item.
  grub2-set-default 1

  Verify the new default kernel
  Check the below file to see the kernel which will be loaded at next boot, 
  crosscheck the numeric value with the menuentry in the /etc/default/grub file.

  cat /boot/grub2/grubenv |grep saved
  saved_entry=1

  Rebuild GRUB2
  Changes to /etc/default/grub require rebuilding the grub.cfg file as follows:
  grub2-mkconfig -o /boot/grub2/grub.cfg

  Test
  Once you have verified everything and rebuilt the GRUB2 configuration file, 
  you can go ahead an reboot the server for changes to take effect.
  shutdown -r now

  To check the current kernel:
  uname -a

Kill a Program

Assuming your process\program name is: stdmaint.sh

  • View program: ps -ef|grep stdmaint
  • Kill by name: pkill stdmaint.sh
  • Kill by Process ID:
    • Get Process ID: pgrep stdmaint OR ps -ae|grep stdmaint
    • Kill program: kill <ProcessID>

To check active SQL before and after you can use this:

column executions      format 999999
column username        format a10
column sid             format 999999
column serial#         format 999999
column users_executing format 9999
column sql_text        format a50
SELECT a.executions, 
       b.username, 
       b.sid, b.serial#,
       a.users_executing, 
       a.sql_text
FROM v$sql a, v$session b
WHERE users_executing > 0 
AND a.address = b.sql_address 
AND a.hash_value = b.sql_hash_value 
ORDER BY address, child_number;

GUI Apps

  • ASM Cfg Assistent: asmca
  • GEdit: gedit
  • Disk Manager: gnome-disks
  • File Mgr: xdg-open .
  • Firefox: firefox
  • Network Connections: nm-connection-editor
  • System Monitor: gnome-system-monitor

The command-line equivalents for any standard menu app can be found by grep-ing/browsing /usr/share/applications. All menu items point to .desktop files. Most are within this directory.

MTU, Confirming MTU Value is Working

These tests can be used to confirm if a given interface MTU value (ex: 9000) is working on a given interface\IP.

traceroute

traceroute rac01-priv --mtu 
traceroute -F rac01-priv 9000

ping

ping <privIP> -c 1 -M do -s 8972
PING 10.15.124.13 (10.15.124.13) 8972(9000) bytes of data.
8980 bytes from 10.15.124.13: icmp_seq=1 ttl=64 time=0.018 ms

--- 10.15.124.13 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.018/0.020/0.022/0.004 ms
PING 10.15.124.14 (10.15.124.14) 8972(9000) bytes of data.
8980 bytes from 10.15.124.14: icmp_seq=1 ttl=64 time=0.167 ms

If the MTU sizes don’t match the ping command will return “Packet needs to be fragmented by DF set”. The reason for the 8972 on *nix devices is that the ICMP/ping implementation doesn’t encapsulate the 28 byte ICMP (8) + IP (20) (ping + standard internet protocol packet) header – thus we must take the 9000 and subtract 28 = 8972.

Example of forcing an error:

rac02>  ping 10.15.124.13 -c1 -M do -s 9999
PING 10.15.124.13 (10.15.124.13) 9999(10027) bytes of data.
ping: local error: Message too long, mtu=9000

iperf

iperf server, 
  rac01> iperf -s -u -i 10

iperf client
  rac02> iperf -c rac01-priv -u

-c is saying this is the client side. rac01-priv is my private network IP address 
-u says to send UDP traffic.

Mounting a Windows\Samba Share

On Windows System

  1. Identify windows system to host share (ex: winsrv01).
  2. Create Windows share (ex: oradata).
  3. Create user used to connect mount point to share (ex: orasvc).
  4. Assign privs to user to use data in share.

On Linux System

  1. Ensure cifs-utils installed.
    yum install cifs-utils -y
  2. Create mount point (ex: /mnt/winsrv01).
  3. Issue mount command.
root> mkdir /mnt/winsrv01
root> mount -t cifs //winsrv01/oradata -o username=orasvc,password=MyPassowrd /mnt/winsrv01

root> df -h
      winsrv01/oradata  190T   79T  112T  42% /mnt/winsrv01

Unmount Volume

umount /mnt/winsrv01

df -h shows it gone if successful.

Configure NTP

It is critical that the time of all your nodes is kept in sync for an Oracle RAC environment. Swift changes in time can lead to Oracle shutting down a node(s) due to inconsistent timers. Use the following line\values for the /etc/sysconfig/ntpd file.

OPTIONS="-g -x -u ntp:ntp -p /var/run/ntpd.pid"

  • -g = panicgate: allows the first adjustment to exceed the panic limit (1000s by default).
  • -p = pidfile: name and path of the file used to record ntpd process ID.
  • -u = Linux user account to own process.
  • -x = slew: make micro time adjustments (rather than in one large adjustment).

-- If CentOS: Disable chrony (the default)

  • systemctl stop chronyd
  • systemctl disable chronyd.service
  • systemctl disable chronyd
  • mv /etc/chrony.conf /etc/chrony.conf.orig
  • systemctl status chronyd

-- System Date & Time: View
date
Tue Nov 22 08:46:20 EST 2016

-- System Date & Time: Set
date MMDDhhmmYYYY
date 112208362016
Tue Nov 22 08:36:00 EST 2016

-- ntpd Service: Installation

  1. yum install -y ntp
  2. systemctl enable ntpd
  3. systemctl start ntpd
  4. systemctl status ntpd
  5. ntpq -p
    • If you don't see a * next to a server your ntpd is not synced to any time source.-
    • It might take up to 5 minutes to sync after system booted.

-- Sync NTP To Time Source
Add server entry to /etc/ntp.conf with prefer clause.

  1. vi /etc/ntp.conf
  2. server <NTP_Auth_Server_IP> prefer
  3. systemctl restart ntpd
  4. ntpq -p
    You should now see a * next to a server.

-- Stop|Disable

  • systemctl stop ntpd
  • systemctl disable ntpdate

-- Status Check

  • ntpq -p
  • ntpdc -c loopinfo
  • ntpstat
  • timedatectl <== Alert: Can produce inaccurate results!
  • For RAC as grid user: cluvfy comp clocksync -n all -verbose

-- Sync NTP Manually
This forces the system time to change to NTP server's time now!

  1. service ntpd stop
  2. ntpdate -s <YourNTPServerIP>
  3. service ntpd start
  4. Check status: ntpq -p

If ntpdate does not work you can use ntpd -qg on that step.

NTP Troubleshooting

sendmail

The following covers Linux system changes required to be able to send an email using sendmail.

Installation

  • yum install m4 telnet mailx -y
  • yum install sendmail sendmail-cf -y

Relay Server Configuration (if used)

  • cp /etc/mail/sendmail.mc /etc/mail/sendmail.mc.orig
  • vi /etc/mail/sendmail.mc
    Note: Remove dnl entry at the beginning of line to uncomment the line.
    • Uncomment and change SMART_HOST entry to your relay email server.
    define(`SMART_HOST', `relay.myrelayserver.local')dnl
    • Uncomment and change MASQUERADE_AS entry to your email server.
    MASQUERADE_AS(`mydomain.com')dnl
    • Uncomment FEATURE(masquerade_envelope).
    FEATURE(masquerade_envelope)dnl
  • Rebuild
    • cd /etc/mail
    • make
    • service sendmail restart

Monitoring

tail -f /var/log/maillog

Usage

mail -s "Subject" ReceverEmailAddress@Domain < PathToEmailMsgFile

mail -s "Email Test 1" michaele@mydomain.com < /dev/null
mail -s "Email Test 2" michaele@mydomain.com < /tmp/MyEmailMsg.txt
mail -s "Email Test 3" "em1@abc.com em2@abc.com" < /dev/null

Attachment Example
mail -a /u01/app/scripts/rpt/db_growth.csv -s "Growth Rpt" scott@oracle.com < /dev/null

scp

If the hostname does not work try the IP and vice-versa.

PUSH

  ping rac02
    10.4.0.152
  rac01> scp /u02/backup/q6s2mkd3_9030_1.rman oracle@10.4.0.152:/u02/tmp_rman

PULL

  ping rac01
    10.230.2.16
  rac02> scp oracle@10.230.2.16:/u02/backup/q6s2mkd3_9030_1.rman /u02/tmp_rman

Automate Password Prompt via sshpass

yum install wget
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/sshpass-1.05-1.el6.x86_64.rpm
rpm -ivh sshpass-1.05-1.el6.x86_64.rpm
which sshpass

sshpass -p 'mypassword' scp /temp/myfile.dmp oracle@192.168.1.42:/u02/exports

sshpass rpm is here.

shutdown

 Reboot:                        shutdown -r now
 Shutdown and Power Off (halt): shutdown -h now

System Messages

# less /var/log/messages
# more -f /var/log/messages
# cat /var/log/messages
# tail -f /var/log/messages
# grep -i error /var/log/messages

Common Linux log files names and usage

  • /var/log/messages : General message and system related stuff
  • /var/log/auth.log : Authenication logs
  • /var/log/kern.log : Kernel logs
  • /var/log/cron.log : Crond logs (cron job)
  • /var/log/maillog : Mail server logs
  • /var/log/httpd/ : Apache access and error logs directory
  • /var/log/boot.log : System boot log
  • /var/log/secure or /var/log/auth.log : Authentication log
  • /var/log/utmp or /var/log/wtmp : Login records file
  • /var/log/yum.log : Yum command log file.

Create\Add New SWAP (using a file)

Change count=8096 to match size you need for SWAP.

-- Create

 dd if=/dev/zero of=/swapfile-additional bs=1M count=8096
 mkswap /swapfile-additional
 chmod 600 /swapfile-additional

-- Make Permanent

 vi /etc/fstab
 /swapfile-additional swap swap    0   0

-- Mount and Enable

 mount -a
 swapon -a

-- Check

 swapon -s
 free -m

Changing SWAP

If required you can enable or repoint the SWAP to another disk after the OS install using these steps:

  1. Allocate a new disk from VirtualBox or your OS (Ex: sdb).
  2. Disable existing SWAP (Ex: /dev/mapper/ol-swap)
    swapoff /dev/mapper/ol-swap
  3. Enable the new disk as your SWAP.
    mkswap /dev/sdb
    swapon -f
  4. Make permenent by editing your /etc/fstab file.
    Line entry: /dev/sdb swap swap defaults 0 0
  5. Reboot to test.
    shutdown -r now
    lsblk

tar

Create (tar all files in current dir)

 tar -cvof scripts.tar *

Create (then delete all tar'd files)

 tar -cvf scott.tar --exclude '*.tar' --remove-files scott*.dmp

Extract

 tar -xvf scripts.tar

List Contents:

 tar --list --file=scripts.tar

telnet

telnet mydomain.com 25

ehlo mydomain.com
mail from: oracle@MyHostName
rcpt to: michaele@mydomain.com
data

Subject: Test

Hello, this is a test.
<CRLF>
.
<CRLF>

tmpfs: /dev/shm

The value of /dev/shm must be larger than your Oracle MEMORY_TARGET value. For instance, if you want your database to use 2048m you could set /dev/shm to 2304m.

Short Term

 mount -t tmpfs shmfs -o size=2304m /dev/shm

Permanently

 vi /etc/fstab and add this entry:
 shmfs /dev/shm tmpfs size=2304m 0

Display Current Value

 df -h

top

z = color mode
t = cpu    (graphic mode)
m = memory (graphic mode)
c = COMMAND detail
j = Justify COMMAND

u = show for just one user
W = Write changes to file (set default values).
Y = Inspect PID

Update OS

root> yum update -y

CentOS

0. Login as root user.
1. Applications -> System Tools -> Software Update
2. Continue.
3. Process runs that checks for updates.
4. Next a Window showing how many updates are available displayed.
5. Select [Install Updates].

vi

vi cheat sheet here.

  • Delete to EOL: D
  • Find string: <Esc> :/EnterString
  • Line Numbers: <Esc> :set number |set nonumber
  • Move
    • Top of File: gg
    • Beginning|End of Line: <-0 | $ ->
    • End of File: shift+g
    • Page Forward|Back: Ctrl+f|Ctrl+b
    • Point in File: <Esc> nn% (note no colon)
  • Search and Replace: <Esc> :%s/pattern/replace/
  • Turn hilite off: <Esc> :noh
  • Write & Quit: <Esc> :wq!
  • Quit: <Esc> :q

Xming

With XMing you can launch GUI Linux applications from a putty session. Set the following values in putty to use XMing: Connection -> SSH -> X11

  X11 forwarding
    [x] Enable X11 forwarding
    X display location: localhost:0
  Remote X11 authentication
     (x) MIT-Magic-Cookie-1

Ensure XMing is running before you run your putty session and launch a GUI app.