Internet, UNIX, Video, Leisure…
Oracle 11g ASM install on Solaris 10 with RAW data files
I was installing a Oracle 11g + ASM server for testing and upgrade purpose when I came across a problem : how do I create RAW « filsystem » for ASM ?
I found a lot of docs for doing this with Linux, but nothing about Solaris.
Of course you will quickly find something about ‘lofs’ and ‘lofsadm’. This is the right starting point. But then ?
It seems ASM wants to have « real » block device for use with ASM. If I’m wrong, please, comment and correct me.
My solution was to :
- create two ‘zeroed’ files with DD, around 8Gb each. They will be used in a RAID 0 redundancy. This should be managed by ASM itself, and need to be configured. Do this as user Oracle
$ cd /opt/oracle $ mkdir disks $ dd if=/dev/zero of=/opt/oracle/disks/baytestdb_asm1_disk1 bs=8192 count=1000000 $ dd if=/dev/zero of=/opt/oracle/disks/baytestdb_asm1_disk2 bs=8192 count=1000000
- use ‘lofiadm’ to link the files as block devices. Do this as root user :
# lofiadm -a /opt/oracle/disks/baytestdb_asm1_disk1 /dev/lofi/1 # lofiadm -a /opt/oracle/disks/baytestdb_asm1_disk2 /dev/lofi/2 # lofiadm Block Device File /dev/lofi/1 /opt/oracle/disks/baytestdb_asm1_disk1 /dev/lofi/2 /opt/oracle/disks/baytestdb_asm1_disk2
- use ‘mknod’ to create a copy of the block device in your Oracle directory and give appropriate rights.
# ls -l /dev/lofi lrwxrwxrwx 1 root root 29 Nov 25 14:39 1 -> ../../devices/pseudo/lofi@0:1 lrwxrwxrwx 1 root root 29 Nov 25 14:39 2 -> ../../devices/pseudo/lofi@0:2 # ls -l /devices/pseudo/lofi@0:1 brw------- 1 root sys 147, 1 Nov 25 14:30 /devices/pseudo/lofi@0:1 # ls -l /devices/pseudo/lofi@0:2 brw------- 1 root sys 147, 2 Nov 25 14:36 /devices/pseudo/lofi@0:2
You will use the numbers 147, 1 and 147, 2 to create the node :
# cd /opt/oracle/disks # mknod baytestdb_asm1_disk_1 c 147 1 # mknod baytestdb_asm1_disk_2 c 147 2 # chown oracle:oinstall baytestdb_asm1_disk_* # chmod 660 baytestdb_asm1_disk_*
- Tell the Oracle ASM Installer to search in the directory (/opt/oracle/disks in my case and NOT /opt/oracle/disks/*)
OK, I admit, names of dd files and block device are not good/clear… I still havn’t found a real good and clear naming. But, heh, I did my part. Now, do yours and propose the names 
I did all this on the latest Solaris 10/08 :
cat /etc/release
Solaris 10 10/08 s10s_u6wos_07b SPARC
Copyright 2008 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 27 October 2008
uname -a
SunOS indus.adm.groupertl.net 5.10 Generic_137137-09 sun4v sparc SUNW,Sun-Fire-T1000
I will not come over all the recommended for Oracle install, like having oracle user and oinstall, oper and dba groups. Once you have all the pre-requisit and the new devices created as above, start here.
Untar, unzip or uncpio the ASM install files and the DB files. I like to do this in /opt/oracle/install.
Oracle disks are in /opt/oracle/disks. Oracle home is in /opt/oracle/shellhome. Oracle product files are in /opt/oracle/product/11. If you have RAC (crs), I do intall in /opt/oracle/product/11/crs_1. For ASM and DBs, I install in /opt/oracle/product/11/oracle_base/asm_1 and db_1.
Once this is done, start the installer as user oracle :
$ /opt/oracle/install/11GR1ASM/database/runInstaller
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 180 MB. Actual 6241 MB Passed
Checking swap space: must be greater than 150 MB. Actual 6564 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2008-11-27_10-30-27AM. Please wait ...-bash-3.00$ Warning: Cannot convert string "-monotype-arial-regular-r-normal--*-140-*-*-p-*-iso8859-1" to type FontStruct
Don’t care for the font error.
As a side note, if you are working on the server from a remote host, using SSH, you will have to redirect the display. You will have all the informations if you don’t. If you are on a Mac OsX, you will need to :
- start the X11 server (found in the Xcode tools I think)
- configure your SSH client to redirect the DISPLAY (this is by default I think – on every Unix OS)
- use the -X -Y options to ssh like : ssh -X -Y oracle@my_server_name
The Oracle installer will open with this window.
Choose the Advanced Install and click next.
Remember the baytest_asm1 name. You may have to use it later when you administer your oracle install.
Enter here, the ASM password. Keep it secure.
In Normal redundancy you will need two disks. They are not used as mirror, but should guaranty some redundancy. High redundancy will require 3 disks. External… you have to manage it. Use this is you already have some RAID you are trusting in
Click Install and wait. Wait. Waiiiiiit……….
Then…
You will have to run the script on the host as root. You will only have to give a directory where some files will be installed. Don’t use the /usr/local/bin offer. Change it to somewhere in /opt/oracle, at least. You will have to do the same thing for the Database install in a few moments.
# /opt/oracle/product/11/oracle_base/asm_1/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /opt/oracle/product/11/oracle_base/asm_1
Enter the full pathname of the local bin directory: [/usr/local/bin]: /opt/oracle/asmbin
Entries will be added to the /var/opt/oracle/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configuration for local CSS has been initialized
Cleaning up Network socket directories
Setting up Network socket directories
Adding to inittab
Startup will be queued to init within 30 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.
Cluster Synchronization Services is active on these nodes.
indus
Cluster Synchronization Services is active on all the nodes.
Oracle CSS service is installed and running under init(1M)
Finished product-specific root actions.
You can click OK and go to the next step.
And you are done.
Let’s go with the DB install now. Just run the DB installer.
$ /opt/oracle/install/11GR1DB/database/runInstaller
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 180 MB. Actual 5825 MB Passed
Checking swap space: must be greater than 150 MB. Actual 6507 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2008-11-27_11-17-05AM. Please wait ...-bash-3.00$ Warning: Cannot convert string "-monotype-arial-regular-r-normal--*-140-*-*-p-*-iso8859-1" to type FontStruct
Oracle Universal Installer, Version 11.1.0.6.0 Production
Copyright (C) 1999, 2007, Oracle. All rights reserved.
You will encounter one or two error message. Just click OK and go on. One done, click OK and Next.
You now have to run a script as root. Copy the script name and start it in a root shell of the host.
# /opt/oracle/product/11/oracle_base/db_1/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /opt/oracle/product/11/oracle_base/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]: /opt/oracle/dbbin
Entries will be added to the /var/opt/oracle/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
And you are done.
Now you should have some running processes on the host. Notice the two listeners, one for ASM, one for the DB.
Also note the /opt/oracle/product/11/oracle_base/asm_1/bin/ocssd.bin process. This one is the link between ASM and the database.
Check next post for the Oracle 11.1.0.6 to 11.1.0.7 upgrade.
27 novembre 2008 - 12:02
I haven’t really worked with ASM a lot but I suspect they do indeed want real raw disks to play with. Basically attach drives to the system, feed em to ASM and you’re done, and in theory at least Oracle keeps « hotspots » where some drives are used too much and others not enough from forming.
As for the naming scheme, you’re on your own.
27 novembre 2008 - 15:36
That is exactly true. The goal here is to have a dev server that can have reduced performances but stick to the production architecture.
Please, check the article again as I changed/added a lot of stuffs.
Thanks for the comment.
9 octobre 2009 - 15:01
Please if possible, could you show examples for: « use ‘mknod’ to create a copy of the block device in your Oracle directory and give appropriate rights ».
20 novembre 2009 - 23:56
It’s in the article :
# cd /opt/oracle/disks
# mknod baytestdb_asm1_disk_1 c 147 1
# mknod baytestdb_asm1_disk_2 c 147 2
# chown oracle:oinstall baytestdb_asm1_disk_*
# chmod 660 baytestdb_asm1_disk_*
7 février 2010 - 05:52
Thank you for the help.
Great Audio info, i’m bookmarking the page for the great content.
20 février 2010 - 12:21
Hi ,im using this model of Western Digital hard drive: WD5002ABYS with no problems. But i thing that rocket raid has HDD compatibility list, where you can find supported drives.
By the way Thank you very much for taking time to share on this subject.
20 février 2010 - 12:49
Compatibility list, unfortunately, changed and they removed the drived that was there and I bought. It was too late for me to give the drives back.
22 mars 2010 - 12:04
when going to create link using lofiadm getting below error:
bash-3.00# lofiadm /ocr/ocrasm
lofiadm: /dev/lofictl: No such file or directory
there is not any file « lofi* » in the ./dev directory.
I am using sun sparc 5.10 and virtualization created on that so I am getting two zone …zone A and zone B for RAC setup. Is this creating any problem?
29 décembre 2010 - 03:23
I just use raw slices on the disks, using format to create a partition map to my liking (don’t use slice 2, or it will wipe out your partition map):
For ASM disks:
format
dis
par
mod
Do you wish to continue creating a new partition
table based on above table[yes]?
Free Hog partition[6]?
Enter size of partition ’0′ [0b, 0c, 0.00mb, 0.00gb]: 1c
Enter size of partition ’1′ [0b, 0c, 0.00mb, 0.00gb]:
Enter size of partition ’3′ [0b, 0c, 0.00mb, 0.00gb]:
Enter size of partition ’4′ [0b, 0c, 0.00mb, 0.00gb]:
Enter size of partition ’5′ [0b, 0c, 0.00mb, 0.00gb]:
Enter size of partition ’7′ [0b, 0c, 0.00mb, 0.00gb]: 10g
Part Tag Flag Cylinders Size Blocks
0 root wm 0 – 0 7.25MB (1/0/0) 14848
1 swap wu 0 0 (0/0/0) 0
2 backup wu 0 – 65532 463.98GB (65533/0/0) 973033984
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 usr wm 1 – 64119 453.97GB (64119/0/0) 952038912
7 unassigned wm 64120 – 65532 10.00GB (1413/0/0) 20980224
Okay to make this the current partition table[yes]?
Enter table name (remember quotes): « asm »
Ready to label disk, continue? y
partition> qui
FORMAT MENU:
disk – select a disk
type – select (define) a disk type
partition – select (define) a partition table
current – describe the current disk
format – format and analyze the disk
repair – repair a defective sector
label – write label to the disk
analyze – surface analysis
defect – defect list management
backup – search for backup labels
verify – read and display labels
save – save new disk/partition definitions
inquiry – show vendor, product and revision
volname – set 8-character volume name
! – execute , then return
quit
format> dis
Then find the slices, and make device nodes for them:
#!/bin/bash
# get device major and minor for c4t600***s6 & create script to create devices
ls -lL /dev/rdsk 2>/dev/null | sed ‘s/,/ /’ | awk ‘/c4t600/ && /s6/ {print « mknod » $10 » c » $5″ » $6}’ > make_devs
echo « chmod -R g+w /asm » >> make_devs
echo « chown -R oracle:dba /asm » >> make_devs
echo « mv -f make_devs /asm/scripts » >> make_devs
chmod 755 make_devs
Then test the io on your new devices:
for i in $(ls ASM*) do
echo » »
echo « Writing a Gigabyte to « $i.
time dd if=/dev/zero of=$i ibs=1024000 obs=1024000 count=1000
Then, when you set up ASM, just set the path to your asm directory, with a wildcard for your drives:
/asm/c1t*
or something like that. I do this for RAC and single instance databases. For the RAC ones, you have to have a drive system that supports dual pathing.