This short article will be about Oracle AFD driver introduced with Oracle 12c. As always I will write only about specific points which are not described in other sources. If you want to get more information about AFD driver refer to Oracle documentation, there are also many blogs describing it.
Oracle AFD stands for Oracle ASM Filter Driver, it is intended to replace ASMLIB driver for Linux. Oracle will develop and extend AFD driver. From main benefits it have filtering possibility, which filters all IO requests to manged devices, only oracle utilities can write to it if filtering is enabled. Oracle says that it also supports Storage thin provisioning which is now included with most of the modern storage producers. In the future releases it should perform also some logical checking during IO.
Oracle recommends to use AFD driver in place of ASMLIB.
Managing utilities and libraries for Oracle AFD are part of oracle software binaries and installed with oracle installation, like ACFS utilities and drivers, actually it is same concept. There is positive and negative sides of it. Positive is that we do not need to separately download and install required drivers, negative side is that we need to configure ASM candidate disks before installation but there is no configuration utility beforehand.
Oracle AFD comes with Oracle GI software. We have two options to install GI in 12c:
- SIHA, Single Instance High Availability installation (former Oracle Restart)
- Clusterware Installation
During installation we should provide ASM disks to configure ASM in case of SIHA installation and additionally to locate Voting files and OCR file in case of Clusterware installation. If we want to use AFD driver from very beginnig, it is clear that we should first use Oracle AFD utilities to label some disks for installation, but these utilities will be available after installation! The only way is to do software-only installation first then load AFD module to Linux kernel and label disks, after that configure HAS or CRS depending on installation mode.
There is one important point, when we do software-only installation oracle copies binaries and relinks them. But it do not create wrapper scripts around this binaries. For example in bin directory of the GI home there is crsctl wrapper script and crsctl.bin binary executable. Now, after software-only installation there will be only crsctl.bin binary and no wrapper script crsctl and for some incomprehensible reason AFD configuration utilities uses wrapper scripts and not directly binaries, it is true for direct utilities like afdroot and when you call them over asmcmd! That is why just doing software-only installation is not enough. Related wrapper scripts are created after home configuration! In case of SIHA installation it is not a problem, because we do not have OCR or Voting files, we can execute roothas.pl to configure GI home and start HAS stack, then we will label disks and configure ASM with asmca.
How to do it in case of Clusterware installation? I will not write long, just step by step instruction for 2 node cluster installation, lets say node1 and node2:
Step 1 on node1:
There is 2 options:
1. Install binaries in software-only installation mode.
It is not the way that I use personally but also the option. After installation we will require two wrapper scripts crsctl and clsecho. Just copy them from any existing 12c GI installation, it can be aslo SIHA installtion. After copying these wrapper scripts to the GI home, check the variables inside them for home locations, if required edit them to point to current home. Now you can use asmcmd for AFD configuration and disk labeling.
2. Use binaries from some already existing GI installation, Oracle Home cloning.
It is the way I always prefer to use. I usually do installation and patching process only ones, then I create tgz package from prepared home and use it for all other installations. It is much faster and simpler process, especially when well scripted 😉 After you will copy home from existing GI installation, it can be also SIHA GI home, you will already have crsctl and clsecho wrapper scripts inside. You must only check the path variables in the scripts header to point to current home.
Step 2 on node1:
Use asmcmd to configure AFD driver, this step will load oracleafd module into the kernel and create related files. It should create /etc/afd.conf file and /etc/init.d/afd script to load AFD module on startup. Check existens of these files manually, because there is BUG when these files not created automatically.
Label required disks.
Do not set AFD discovery string at this stage. To set it correctly CRS stack should be up.
Step 3 on node2:
Do Step 1 on node2. Then configure AFD driver but do not label disks because they are already labeled at step 2, just use asmcmd afd_scan to find them. If you are not using default disk path /dev/sd* then you must specify diskpath pattern during scan asmcmd afd_scan ‘/dev/mapper/*’.
Step 4 on node1:
Now after candidate disks are configured on both nodes, configure CRS by executing root.sh, first on node1 then on node2. And finalize configuration as per documentation.
After CRS statck will be up and running on all nodes. With oracle or grid user, not root, execute asmcmd afd_dsset command to set AFD discovery string: asmcmd afd_dsset ‘/dev/mapper/*’
I also will recommend to set ASM discovery string to: asmcmd dsset ‘AFD:*,/dev/mapper/*’
Important note regarding disk sector size.
Oracle AFD by default uses Physical Sector size of the Disks. To check Logical and Physical Sector sizes use fdisk with -l option: fdisk -l /dev/dsc
I personally will suggest to use Logical Sector size which is usually 512 bytes. Last years there is new disks on the market with 4k sector size. But Oracle software is not 100% ready for 4k Sector size disks by my opinion, there is many related bugs. Also, it is possible that you will have disks with different Physical sector sizes on same machine, in this case you must use Logical sector size, because you can not have disks with different sector sizes in one Disk Group.
AFD driver had the parameter to control which sector size to use:
license: Oracle Corporation
description: ASM Filter Driver
author: Oracle Corporation
vermagic: 3.10.0-229.1.2.el7.x86_64 SMP mod_unload modversions
parm: oracleafd_use_logical_block_size:Non-zero value implies using logical sectorsize (int)
Take attention on the last line “oracleafd_use_logical_block_size:Non-zero value implies using logical sectorsize“.
We can set it dynamically:
echo 1 > /sys/module/oracleafd/parameters/oracleafd_use_logical_block_size
But, after reloading of the AFD driver manually or server reboot it is reset back!
To make it permanent on Red-Hat or Oracle Linux create file in folder /etc/modprobe.d to set module parameter for oracleafd module:
echo "options oracleafd oracleafd_use_logical_block_size=1" > oracleafd.conf
Now stop HAS or CRS and restart AFD module:
After reboot module should use correct value.
(c) Aychin Gasimov, 05/2016, Munich, Germany