What is it?
RDM is mapping file used to map a LUN directly to VM bypassing VMFS layer. This LUN can be formatted using any File System (NTFS or FAT32) without the need to format it using VMFS and placing VMDK file on top of it.
The mapping file is located with the virtual machine directory. Here is an example of RDM mapping file for a VM called VM-01 which has pRDM LUN and vRDM LUN (will cover them later in details).
~ # cd /vmfs/volumes/SAN-DATASTORE/VM-01
/vmfs/volumes/507fd84d-e9b583ac-b370-001b244d6cb2/VM-01 # ls -la
drwxr-xr-x 1 root root 2800 Nov 3 12:18 .
drwxr-xr-t 1 root root 2100 Nov 2 21:33 ..
-rw-r--r-- 1 root root 73 Oct 20 10:21 VM-01-e61f5754.hlog
-rw------- 1 root root 2147483648 Nov 3 11:39 VM-01-e61f5754.vswp
-rw------- 1 root root 42949672960 Nov 3 12:49 VM-01-flat.vmdk
-rw------- 1 root root 8684 Nov 3 11:40 VM-01.nvram
-rw------- 1 root root 541 Nov 3 11:38 VM-01.vmdk
-rw-r--r-- 1 root root 0 Oct 19 15:10 VM-01.vmsd
-rwxr-xr-x 1 root root 3679 Nov 3 12:18 VM-01.vmx
-rw------- 1 root root 0 Oct 20 19:04 VM-01.vmx.lck
-rw-r--r-- 1 root root 260 Nov 3 12:18 VM-01.vmxf
-rwxr-xr-x 1 root root 3680 Nov 3 12:18 VM-01.vmx~
-rw------- 1 root root 32212254720 Nov 3 12:18 VM-01_1-rdm.vmdk ---------------------> vRDM mapping file
-rw------- 1 root root 482 Nov 3 12:18 VM-01_1.vmdk ----------------------> Descriptor File
-rw------- 1 root root 32212254720 Nov 3 12:18 VM-01_2-rdmp.vmdk ---------------------> pRDM mapping file
-rw------- 1 root root 494 Nov 3 12:18 VM-01_2.vmdk ----------------------> Descriptor File
-rw-r--r-- 1 root root 579252 Oct 20 10:21 vmware-1.log
-rw-r--r-- 1 root root 139522 Oct 20 19:03 vmware-2.log
-rw-r--r-- 1 root root 307364 Nov 3 12:19 vmware.log
-rw------- 1 root root 112197632 Oct 20 19:04 vmx-VM-01-3860813652-1.vswp
Notice the difference between the descriptor file of a VMDK Hard drive and RDM LUN Hard Drive.
Important Note: The size of the mapping file equals to the size of the LUN. Although RDM LUN is obtained directly from SAN, the same space of the LUN should be maintained in VMFS datastore due to mapping file.
How it works?
When a LUN is opened for access from inside the VM, the mapping file is read to obtain the reference to the raw LUN (because mapping file is having the metadata for the LUN). Thereafter, reads and writes go directly to the raw LUN rather than going through the mapping file.
VM .vmx file is having reference pointing to RDM mapping file and not the LUN. For our example VM-01
scsi0:1.fileName = "VM-01_1.vmdk"
scsi0:1.mode = "persistent"
scsi0:1.ctkEnabled = "FALSE"
scsi0:1.deviceType = "scsi-hardDisk"
scsi0:1.present = "TRUE"
scsi0:1.redo = ""
scsi0:2.fileName = "VM-01_2.vmdk"
scsi0:2.mode = "independent-persistent"
scsi0:2.ctkEnabled = "FALSE"
scsi0:2.deviceType = "scsi-hardDisk"
scsi0:2.present = "TRUE"
scsi0:2.redo = ""
When to use?
Always VMFS is preferred over RDM. But there are some scenarios where RDM is required:
- SAN snapshots: Using RDM enables backup software's to ‘see’ inside the LUN and to create a consistent snapshot of the data contained within the LUN. This allows for replication of data (snapshots) to a second SAN (on a different physical location) for Disaster Recovery.
- In-band management software for your SAN running in Virtual Machines might require direct access to a management LUN. This usually requires physical compatibility mode.
In any MSCS clustering scenario that spans physical hosts cluster data and quorum disks should be configured as RDMs rather than as virtual disks on a shared VMFS. The reason is that cluster members need to have access to shared LUN (quorum).
What are the available types?
RDM has two compatibility modes:
1. Virtual Mode
In this mode all SCSI commands from the VM to LUN will pass through VMFS layer except READ & WRITE. Those will be passed directly to the LUN.
If you check the device manager of the VM you will see the RDM LUN listed as VMware Virtual Disk SCSI Disk Device. This is exactly the same as VMDK Drive added to the VM. The physical characteristics of the LUN are hidden from the VM.
The advantage of vMode is that you can still use Distributed Locking Mechanism of VMFS as well as snapshots of RDM LUN drives.
2. Physical Mode
In this mode all SCSI commands are passed to RDM LUN directly from the VM expect REPORT command virtualized by VMkernal.
If you check the device manager of the VM you will see the RDM LUN listed based on the physical name/characteristics of the storage array.
The advantage of the full exposure of the LUN characteristics to the VM can be seen when using SAN Management Tools and MSCS services. Also, RDM LUNs in physical mode can have a size greater than 2TB.
Another important feature inherited with RDM is Dynamic Name Resolution. Each LUN is having unique ID in RDM mapping file used by VMFS. In case the path to the LUN modified due to failover, for example, the ID of the LUN will be change. Dynamic Name Resolution feature will automatically discover the new name and associate it in the RDM mapping file to resume operations.
VMFS vs. RDM
Here is a comparison between the types of disk drives available and the supported features.
RDM interoperatibility with vSphere Advanced Features
Both pRDM and vRDM support vSphere advnaced features such as HA, DRS, SDRS, VM, SVM. However there are some limitations for some features.
vMotion- Files for a virtual machine are not relocated when it is vMotioned.
- The virtual machine is re-registered to the destination host.
- Any RDMs remain as RDMs when the virtual machine is registered to another host. That is, no changes to the virtual machine itself are made.
Cold migration can be for resources, datastores, or both while the VM is powered-off. In case migrating datastore is selected, here are some considerations for RDM:
- Any non-RDM virtual disks are physically moved to the destination.
- The virtual machine configuration files are physically moved to the destination.
- Raw LUNs themselves cannot be moved, as they are raw disks presented from the SAN. However, the pointer files (RDMs) can be relocated if required.
- During the migration you can use the advanced section of the migration wizard and select if you would like to maintain the same format of the files at the destination (remain as RDM) or convert it to a thick/thin disk.
- The virtual machine registration changes, but the files are left untouched.
When cloning a VM with RDMs, pRDM & vRDM contents will be copied into VMDK virtual disks. If you want to clone a virtual machine without its RDMs, remove them from the configuration of the virtual machine before cloning. You can delete the RDM from the disk when removing it (the raw LUN contents are not changed, only the RDM mapping file is deleted). Re-add them to the configuration when completed.
Storage vMotion- When you perform Storage vMotion, the virtual machine files are physically relocated to a destination datastore. The same host retains ownership or registration of the virtual machine after Storage vMotion completes.
- For vSphere 4.0 and higher:
# Virtual disks and virtual mode RDM pointer files can be relocated to the destination datastore.
# Virtual disks can be converted to thick-provisioned or thin-provisioned disks during migration as long as the destination is not an NFS datastore.
# Physical mode RDM pointer files can be relocated to the destination datastore, but physical mode RDM data cannot be migrated.
# Virtual mode RDM data can be migrated with storage vMotion, provided sufficient licensing is present.
- In ESX 5.0, during the Storage vMotion, virtual mode RDMs can be converted to thick/thin provisioned disks by selecting the appropriate options under the Advanced section of the migration wizard. Physical mode RDMs can only be converted with a cold migration.
- If you are attempting a Storage vMotion of a virtual mode RDM using the Advanced method and the RDM pointer mapping file is already present in the target datastore, the Storage vMotion finishes quickly, but without moving the data. This is because the Storage vMotion detects that the source and target datastores for the mapping file are the same, and therefore concludes that no movement is needed.
To work around this issue:
1. Remove the RDM from the virtual machine, and delete it from disk to ensure the mapping file is deleted. This will not delete the data in the RDM.
2. Add the RDM again, and put the mapping file in a different datastore than the ultimate target of the storage vMotion.
3. After the virtual machine reconfiguration is complete, attempt the storage vMotion again.
1. RDM can't be used with NFS datastore.
2. SVM can't be used with NPIV
In case RDM is greyed out:
1. Make sure that you have a free LUN which support RDM
2. Make sure that RDMFiler Advanced parameter is unchecked
Note: In case a LUN is used by ESXi host (even if partially), you can't use it as RDM to VM and vice versa. The reason is that OS or Hypervisor will apply lock into the LUN which will prevent it to be used by other system.
After adding the RDM to VM, you don't need to format it (assuming that it was used on previous system and was previously formatted). You need ONLY to bring it online using disk management. In case of fresh new RDM assigned to VM, it needs to be formatted for first use.
Note: RDM isn’t support with vCloud and should be avoided.