Tuesday, 6 November 2012

Raw Device Mapping (RDM) DeepDive

What is it?

RDM is mapping file used to map a LUN directly to VM bypassing VMFS layer. This LUN can be formatted using any File System (NTFS or FAT32) without the need to format it using VMFS and placing VMDK file on top of it.

The mapping file is located with the virtual machine directory. Here is an example of RDM mapping file for a VM called VM-01 which has pRDM LUN and vRDM LUN (will cover them later in details).

~ #  cd /vmfs/volumes/SAN-DATASTORE/VM-01

/vmfs/volumes/507fd84d-e9b583ac-b370-001b244d6cb2/VM-01 # ls -la
drwxr-xr-x    1 root     root          2800 Nov  3 12:18 .
drwxr-xr-t    1 root     root          2100 Nov  2 21:33 ..
-rw-r--r--    1 root     root            73 Oct 20 10:21 VM-01-e61f5754.hlog
-rw-------    1 root     root     2147483648 Nov  3 11:39 VM-01-e61f5754.vswp
-rw-------    1 root     root     42949672960 Nov  3 12:49 VM-01-flat.vmdk
-rw-------    1 root     root          8684 Nov  3 11:40 VM-01.nvram
-rw-------    1 root     root           541 Nov  3 11:38 VM-01.vmdk
-rw-r--r--    1 root     root             0 Oct 19 15:10 VM-01.vmsd
-rwxr-xr-x    1 root     root          3679 Nov  3 12:18 VM-01.vmx
-rw-------    1 root     root             0 Oct 20 19:04 VM-01.vmx.lck
-rw-r--r--    1 root     root           260 Nov  3 12:18 VM-01.vmxf
-rwxr-xr-x    1 root     root          3680 Nov  3 12:18 VM-01.vmx~
-rw-------    1 root     root     32212254720 Nov  3 12:18 VM-01_1-rdm.vmdk ---------------------> vRDM mapping file
-rw-------    1 root     root           482 Nov  3 12:18 VM-01_1.vmdk ----------------------> Descriptor File
-rw-------    1 root     root     32212254720 Nov  3 12:18 VM-01_2-rdmp.vmdk ---------------------> pRDM mapping file
-rw-------    1 root     root           494 Nov  3 12:18 VM-01_2.vmdk ----------------------> Descriptor File
-rw-r--r--    1 root     root        579252 Oct 20 10:21 vmware-1.log
-rw-r--r--    1 root     root        139522 Oct 20 19:03 vmware-2.log
-rw-r--r--    1 root     root        307364 Nov  3 12:19 vmware.log
-rw-------    1 root     root     112197632 Oct 20 19:04 vmx-VM-01-3860813652-1.vswp

Notice the difference between the descriptor file of a VMDK Hard drive and RDM LUN Hard Drive.
Important Note:  The size of the mapping file equals to the size of the LUN. Although RDM LUN is obtained directly from SAN, the same space of the LUN should be maintained in VMFS datastore due to mapping file.

How it works?

When a LUN is opened for access from inside the VM, the mapping file is read to obtain the reference to the raw LUN (because mapping file is having the metadata for the LUN). Thereafter, reads and writes go directly to the raw LUN rather than going through the mapping file.
VM .vmx file is having reference pointing to RDM mapping file and not the LUN. For our example VM-01

scsi0:1.fileName = "VM-01_1.vmdk"
scsi0:1.mode = "persistent"
scsi0:1.ctkEnabled = "FALSE"
scsi0:1.deviceType = "scsi-hardDisk"
scsi0:1.present = "TRUE"
scsi0:1.redo = ""
scsi0:2.fileName = "VM-01_2.vmdk"
scsi0:2.mode = "independent-persistent"
scsi0:2.ctkEnabled = "FALSE"
scsi0:2.deviceType = "scsi-hardDisk"
scsi0:2.present = "TRUE"
scsi0:2.redo = ""

When to use?

Always VMFS is preferred over RDM. But there are some scenarios where RDM is required:

SAN-based features

- SAN snapshots: Using RDM enables backup software's to ‘see’ inside the LUN and to create a consistent snapshot of the data contained within the LUN. This allows for replication of data (snapshots) to a second SAN (on a different physical location) for Disaster Recovery. 
- In-band management software for your SAN running in Virtual Machines might require direct access to a management LUN. This usually requires physical compatibility mode.


In any MSCS clustering scenario that spans physical hosts cluster data and quorum disks should be configured as RDMs rather than as virtual disks on a shared VMFS.  The reason is that cluster members need to have access to shared LUN (quorum).

What are the available types?

RDM has two compatibility modes:

1. Virtual Mode 
In this mode all SCSI commands from the VM to LUN will pass through VMFS layer except READ & WRITE. Those will be passed directly to the LUN.

If you check the device manager of the VM you will see the RDM LUN listed as VMware Virtual Disk SCSI Disk Device. This is exactly the same as VMDK Drive added to the VM. The physical characteristics of the LUN are hidden from the VM.

The advantage of vMode is that you can still use Distributed Locking Mechanism of VMFS as well as snapshots of RDM LUN drives.

2. Physical Mode
In this mode all SCSI commands are passed to RDM LUN directly from the VM expect REPORT command virtualized by VMkernal.

If you check the device manager of the VM you will see the RDM LUN listed based on the physical name/characteristics of the storage array.

The advantage of the full exposure of the LUN characteristics to the VM can be seen when using SAN Management Tools and MSCS services. Also, RDM LUNs in physical mode can have a size greater than 2TB.
Another important feature inherited with RDM is Dynamic Name Resolution. Each LUN is having unique ID in RDM mapping file used by VMFS. In case the path to the LUN modified due to failover, for example, the ID of the LUN will be change. Dynamic Name Resolution feature will automatically discover the new name and associate it in the RDM mapping file to resume operations.


Here is a comparison between the types of disk drives available and the supported features.

RDM interoperatibility with vSphere Advanced Features
Both pRDM and vRDM support vSphere advnaced features such as HA, DRS, SDRS, VM, SVM. However there are some limitations for some features.

- Files for a virtual machine are not relocated when it is vMotioned. 
- The virtual machine is re-registered to the destination host. 
- Any RDMs remain as RDMs when the virtual machine is registered to another host. That is, no changes to the virtual machine itself are made.

Cold Migration
Cold migration can be for resources, datastores, or both while the VM is powered-off. In case migrating datastore is selected, here are some considerations for RDM:

- Any non-RDM virtual disks are physically moved to the destination. 
- The virtual machine configuration files are physically moved to the destination. 
- Raw LUNs themselves cannot be moved, as they are raw disks presented from the SAN. However, the pointer files (RDMs) can be relocated if required. 
- During the migration you can use the advanced section of the migration wizard and select if you would like to maintain the same format of the files at the destination (remain as RDM) or convert it to a thick/thin disk. 
- The virtual machine registration changes, but the files are left untouched.

When cloning a VM with RDMs, pRDM & vRDM contents will be copied into VMDK virtual disks. If you want to clone a virtual machine without its RDMs, remove them from the configuration of the virtual machine before cloning. You can delete the RDM from the disk when removing it (the raw LUN contents are not changed, only the RDM mapping file is deleted). Re-add them to the configuration when completed.

Storage vMotion
- When you perform Storage vMotion, the virtual machine files are physically relocated to a destination datastore. The same host retains ownership or registration of the virtual machine after Storage vMotion completes. 
- For vSphere 4.0 and higher: 
# Virtual disks and virtual mode RDM pointer files can be relocated to the destination datastore. 
# Virtual disks can be converted to thick-provisioned or thin-provisioned disks during migration as long as  the destination is not an NFS datastore. 
# Physical mode RDM pointer files can be relocated to the destination datastore, but physical mode RDM data cannot be migrated. 
# Virtual mode RDM data can be migrated with storage vMotion, provided sufficient licensing is present.


- In ESX 5.0, during the Storage vMotion, virtual mode RDMs can be converted to thick/thin provisioned disks by selecting the appropriate options under the Advanced section of the migration wizard. Physical mode RDMs can only be converted with a cold migration. 
- If you are attempting a Storage vMotion of a virtual mode RDM using the Advanced method and the RDM pointer mapping file is already present in the target datastore, the Storage vMotion finishes quickly, but without moving the data. This is because the Storage vMotion detects that the source and target datastores for the mapping file are the same, and therefore concludes that no movement is needed.

To work around this issue: 
1. Remove the RDM from the virtual machine, and delete it from disk to ensure the mapping file is deleted. This will not delete the data in the RDM. 
2. Add the RDM again, and put the mapping file in a different datastore than the ultimate target of the storage vMotion. 
3. After the virtual machine reconfiguration is complete, attempt the storage vMotion again.

RDM Restrictions

1. RDM can't be used with NFS datastore. 
2. SVM can't be used with NPIV

RDM Tips

In case RDM is greyed out:

1. Make sure that you have a free LUN which support RDM 
2. Make sure that RDMFiler Advanced parameter is unchecked

Note: In case a LUN is used by ESXi host (even if partially), you can't use it as RDM to VM and vice versa. The reason is that OS or Hypervisor will apply lock into the LUN which will prevent it to be used by other system.

After adding the RDM to VM, you don't need to format it (assuming that it was used on previous system and was previously formatted). You need ONLY to bring it online using disk management. In case of fresh new RDM assigned to VM, it needs to be formatted for first use.

Note: RDM isn’t support with vCloud and should be avoided.


  1. Very nice post about "what is an RDM" and clear difference between vRDM and pRDM!

  2. This is great, thank you.

  3. Hi,
    Absolutely great post!
    Maybe its a good idea to also cover the .rdmp files that are left in the datastore after erasing a LUN.
    I know many people ask questions about this issue.

  4. Will the block size of RDM be the same as the OS(4KB if NTFS) or will VMFS play a part and make it at 1MB?

    1. If your LUN was original formatted using NTFS (for example), the block size will remain 4KB when attached to VM as RDM. Even if you attach new LUN to windows VM as RDM and formatted the LUN from windows OS, the block size will be 4KB for NTFS formatting.

      In case of VMFS, the hypervisor will format the LUN to join a datastore. For RDM, the hypervisor won't see the LUN and will attach to directly to VM. It just uses RDM mapping file to point the VM to the LUN.

      Hope this clarified.

    2. This comment has been removed by the author.

  5. Hi MohammedI, have left doubts.

    1) pRDM vs VRDM

    With pRDM you can do vMotion -> Change host with VM on? (Hot vMotion)

    With VRDM you can do vMotion -> Change host with VM on? (Hot vMotion)

    2) Convert

    you can convert a disk pRDM in VRDM?

    Thanks for the info.

  6. Hi
    nice post
    I have an issue with a VM, with 2 hdisks (one a classical vmdk and one in virtual RDM mode)?
    They are both seen as vmware virtual disk SCSI on the windows side

    When I try to migrate with GUI (right click and then migrate ), trying to migrate config and hdisk 1 to datastore 1 and hdsk 2 (the vRDM to datastore 2), I have the message :
    The requested Storage VMotion would move a virtual machine's disks without assigning the virtual machine a new home, but such a move is not supported on the Source host
    I have a Vcenter 4 and unfortunatly some old ESX hosts in 3.5..

    My goal is to move the RDM to a vmdk as it is said in the manuals ..

    Any idea ?



  7. Hi Mohammed,

    We have been trying to attach a Samsung 840 SSD as an RDM for a Linux VM. The Raw Devices option is always disabled in vCenter during Add Hard Disk wizard, and attempts to manually create via CLI using vmfsktools -z or -r also fail.

    Any suggestions on how to troubleshoot?


  8. Hi Mohammed,
    I have deleted the VM from disk (Decomissioned) which had RDM mappings but missed to take a note of RDM details also missed to remove them from VM. Now I need those RDM details to share with storage person so that he can take that space from the SAN end & reassign those LUNs as RDM to cluster so that I can use them for another VM.
    Can you please let me know is there any way I can get RDM details of deleted VM?
    Your help would be much appreciated!
    Thanks in advance!

  9. Great article!! This is very important information for us.I like all content and information.I have read it.You know more about this please visit again.

    VMware Training in Chennai

  10. Good morning guys , Great Article .

    I am however concerned about the statement that says mapping file on VMFS is same size as backing RDM . I have seen the files in the past but it looks like a "fake" file . For example , are you saying if you have a 50TB RDM , you would need a VMFS volume that also has 50TB free . That sounds like a waste .

    Any comments welcome .

    Important Note: The size of the mapping file equals to the size of the LUN. Although RDM LUN is obtained directly from SAN, the same space of the LUN should be maintained in VMFS datastore due to mapping file.

    1. I did some checking . The pRDM.VMDK are indeed fake . The VMFS Volume containing the Symlink is only 1 TB , but it hosts a total of 6 TB of physical RDMs . So at least when dealing with Physical raw disks , in ESX55 , you do not require the same amount of Free Space on VMFS pointer data store .

      This VMFS store is 1 TB , see below up to 6TB Pointers >

      /vmfs/volumes/513dabd6-f5b30e38-fe6e-0025b50a008f/XXXX # ls -ltr
      total 0
      -rw------- 1 root root 2201282334720 Jun 5 2016 XXXX_5-rdmp.vmdk
      -rw------- 1 root root 2418939694080 Jun 5 2016 XXXX_6-rdmp.vmdk
      -rw------- 1 root root 118123246080 Jun 5 2016 XXXX_2-rdmp.vmdk
      -rw------- 1 root root 1649333145600 Jun 5 2016 XXXX_11-rdmp.vmdk
      -rw------- 1 root root 2287647774720 Jun 5 2016 XXXXL_1-rdmp.vmdk
      -rw------- 1 root root 499 Apr 7 18:53 XXXX_2.vmdk
      -rw------- 1 root root 501 Apr 7 18:54 XXXX_5.vmdk
      -rw------- 1 root root 501 Apr 7 19:35 0XXXX_1.vmdk
      -rw------- 1 root root 501 Apr 7 19:56 XXXX_6.vmdk

  11. Thanks so much for this information. I have to let you know I concur on several of the points you make here and others may require some further review, but I can see your viewpoint.
    vmware training uk

  12. Nice post.Thank you,I found the best resource to update my knowledge.Hope this information can help me to crack GIS jobs in Hyderabad.

  13. That's the contents i was looking from two days. Thanks for sharing

  14. Your very own commitment to getting the message throughout came to be rather powerful and have consistently enabled employees just like me to arrive at their desired goals.

    Java training in Chennai

    Java training in Bangalore

  15. After reading this web site I am very satisfied simply because this site is providing comprehensive knowledge for you to audience.
    Thank you to the perform as well as discuss anything incredibly important in my opinion. We loose time waiting for your next article writing in addition to I beg one to get back to pay a visit to our website in

    Selenium training in bangalore
    Selenium training in Chennai
    Selenium training in Bangalore
    Selenium training in Pune
    Selenium Online training

  16. The two most important features of any VRM device are the transmitter power and the beam angle. Power can range from 2 to 4 Kilowatts. The higher the power, the better your transmitter can see through fog and rain and the further the signal can reach, reference to here. Power is a factor in how well your radar does in bad weather. Blinding rain can cause low power marine radar to be less than effective.

  17. Great Article… I love to read your articles because your writing style is too good,
    its is very very helpful for all of us and I never get bored while reading your article because,
    they are becomes a more and more interesting from the starting lines until the end.
    Java training in Annanagar
    Java training in Chennai
    Java training in Chennai
    Java training in Electronic city
    Java training in Marathahalli

  18. I believe that your blog will surely help the readers who are really in need of this vital piece of information. Waiting for your updates.
    mobile service center in vadapalani
    mobile service center in porur
    best mobile service center

  19. There are lots of easy ways to raw device mapping. I like to get big pads of newsprint. You can usually find them with the kids art supplies or at an office supply store. I do mine in pencils so I can erase and move as I please. For that reason I also enjoy mapping with sticky notes on a wall or big window.

  20. It's Really Amazing and very informatic article. It's great one tips About Tech Knowleadge. It's really Helpful Post.

  21. This is really an awesome post, thanks for it. Keep adding more information to this.selenium training in bangalore

  22. Really it was an awesome article,very interesting to read.You have provided an nice article,Thanks for sharing.coded ui training in bangalore

  23. Being new to the blogging world I feel like there is still so much to learn. Your tips helped to clarify a few things for me as well as giving.big data testing training in bangalore

  24. Linking is very useful thing.you have really helped lots of people who visit blog and provide them use full information.mobile application testing training in bangalore

  25. Your articles really impressed for me,because of all information so nice.manual testing training in bangalore

  26. Thanks for sharing this blog. This very important and informative blog.web designing training in bangalore

  27. Such a great word which you use in your article and article is amazing knowledge. Thank you for sharing it.

    Became An Expert In UiPath Course ! Learn from experienced Trainers and get the knowledge to crack a coding interview, @Softgen Infotech Located in BTM.

  28. wonderful thanks for sharing an amazing idea. keep it...

    Softgen Infotech is the Best Oracle Training institute located in BTM Layout, Bangalore providing quality training with Realtime Trainers and 100% Job Assistance.

  29. Thank you so much for the great and very beneficial stuff that you have shared with the world.

    Become an Expert In Python Training in Bangalore ! The most trusted and trending Programming Language. Learn from experienced Trainers and get the knowledge to crack a coding interview, @Softgen Infotech Located in BTM Layout.

  30. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    servicenow online training

  31. Effective blog with a lot of information. I just Shared you the link below for Courses .They really provide good level of training and Placement,I just Had VMware Classes in this institute , Just Check This Link You can get it more information about the VMware course.

    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

  32. I believe that your blog will surely help the readers who are really in need of this vital piece of information. Waiting for your updates.
    AWS training in Chennai

    AWS Online Training in Chennai

    AWS training in Bangalore

    AWS training in Hyderabad

    AWS training in Coimbatore

    AWS training

  33. Good Post! , it was so good to read and useful to improve my knowledge as an updated one, keep blogging.After seeing your article I want to say that also a well-written article with some very good information which is very useful for the readers....thanks for sharing it and do share more posts likethis. https://www.3ritechnologies.com/course/salesforce-training-in-pune/