July 15, 2015

Storage I/O Control (SIOC)


VMware vSphere Storage I/O Control (SIOC) provides I/O prioritization of virtual machines running on a cluster of ESXi hosts with access to shared storage. It extends the constructs of shares and limits, which existed for CPU and memory, to manage storage utilization.

Use SIOC to configure rules and policies to specify the business priority of each virtual machine using shares and limits. When I/O congestion is detected, Storage I/O Control dynamically allocates the available I/O resources to virtual machines according to the rules and policies, improving service levels and consolidation ratios.

At a basic level SIOC is monitoring the end to end latency of a datastore. When there is congestion, SIOC reduces the latency by throttling back virtual machines that are using excessive I/O. SIOC will use the share values assigned to the virtual machine’s VMDKs to prioritize access to the datastore.

The purpose of SIOC is to address the noisy neighbor problem, i.e. a low priority virtual machine impacting other higher priority virtual machines due to the nature of the application and the I/O characteristics of the low priority VM.

SIOC allows vSphere administrators to assign relative priority to storage I/O as well as assign storage I/O limits to VMs.

Once enabled for a specific datastore, SIOC will monitor the datastore, summing up the disk shares for each of its VMDK files. When an ESXi host detects storage congestion through an increase of latency beyond a user-configured threshold, it will apply the settings configured for that VM.

SIOC resolves imbalances by limiting the amount of I/O operations a host can issue for a datastore. The result is that VMware administrators can ensure that VMs that need priority access to storage resources get the resources they need.

Storage I/O Control (SIOC) requires Enterprise Plus licensing..


Universally Unique Identifier (UUID)


A Universally Unique IDentifier (UUID) is a 16-octet (128-bit) number. In its canonical form, a UUID is represented by 32 lowercase hexadecimal digits, displayed in five groups individually separated by hyphens, in the form 8-4-4-4-12. In general, UUID is used to uniquely identify an object or entity on the Internet.

VMware storage architecture has multiple, unique identifiers:
  • NAA & EUI (most common):
    • Network Address Authority  & Extended Unique Identifier
    • Guaranteed to be unique to the LUN
    • The preferred method of identifying LUNs
    • Generated by the storage device
  • MPX  (local datastores):
    • For devices that do not provide an NAA number, ESXi generates an MPX identifier
    • Represents the local LUN or disk
    • Takes the form mpx.vmhba<Adapter>:C<Channel>:T<Target>:L<LUN>, e.g. mpx.vmhba33:C0:T1:L0
    • Can be used in the exact same way as the NAA identifier
  • VML: Can be used interchangeably with the NAA identifier and the MPX identifier
    • Generally used for operations with utilities such as vmkfstools
  • Path identifiers, e.g. vmhba1:C0:T0:L1
    • Used exclusively to identify a path to the LUN
    • Generally used for operations with utilities such as vmkfstools
  • Target identifiers, e.g. fc.200100e08ba5ff63:210100e08ba5ff63
We are now adding UUID – Universally Unique IDentifier to the list.

"In addition to being universally unique, no centralized authority is required to administer them."

In vSphere, the UUID is a 16-octet (128-bit) number. It is represented as 16-hexidecimal number pairs. The 16 bytes of this value are separated by spaces, except for a dash between the eighth and ninth hexadecimal pairs.

An example UUID looks like this:
uuid.bios = "00 11 22 33 44 55 66 77-88 99 aa bb cc dd ee ff"

VMware uses the UUID to generate a MAC address for a VM.

The UUID value is based on the physical host’s System Management BIOS (SMBIOS) and the path to the virtual machine’s configuration (.vmx) file.

The UUID is stored in the SMBIOS system information (the BIOS of the VM) descriptor and in the file system superblock.

Within the VMX configuration file the UUID information is stored in three variables: uuid.bios, uuid.location and vc.uuid.
  • uuid.bios
    • globally unique identifier
    • generated when a VM is powered on or reset
  • uuid.location
    • hash based on the current path of the VM
    • generated whenever the VM is migrated
  • vc.uuid
    • used by vCenter to identify VM
    • generated when you add VM to inventory (or create VM
The UUID value must be surrounded by quotation marks. A sample configuration is shown below:
Associating a UUID with a virtual machine allows that virtual machine to be uniquely identified even if its network configuration is changed.

Note: VMware uses the form 8-8-4-12 instead of the formal 8-4-4-4-12 form.


SCSI Reservations & Atomic Test and Set (ATS)

SCSI Reservations

SCSI reservations are used to control access to a shared SCSI device such as a LUN. An initiator or host sets a reservation/lock on a LUN in order to prevent another host from making changes to it. This is similar to the file-locking concept.

A SCSI reservation conflict occurs if a host tries to access a datastore that was locked by another host.

A Logical Unit Number (LUN) is an individual, unique, block-based storage device, the term LUN is often used interchangeably with disk and datastore, depending on the context.

In a shared storage environment, when multiple hosts access the same Virtual Machine File System (VMFS) datastore, specific locking mechanisms are used. These locking mechanisms prevent multiple hosts from concurrently writing to the metadata and ensure that no data corruption occurs.

SCSI Reservations
SCSI reservation is a technique that manages disk contention by preventing I/O on an entire LUN for any ESXi host or VMs (other than the host/VM issuing the SCSI reservation). It is important not to have frequent SCSI reservations, as this could hinder performance.

VMFS uses SCSI reservations to give it exclusive access when it performs certain VMFS operations that modify the metadata of a datastore. Examples of VMFS operations include creating a virtual machine or template, turning a VM on, or creating or deleting a file. The host releases the lock once the operation is complete.

SCSI reservations lock an entire storage device while an operation that requires metadata protection is performed. After the operation completes, VMFS releases the reservation and other operations can continue. Because this lock is exclusive, excessive SCSI reservations by a host can cause performance degradation on other hosts that are accessing the same VMFS.

There are two main categories of operation under which VMFS makes use of SCSI reservations.
  • Category 1: is for VMFS data-store level operations. These include opening, creating, resignaturing, and expanding/extending of VMFS data-store.
  • Category 2: involves the acquisition of locks. These are locks related to VMFS specific metadata (called cluster locks) and locks related to files (including directories).
These are some examples of VMFS operations that require locking metadata:
  • Creating a VMFS datastore
  • Expanding a VMFS datastore onto additional extents
  • Powering on a virtual machine
  • Acquiring a lock on a file
  • Creating or deleting a file
  • Creating a template
  • Deploying a virtual machine from a template
  • Creating a new virtual machine
  • Migrating a virtual machine with vMotion
  • Growing a file, for example, a snapshot file or a thin provisioned virtual disk
  • For the zeroed thick type of virtual disk the reservation is required only when zeroing the blocks.
VMFS uses SCSI reservations on storage devices that do not support hardware acceleration.

Atomic Test and Set (ATS)
The Atomic Test and Set (ATS) primitive is used for locking on VMFS datastores for VMware vSphere Storage APIs for Array Integration (VAAI) compatible storage arrays. It is superior to the SCSI Reservation locking technique.

VMFS Locking Mechanisms
VMFS supports SCSI reservations and ATS locking. At vSphere 6 and depending on its configuration and the type of underlying storage, a VMFS datastore can exclusively use the ATS locking mechanism (ATS-only), or use a combination of ATS and SCSI reservations (ATS+SCSI).

For storage devices that support hardware acceleration, VMFS uses the ATS algorithm, also called hardware assisted locking. This reduces SCSI reservations and enables the ESXi host to integrate with compliant storage arrays and unload locking activities directly to the storage array controller. Unlike with SCSI reservations, ATS supports discrete locking per disk sector.
  • ATS-Only Mechanism
    For compliant storage devices (i.e. those that support T10 standard-based VAAI specifications), VMFS provides ATS locking, also called hardware assisted locking.

    The ATS algorithm supports discrete locking per disk sector. This is in contrast with SCSI reservation which locks an entire storage device while an operation that requires metadata protection is performed.

    After the operation completes, VMFS releases the reservation and other operations can continue.

    All newly formatted VMFS5 datastores use the ATS-only mechanism if the underlying storage supports it, and never use SCSI reservations.

  • ATS+SCSI Mechanism
    A VMFS datastore that supports the ATS+SCSI mechanism is configured to use ATS and attempts to use it when possible. If ATS fails, the VMFS datastore reverts to SCSI reservations.

    Datastores that use the ATS+SCSI mechanism include VMFS5 datastores that were upgraded from VMFS3 and new VMFS5 datastores on storage devices that do not support ATS.
With the storage hardware assistance, your host performs these operations faster and consumes less CPU, memory, and storage fabric bandwidth.