blog posts

What is NVMe?

NVMe (non-volatile memory express) is a host controller interface and storage protocol created to accelerate the transfer speed of data. Between enterprise and client systems and solid-state drives over a computer’s high-speed Peripheral Component Interconnect Express bus.

So NVMexpress is commonly use for solid-state storage, main memory, cache memory or backup memory. It provides an alternative to the Small Computer System Interface (SCSI) standard and the Advanced Technology Attachment (ATA) . Standard for connecting and transmitting data between a host system and a target storage device. NVMe  designed for use with faster media. Also The main advantages of NVMe-based PCIe SSDs over other storage types are the reduced latency and higher input/output operations per second (IOPS).

As solid-state technology became the preferred medium in the storage market. It quickly became clear that existing interfaces and protocols — such as Serial Advanced Technology Attachment (SATA) and Serial-Attached SCSI (SAS) — were no longer suitable in data center environments. In early 2011, the initial NVMe spec released. Nearly 100 tech companies  involved in its development.

NVMexpress is also a key enabler of evolving technologies and applications such as the internet of things, artificial intelligence and machine learning, which can all benefit from the low latency and high-performance improvements provided by NVMe-attached storage.

How does NVMexpress work?

The NVMexpress standard does not state NVMe’s main use; however, it does define a register interface, command set and collection of features for PCIe-based SSDs with the goals of high performance and interoperability across a broad range of NVM systems.

The NVMexpress protocol can support any form of non-volatile memory. Such as SSDs that use different types of non-volatile memory, including NAND flash. NVMe reference drivers are available for a variety of operating systems, including Windows and Linux.

In addition An NVMe SSD connects through a PCIe bus or M.2 and U.2 connectors. The NVMexpress protocol, attached with these connections, enables lower latency and higher IOPS along with a reduction in power use.

NVMe commands map input/output (I/O) and responses to shared memory in a host computer over a PCIe interface. The NVMexpress interface supports parallel I/O with multicore processors to facilitate high throughput and address central processing unit (CPU) bottlenecks.

Why is NVMe important?

Designed for high-performance non-volatile storage media such as SSDs, NVMe is well-suited for highly demanding, compute-intensive settings. For example, NVMe can handle enterprise workloads while leaving a smaller infrastructure footprint and consuming less power.

By comparison, there is a large performance difference between NVMexpress and SATA. For example, NVMexpress experiences much less latency than SAS and SATA protocols. This increase in performance and reduced latency means that NVMe can be used with workload-intensive applications that need real-time processing while avoiding bottlenecks.

Enterprise environments and data centers can take advantage of the high-performance granted by NVMexpress-based storage.

What are NVMe’s use cases?

Because of the blistering performance for flash storage NVMexpress provides, along with its ability to handle a high number of queues and commands, NVMexpress is suitable for:

• Professional and prosumer use, handling tasks such as graphics editing.
• Applications with large queue depths for storage I/O, including databases and some web operations.
• High-performance computing. Specifically, in applications where low latency is critical.
• Areas that require the storage of large amounts of data, such as in AI, machine learning, advanced analytics and big data.
• Relational databases. The better performance of NVMexpress flash memory systems lowers the number of physical servers needed.
• Applications that need to retrieve or store data in real time, such as finance and e-commerce apps.

What are the benefits of NVMe?

Benefits of NVMexpress include:

• NVMe drives can send commands twice as fast compared to AHCI SATA drives.
• NVMe SSDs have a latency of only a few microseconds, while SATA SSDs have latency between 30 and 100 microseconds.
• Efficient storage, management and data access.
• Has a much higher bandwidth compared to SATA and SAS.
• Supports multiple form factors, including M.2, U.2 and connections.
• Data can be divided and streamlined.
• Supports tunneling protocols, which address privacy.

What are the ِDisadvantages of NVMe?

Potential drawbacks include:

• A lack of support for NVMe on legacy systems.
• Not cost-effective at storing large volumes of data. It is more expensive compare to spinning drives based on storage capacity.
• It is typically use with the M.2 format, which may further limit drive selection.

History and evolution of NVM Express

The Non-Volatile Memory Host Controller Interface (NVMHCI) Workgroup began developing the NVMe specification in 2009 and published the 1.0 version on March 1, 2011. This specification included the queuing interface, the NVM command set.

Administration command set and security features. Other noteworthy dates include:

• On Oct. 11, 2012, the NVMHCI Workgroup, commonly known as the NVM Express Workgroup, released NVMe 1.1, which added support for SSDs with multiple PCIe ports to enable multipath I/O and namespace Other new capabilities included autonomous power state transitions during idle time to reduce energy needs and reservations, meaning two or more hosts could coordinate access to a shared namespace to improve fault tolerance.
• In May 2013, the NVM Express Workgroup held its first Plugfest to enable companies to test their products’ compliance to the NVMe specification and to check interoperability with other NVMe products.
• In March 2014, the NVM Express Workgroup  incorporated under the NVM Express organization name. The group later became known simply as NVM Express Inc. The nonprofit organization has more than 100 technology member companies.
• On Nov. 17, 2015, the NVM Express organization ratified the 1.0 version of the NVM Express Management Interface (NVMe-MI) to provide an architecture and command set to manage a non-volatile memory subsystem out of band. NVMe-MI enables a management controller to perform tasks such as SSD device and capability discovery, health and temperature monitoring, and non-disruptive firmware updates. Without NVMe-MI, IT managers generally relied on proprietary, vendor-specific management interfaces to enable the administration of PCIe SSDs.
• In June 2017, NVM Express released NVMe 1.3. Highlights centered on sanitize operations, a new framework known as Directives, and virtualization
• In 2020, the NVMe Zoned Namespace Command Set Specification was ratified. This specification enables NVM to isolate and evolve command sets for emerging technologies, including Zoned Namespaces, Key Value and computational storage.
• In June 2021, NVMe 2.0  released. The specification  restructured to enable faster and easier development of NVm and to support a more diverse NVMe environment.

NVMe form factors and standards

The need for a storage interface and protocol to better exploit NAND flash’s performance potential in enterprise environments was the driving force behind the development of NVMe. But reimagining the connection standard opened the doors to several different types of interface implementations that could stay within the bounds of the new spec while offering a variety of implementation options.

In short order, a number of flash form factors conforming to NVMe specifications emerged, including the new form factors for SSDs dubbed M.2 and U.2.

• AIC. The AIC form factor enables manufacturers to create their own cards that fit into the PCIe bus without worrying about storage bay designs or similar limitations. The cards are often designed for special use cases and may include additional processors and other chips to enhance the performance of the solid-state storage.
• M.2. The M.2 form factor was developed to take advantage of NAND flash’s compact size and low heat discharge. As such, M.2 NVMexpress devices are not intended to fit into traditional drive bay compartments, but rather to be deployed in much smaller spaces. Often described as about the size of a stick of gum, M.2 SSDs measure 22 mm wide and about 80 mm long. Although some products may be longer or shorter.
• U.2. Unlike the M.2 form factor, U.2 SSDs were designed to fit into existing storage bays originally intended for standard SATA or SAS devices. U.2 SSDs look similar to older media, as they typically use the 2.5-inch or 3.5-inch enclosures that are familiar housings for HDDs. The idea, of course, was to make it as easy as possible to implement NVMe technology with as little reengineering as possible.
• EDSFF. Another, less widely deployed NVMe form factor is enterprise and data center SSD form factor. The goal of EDSFF is to bring higher performance and capacities to enterprise-class storage systems. Perhaps the best-known example of EDSFF flash is Intel’s E1.L long and E1.S short flash devices, which are provided in what was originally referred to as the “ruler” form factor.

NVMe over Fabrics

NVM Express Inc. published version 1.0 of the NVMexpress over Fabrics (NVMe-oF) specification on June 5, 2016. NVMe-oF is designed to extend the high-performance and low-latency benefits of NVMe across network fabrics.That connect servers and storage systems. The NVMe-oF 1.1 specification, which was released in 2019. Offers improved fabric communication, finer grain I/O resource management, and end-to-end flow control and support for NVMe/TCP. Both specifications also offer lower latency, improved management and provisioning of flash and remote storage access.

Fabric transports include NVMe-oF using remote direct memory access (RDMA) and NVMexpress-oF mapped to FC. A technical subgroup of NVM Express Inc. worked on NVMe-oF with RDMA, while the T11 committee of the International Committee for Information Technology Standards (INCITS) developed NVMexpressover FC (FC-NVMexpress).

The NVMe-oF specification is largely the same as the NVMe specification. One of the main differences between them is the methodology for transmitting and receiving commands and responses. NVMe is design for local use and maps commands and responses to a computer’s shared memory via PCIe. By contrast, NVM over Fabrics employs a message-based system to communicate between the host computer and target storage device.

The stated design goal for NVMexpress-oF was to add no more than 10 microseconds of latency for communication between an nvm host computer and a network-connected NVMe storage device, in comparison to the latency associated with an NVMe storage device using a local computer’s PCIe bus.