SPECstorage™ Solution 2020_ai_image Result

Copyright © 2016-2020 Standard Performance Evaluation Corporation

SPECstorage(TM) Subcommittee SPECstorage Solution 2020_ai_image = 190 AI_Jobs
Reference Submission in the Public Cloud Overall Response Time = 1.26 msec


Performance

Business
Metric
(AI_Jobs)
Average
Latency
(msec)
AI_Jobs
Ops/Sec
AI_Jobs
MB/Sec
100.6174350978
200.63187011955
300.649130512935
400.650174023912
500.673217524892
600.709261035868
700.790304546847
800.895348047823
900.971391558802
1001.078435059780
1101.1784785610759
1201.2455220611737
1301.3565655712716
1401.4326090813692
1501.6506525814668
1601.7806960915647
1702.1597395916624
1802.5397831017601
1904.3698240918560
Performance Graph


Product and Test Information

Reference Submission in the Public Cloud
Tested bySPECstorage(TM) Subcommittee
Hardware AvailableNovember 2020
Software AvailableOctober 2020
Date TestedNovember 2020
License Number4553
Licensee LocationsCampbell, California

Weka has developed a software-defined storage architecture that offers complete hardware independence while delivering an extremely performant solution that offers the flexibility of multiple protocol support. The design philosophy behind the Weka file system (WekaFS) was to create a single storage architecture, that runs on industry standard servers or on instances in the public cloud, with the performance of all-flash arrays, the simplicity and feature set of network-attached storage (NAS), and the scalability and economics of the cloud. At the heart of the design is WekaFS, a fully distributed, parallel file system that was written entirely from scratch to deliver the highest-performance file and object services leveraging NVMe flash as its primary storage tier between the application and the storage. Internally, and transparent to the application layer, the software seamlessly expands the namespace to a hard disk drive (HDD) layer built on any S3 compliant object storage system.

Solution Under Test Bill of Materials

Item NoQtyTypeVendorModel/NameDescription
18AWS EC2 InstancesAWSc5n.9xlargeAmazon C5n EC2 instances are ideal for high compute applications (including High Performance Computing (HPC) workloads, data lakes, and network appliances, c5n.9xlarge instances have 36 vCPU, 96GiB of memory, 50Gbps networking
28AWS EC2 InstancesAWSi3en.12xlargeAmazon i3en family of EC2 instances provides dense Non-Volatile Memory Express (NVMe)SSD instance storage optimized for low latency, high random I/O performance, high sequential disk throughput, i3en.12xlarge instances have 48 vCPU, 384GiB of memory, 4x7500GB NVMe SSDs, 50Gbps networking

Configuration Diagrams

  1. Solution Under Test

Component Software

Item NoComponentTypeName and VersionDescription
1WekaFSFile SystemVersion 3.8.3WekaFS is a parallel and distributed POSIX compliant file system that scales across NVMe enabled compute nodes, creating a single storage cluster that delivers high performance access to data. Each node in the cluster contains equal amount of data and metadata for optimal parallel file access and performance.
2Amazon Linux AMIOperating SystemVersion 2017.09The Amazon Linux AMI is a supported and maintained Linux image provided by Amazon Web Services for use on Amazon Elastic Compute Cloud (Amazon EC2).It is deployed on the 8 i3en.12xlarge storage nodes.
3Amazon Linux 2Operating SystemVersion 2017.09Amazon Linux 2 is the next generation of Amazon Linux, from Amazon Web Services (AWS). It is deployed on the 8 c5n.9xlarge compute nodes.

Hardware Configuration and Tuning - Virtual

Storage Nodes
Parameter NameValueDescription
SR-IOVEnabledEnables CPU virtualization technology

Hardware Configuration and Tuning Notes

SR-IOV was enabled on EC2 instance BIOS. No additional hardware tuning was required. The MTU is required and valid for all environments.

Software Configuration and Tuning - Virtual

Component Name
Parameter NameValueDescription
Jumbo Frames9001Enables jumbo frames up to 9001 bytes of Ethernet frames

Software Configuration and Tuning Notes

None

Service SLA Notes

AWS uses commercially reasonable efforts to make the Included Products and Services each available with a Monthly Uptime Percentage of at least 99.99%, in each case during any monthly billing cycle. (Monthly Uptime Percentage) is calculated by subtracting from 100% the percentage of minutes during the month in which any of the Included Products and Services, as applicable, was in the state of "Region Unavailable".

Storage and Filesystems

Item NoDescriptionData ProtectionStable StorageQty
1Amazon EC2 i3en.12xlarge instances with WekaFS as a shared file system, each with 4 x 7,500GB NVMe SSDs4+2Yes8
Number of Filesystems1
Total Capacity98 TiB
Filesystem TypeWekaFS

Filesystem Creation Notes

WekaFS is deployed on Amazon through a cloud formation template that can be launched from Marketplace. The Weka software is installed and the file system is configured as part of the cloud formation process. No additional file system creation steps are required.

Storage and Filesystem Notes

None

Transport Configuration - Virtual

Item NoTransport TypeNumber of Ports UsedNotes
150Gbps Ethernet Virtual NIC16Virtual network

Transport Configuration Notes

None

Switches - Virtual

Item NoSwitch NameSwitch TypeTotal Port CountUsed Port CountNotes
1AWS50Gbs Ethernet with Enhanced Networking1616Virtual network

Processing Elements - Virtual

Item NoQtyTypeLocationDescriptionProcessing Function
1384vCPUi3en.12xlarge storage instances3.1 GHz all core turbo Intel® Xeon® Scalable (Skylake) processorsWekaFS, RAID, Network communication, Storage functions
2288vCPUc5n.9xlarge client instances3.0 GHz Intel® Xeon® Platinum processorsWekaFS Client and benchmark

Processing Element Notes

None

Memory - Virtual

DescriptionSize in GiBNumber of InstancesNonvolatileTotal GiB
AWS EC2 i3en.12xlarge instance memory3848V3072
AWS EC2 c5n.9xlarge instance memory968V768
Grand Total Memory Gibibytes3840

Memory Notes

None

Stable Storage

WekaFS uses the EC2 instances which provide stable storage. The data is journeled as well as UPS protected across multiple layers of the AWS infrastructure.

Solution Under Test Configuration Notes

The solution under test was a standard WekaFS enabled cluster in dedicated server mode. The solution will handle both large file I/O as well as small file random I/O and metadata intensive applications. No specialized tuning is required for different or mixed use workloads. None of the components used to perform the test were patched with Spectre or Meltdown patches

Other Solution Notes

None

Dataflow

None

Other Notes

None

Other Report Notes

None


Generated on Thu Dec 17 10:40:40 2020 by SpecReport
Copyright © 2016-2020 Standard Performance Evaluation Corporation