Maysastor First Impressions

Update (2022-8-6): I’ve added some information on the rather underwhelming performance of iSCSI on a 10 year old low-end Synology NAS.

At home, I recently added a Minisforum HX90 to my home collection, and added it to my existing Kubernetes cluster (a hodgepodge of several things). I decided I’d stop using a Synology NAS (via iSCSI) for storage, and evaluate a a more “native” approach. In this case, I decided to experiment with Mayastor , which is a CNCF project and part of OpenEBS. One of it’s main goals is to be a low-overhead but flexible abstraction.

Installation was super easy using Helm. One thing that isn’t obvious from the docs is that the design of the agent (which runs on every tainted node) will burn an entire core. The reason for this is to ensure the lowest latency, which makes some sense.

In addition to the normal mayastor (single) mayastor-3 (triple replica) StorageClass , I added a duplex replica:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mayastor-2
parameters:
  ioTimeout: "60"
  local: "true"
  protocol: nvmf
  repl: "2"
provisioner: io.openebs.csi-mayastor
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

Then, using some guidance on the website, I ran some benchmarks on my specific setup. In this case, the backing store is a NVMe PCIe 3.0 Samsung Pro 970 SSD. The results are a bit confusing , unfortunately:

To put it in context, I’ve also included the iSCSI performance using the support in the Synology DS413 that I have been using forever as an internal file server. I think it has a dual-core 1Ghz MIPS CPU in it, so it’s not fast.

What I find curious is that I really expected there to be a less than 60% hit on bandwidth when using Mayastor. Interestingly, and quite curious as well, simplex and duplex replicas had nearly identical behaviors.

Future digging, but for now the performance “fast enough” for what I poke around at.