Update (2022-8-6): I’ve added some information on the rather underwhelming performance of iSCSI on a 10 year old low-end Synology NAS.
At home, I recently added a Minisforum HX90 to my home collection, and added it to my existing Kubernetes cluster (a hodgepodge of several things). I decided I’d stop using a Synology NAS (via iSCSI) for storage, and evaluate a a more “native” approach. In this case, I decided to experiment with Mayastor , which is a CNCF project and part of OpenEBS. One of it’s main goals is to be a low-overhead but flexible abstraction.
Installation was super easy using Helm. One thing that isn’t obvious from the docs is that the design of the agent (which runs on every tainted node) will burn an entire core. The reason for this is to ensure the lowest latency, which makes some sense.
In addition to the normal
mayastor-3 (triple replica)
added a duplex replica:
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: mayastor-2 parameters: ioTimeout: "60" local: "true" protocol: nvmf repl: "2" provisioner: io.openebs.csi-mayastor reclaimPolicy: Delete volumeBindingMode: WaitForFirstConsumer
Then, using some guidance on the website, I ran some benchmarks on my specific setup. In this case, the backing store is a NVMe PCIe 3.0 Samsung Pro 970 SSD. The results are a bit confusing , unfortunately:
|Configuration||Read BW MB/s||Write BW MB/s|
|iSCSI against Synology DS413||1.8||1.825|
To put it in context, I’ve also included the iSCSI performance using the support in the Synology DS413 that I have been using forever as an internal file server. I think it has a dual-core 1Ghz MIPS CPU in it, so it’s not fast.
What I find curious is that I really expected there to be a less than 60% hit on bandwidth when using Mayastor. Interestingly, and quite curious as well, simplex and duplex replicas had nearly identical behaviors.
Future digging, but for now the performance “fast enough” for what I poke around at.