Thanks! We joined the Storage WG and will work with Ben on CSI and future projects.

The use case was running Rook Block storage on-top of ephemeral/instance storage on EC2 instances vs. using EBS storage. Rook would handle the replication of data across instances and stripe across them for performance. Pods in the cluster would see this like any other volume.

For Pod failover, the detach / detach cycle is much faster than EBS. One of our users compared EBS to Rook [1] and showed that Rook volume failover happened in less than minutes vs. up to an hour with EBS.

Also EBS volumes only support a single writer (ReadWriteOnce in K8S) which makes them a poor candidate for hot failover scenarios underneath, say, Postgres or MySql. With the work we’re doing on the Rook Volume Plugin [2] we plan to support ReadWriteMany to support a hotter failover where the app/service ontop can handle the fencing.

Finally, there are cost and performance tradeoffs for running on-top of ephemeral/instance storage vs. EBS. For example, a lot of the instance storage is unused in most deployments and has a high performance.

Happy to discuss in more detail.

Thanks!

Bassam

[1] https://gitter.im/rook/rook?at=58baff6f872fc8ce62b6ee26

[2] https://github.com/kubernetes/kubernetes/pull/46843