DEV Community

Discussion on: live migrating VM's - cool tech for the future, that you should avoid using today (Review & Benchmark of GCP E2 Instances)

Collapse
 
tytso profile image
Theodore Ts'o

(Disclaimer, I work for Google, but these comments are my own observations.)

For what it's worth, there are workloads where e2 is a win --- where using an e2 VM is faster than n1 and cheaper to boot. Using gce-xfstests[1], which is a kernel regression test suite runner for file systems. It's mostly disk I/O bound, but it does have enough CPU that e2 was a win.

[1] thunk.org/gce-xfstests

Command-line: gce-xfstests -c ext4/4k -g auto --machtype

GCE MachType    Time        Cost              Savings
n1-standard-2       6178s     20.5 cents        ---
e2-standard-2       5942s     15.0 cents       26.8%
n2-standard-2       5513s     18.6 cents        9.3%

(Cost includes a 10GB PD-Standard OS disk and 100GB PD-SSD test disk.)

Note that for this particular use case, the sustained use discount isn't applicable, because it's not something which is continuously running. I'll make changes to the kernel and either run the auto group, which takes a bit over an hour and a half, or run the quick group, which will give me results in around 20 minutes.

5 cents savings per run might not seem like a big deal, but I tend to run a lot of tests, since a command like "gce-xfstests smoke" plucks the built kernel from my sources, launches a test appliance VM, and then results will land in my inbox a short while later.

For more complete testing, (especially before I send a pull request upstream), I'll use the Lightweight Gce-xfstests Test Manager (or LGTM) to automatically launch 11 VM's to run the auto group in a large number of different fs configurations. So a typical test VM is only going to be running for 2 hours (plus or minus), and so it's very bursty. But isn't that one of the best ways to use the public cloud?

Example e-mail'ed report from gce-xfstests using the LGTM runner:

TESTRUNID: ltm-20200416014919
KERNEL:    kernel 5.7.0-rc1-xfstests-00045-g60480efd3f37 #1663 SMP Thu Apr 16 00:53:05 EDT 2020 x86_64
CMDLINE:   full --kernel gs://gce-xfstests/bzImage
CPUS:      2
MEM:       7680

ext4/4k: 490 tests, 36 skipped, 5983 seconds
ext4/1k: 494 tests, 2 failures, 44 skipped, 6362 seconds
  Failures: generic/383 generic/476
ext4/ext3: 559 tests, 111 skipped, 8435 seconds
ext4/encrypt: 564 tests, 127 skipped, 3296 seconds
ext4/nojournal: 549 tests, 2 failures, 99 skipped, 5000 seconds
  Failures: ext4/301 ext4/304
ext4/ext3conv: 488 tests, 36 skipped, 5385 seconds
ext4/adv: 495 tests, 5 failures, 43 skipped, 5365 seconds
  Failures: generic/083 generic/269 generic/475 generic/476
    generic/477
ext4/dioread_nolock: 488 tests, 36 skipped, 5287 seconds
ext4/data_journal: 539 tests, 2 failures, 89 skipped, 5379 seconds
  Failures: generic/371 generic/475
ext4/bigalloc: 471 tests, 43 skipped, 6981 seconds
ext4/bigalloc_1k: 480 tests, 1 failures, 54 skipped, 4794 seconds
  Failures: generic/383 
Totals: 4899 tests, 718 skipped, 12 failures, 0 errors, 61865s

FSTESTIMG: gce-xfstests/xfstests-202004131938
FSTESTPRJ: gce-xfstests
FSTESTVER: blktests cd11d00 (Sat, 7 Mar 2020 13:56:25 -0800)
FSTESTVER: e2fsprogs v1.45.6 (Sat, 21 Mar 2020 00:24:04 -0400)
FSTESTVER: fio  fio-3.19 (Thu, 12 Mar 2020 11:12:50 -0600)
FSTESTVER: fsverity v1.0 (Wed, 6 Nov 2019 10:35:02 -0800)
FSTESTVER: ima-evm-utils v1.2 (Fri, 26 Jul 2019 07:42:17 -0400)
FSTESTVER: nvme-cli v1.10.1 (Tue, 7 Jan 2020 13:55:21 -0700)
FSTESTVER: quota  9a001cc (Tue, 5 Nov 2019 16:12:59 +0100)
FSTESTVER: util-linux v2.35.1 (Fri, 31 Jan 2020 10:22:38 +0100)
FSTESTVER: xfsprogs v5.5.0 (Fri, 13 Mar 2020 10:53:45 -0400)
FSTESTVER: xfstests-bld a7c948e (Mon, 13 Apr 2020 18:33:19 -0400)
FSTESTVER: xfstests linux-v3.8-2717-g8c2da11c (Sun, 22 Mar 2020 14:33:53 -0400)
FSTESTSET: -g auto
FSTESTOPT: aex