[clug] Why virtual x86 machines?

[Stephen Hocking]

Well, you don't need a Xeon to run virtual machines - most desktop
processors are quite happy to. I'm running a few VMs on a Ryzen 7 over
in my rack, a couple on my macbook pro, some more on my mac mini more
on various Linux laptops. They're a great way of prototyping stuff and
then being able to throw it away. The snapshot ability of various
hypervisors makes that method of experimentation quite easy.

On Thu, 20 Aug 2020 at 21:09, Hugh Fisher via linux
<linux at lists.samba.org> wrote:
>
> Inspired by the questions about KVM, I've been doing some reading on
> virtual machines and containers and some of the other new abstraction
> & protection mechanisms being used today. I like to write things down
> to clarify my thinking, and am posting this to the list in the hope
> that people with more knowledge will correct me if I'm wrong. And I do
> have questions, at the end.
>
> First up I'm not including the Java Virtual Machine, or the similar
> bytecode like systems used in .NET, Python, etc. Those are designed
> for user level programs, not OS kernels. And I'm not including
> emulation/simulation where machine instructions are interpreted by
> another program, because then it's turtles all the way down. A 6502
> Apple II running ProDOS can be emulated by a program on a M68030
> Macintosh running System 7 which itself is being emulated by a program
> running on a PowerPC Macintosh running MacOS X ...
>
> So, a virtual machine, usually associated with a hypervisor and guest
> operating system kernels, executes as many as possible machine
> instructions on the actual CPU hardware. (Using the old definition
> that you can kick hardware, but only swear at software. And just skip
> over microcode.)
>
> From my old Andy Tanenbaum textbook the first virtual machine in
> widespread use was VM/370 for IBM mainframes, around 1970. I think the
> history is important because of a question I'll bring up later.
>
> A 370 series IBM mainframe, ancestor of the backwardly compatible zOS
> mainframes still sold today, could easily cost a million dollars. A
> 370 mainframe would run an entire bank financial system, or an entire
> airline reservation network. Which was awkward if a new release of the
> operating system was due and you wanted to test that all your software
> would still work. Shut down everything while you reboot into a beta
> OS? Buy another million dollar mainframe just for testing?
>
> VM/370 was what today we call a hypervisor, that could run multiple
> guest operating systems side by side on a single CPU, providing each
> operating system its own "virtual 370". Now the bank could run VM/370
> on its single mainframe, with say 90% of machine resources allocated
> to the guest production OS and the rest given to whatever the
> developers wanted.
>
> This was a major technical achievement. Then, like now, the operating
> system distinguished 'user mode' from 'kernel' or 'privileged' or
> 'system' mode. User mode machine instructions could not modify virtual
> memory page tables, issue DMA instructions to IO hardware, and so on.
> Only kernel code could do that. So unlike a regular operating system
> the hypervisor had to work with guest operating system kernels
> executing privileged machine instructions. The guest kernels didn't
> know that they were running on a virtual 370, so it was up to the
> hypervisor to ensure that if, say, one guest OS disabled interrupts,
> this wouldn't shut down every other guest.
>
> Once IBM got VM/370 to work, it was a big hit. It was so popular both
> inside and outside IBM that some new instructions and microcode
> modifications were added to the 370 machine architecture to make IO
> and memory paging within the guest operating systems more efficient.
>
> And IBM then developed CMS, a hypervisor-aware operating system kernel
> designed to run only on VM/370. A conventional OS protects multiple
> users from affecting each other, whether deliberate or accidental. CMS
> was a single user OS, and VM/370 gave every user their own copy on
> their own virtual 370. Even if there was a kernel exploit in the CMS
> operating system (not the hypervisor), the only person you could
> attack would be yourself. CMS was a smaller and simpler operating
> system because it didn't duplicate functions that VM/370 was already
> doing.
>
> Now fast forward to the 21st century. If you
>     cat /proc/cpuinfo
> on an x86 Linux system and you see 'vmx' in the output, you have the
> Intel virtual machine hardware extensions. The original x86
> architecture had Ring 0 for privileged machine instructions as used by
> operating system kernels. The virtual hardware extensions add Ring -1
> for a hypervisor such as VMWare, which can run multiple guest Linux or
> MS Win kernels side by side. Each of these thinks it is running with
> Ring 0 privilege and can update page tables, issue IO instructions to
> PCI slots or disk controllers, and so on.
>
> So Intel virtual x86 is just like VM/370. Except ... x86 computers
> don't cost a million dollars.
>
> So my most important question, why bother? Just buy another CPU.
>
> I did a quick price comparison on www.mwave.com.au. The cheapest Intel
> Xeon is about $4,000 and it's possible to spend $14,000 if you want
> to. For those amounts of money you could buy a shoebox up to shipping
> container full of Raspberry Pis, complete 64 bit Ghz systems with RAM
> and ports. Or if you have to stay within the x86 family, Intel
> Celerons are at least five times cheaper than Xeons. Looking instead
> at power budget, the cheapest Xeon CPU consumes as many watts as five
> entire Raspberry Pis.
>
> Looking at these prices I understand why Intel want us to virtualise
> x86 CPUs and run multiple guest operating systems. I don't see why
> anyone else would want to.
>
> But since datacentres and cloud systems do use hypervisors I must be
> missing something. Anyone want to explain?
>
>
> Second question, are there custom Linux kernels designed to run on
> hypervisors? Not a Container OS, which I think is something else, but
> like CMS designed to be single user or otherwise not duplicate what
> the hypervisor is already doing?
>
>
> And lastly I'm assuming that there's nothing in virtual x86 design and
> implementation that VM/370 didn't already do. Am I wrong? What new and
> interesting uses for hypervisors have been thought of?
>
>
> --
>
>         cheers,
>         Hugh Fisher
>
> --
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux



-- 

  "I and the public know
  what all schoolchildren learn
  Those to whom evil is done
  Do evil in return" W.H. Auden, "September 1, 1939"