Wednesday, September 3, 2014

Open Firmware for qemu-system-ppc -M prep

I'm glad to announce the result of some weekends of work: a new firmware which is not just open, but is also powerful (it runs on Power CPUs), free (the individual files are licensed under MIT and BSD licenses) and of course legendary - it's based on the original PReP firmware from Mitch Bradley's FirmWorks company.

That's how it happened: Mitch licensed the files from the original PReP firmware under the MIT license, I adjusted them to the current OFW tree and added some build files. Then I gathered a small firmware for QEMU PReP machine which used a Cirrus Logic graphics adapter (need a one line change in QEMU code to enable it). Then I added a driver for QEMU "standard" VGA Adapter (also known as Bochs Graphic Adapter) and this night Mitch has committed it together with my atapi fixes as svn.3738 here:

http://code.coreboot.org/p/openfirmware/source/tree/HEAD/

What's in the tree? The original 1994-2000 stuff landed in the /cpu/ppc/prep directory. The build files for generic PReP firmware on top of the generic PPC firmware are in the /cpu/ppc/prep/build folder. And QEMU-specific stuff is in /cpu/ppc/prep/qemu folder. For those who don't want to compile it, but would like to try it straight away I've built two binaries: qprepofw-vga-svn-3738.rom which uses QEMU in graphics mode, and qprepofw-serial-svn-3738.rom for serial console (in the current QEMU version I found no way for guest to see if qemu-system-ppc -M prep is launched with an additional -nographic option).

How to use it:


qemu-system-ppc -M prep  -bios qprepofw-vga-svn-3738.rom  -cpu 603 -hda /path/hdd-image -fda /path/floppy -cdrom path/cd


or


qemu-system-ppc -M prep -bios qprepofw-serial-svn-3738.rom  -cpu 603 -hda /path/hdd-image -fda /path/floppy -cdrom path/cd -nographic


And then

boot disk
or
boot cdrom
or
boot floppy

The advantages over OpenHackWare (default QEMU firmware for PReP target) and OpenBIOS:
  • can boot from floppies and cdrom images. It's possible to boot for instance NetBSD-6.1.3-prep.iso. Of Course, It Runs NetBSD!™
  • supports QEMU's -vga std, -vga cirrus graphic adapters and serial console
  • is written in Forth (OpenHackWare is not), has command history, Forth syntax highlighting and code completion.
  • uses the original layout for NVRAM
The drawbacks:
  • doesn't detect the amount of RAM present and the screen resolution. For now I've just hard-coded 128MiB and 1024x768. Is a trivial thing to support once QEMU chooses the way this information is passed to the firmware (OHW and OpenBIOS use different mechanisms, there are patches which are removing the former one and adding the later, but they are not in upstream yet)
  • can't boot from Apple Partition Boot Code. The original RS/6000 firmware was probably not able to do it either, but OpenBIOS can do it and maybe it's a nice feature for testing (not sure if it is important yet: on one hand there are more PPC distributions for Macs, on the other hand, Macs had slightly different hardware and firmware than PReP machines)
So, digital archaeologists,  have fun and please report if you could do with OHW something which OFW can't.

Stay tuned...

17 comments:

Andrei Warkentin said...

Very cool.

This is BE only I presume? I don't think qemu does LE yet...

A

atar said...

The firmware can be restarted in LE, but throws an exception before getting to the PCI initialization.

It seems that qemu does LE, and switching endianness on the flight works fine for CPU, but some devices have problems with it.

There might be problems with this respect in OFW, but yes, I tend to blame QEMU. :-)

Do you have some cool software for LE?

Andrei Warkentin said...

The pci bridge needs to have the unmunging/lane-reversal support as well (port 0x092, or PICR1 LE_MODE on MPC106/Grackle).

I certainly don't, but it would be awesome to get PReP NT 4.0 booting, which is LE. I've been playing with getting OpenBoot-PPCLE working, but the qemu sources I am hacking on are pretty stale - perhaps the current sources handle LE/ILE a lot better. Having real PReP firmware in LE mode would be a lot better.

The first bit of code to run would be veneer.exe, which is an XCOFF binary implementing the ARC firmware interface using OF.

Unknown said...

veneer.exe is a bit complicated to run:
* it's designed for 601, 603, 604 and 620 (PVRS: 1,3,4,6,7,8,9 and 20). It need some patches for other CPUs.
* it's not actually X-COFF, it's Microsoft COFF which is MZ-PE without the DOS stub. Binutils can't handle it, otool either.
* You still need a new HAL for it.
* It does sort of load on PowerMac3,6 but it doesn't actually run (still one step further than OpenBios).

To make it chicken-egg, you can't recompile it (after patching it) without running it. OTOH, I'm pretty sure that updating veneer is over-kill and simply writing a new simplified bootloader might be more appropriate. I could take on that project if anyone joins me on the HAL part.

The Motorola 27-82660 HAL is a modification of the Motorola Eagle HAL. For both, if there is a BOOTOLDHARDWARE variable with any content it will attempt to boot anyway. There is also DBGHAL which makes it more verbose.
The FirePower HAL and the Generic PPC HAL are more interesting. It emulates an x86 CPU for PCI BIOS calls (like x86emu in the Linux Kernel).
The Victory HAL has a complete PCI implementation that is independent of CPU Emulators.
The Woodfield HAL was created specifically for a VGA card.

A unified (modern) PCI HAL could be made that uses OpenFirmware's framebuffer until the fb driver takes over but it's beyond my skill-set.

What I've found interesting is https://github.com/andreiw/prephv . Apparently Andrei Warkentin got a bit further with his research.

P.S.: Some time ago, I've compiled a Tiger PPC Framebuffer for VMsvga2 with minor changes from the x86 VMsvga2 by Zenith. Anyone curious to try? It works on Intel Tiger and it compiles on PPC Tiger, so it might work.

atar said...

Interesting. Were all those HALs provided on separate ARC floppies?

Razvan Vilt said...

The HALs are on the NT 4.0 Workstation CD in plain sight. Microsoft correctly assumed that any RISC Workstation from that era would have an optical drive. Actually, the ARC standard mandates one. CHRP might also require one.

Razvan:Desktop razvan$ hdiutil attach NTWKS40A.iso
/dev/disk1                                        /Volumes/NTWKS40A
Razvan:Desktop razvan$ cat /Volumes/NTWKS40A/CDROM_W.40
JBCE
Razvan:Desktop razvan$ ls ‑lA /Volumes/NTWKS40A/
total 892
drwxr‑xr‑x  1 razvan  staff  109048 Oct 14  1996 ALPHA
‑rwxr‑xr‑x  1 razvan  staff     176 Oct 14  1996 AUTORUN.INF
‑rwxr‑xr‑x  1 razvan  staff       6 Oct 14  1996 CDROM_W.40
drwxr‑xr‑x  1 razvan  staff     542 Oct 14  1996 DRVLIB
drwxr‑xr‑x  1 razvan  staff  111634 Oct 14  1996 I386
drwxr‑xr‑x  1 razvan  staff    3002 Oct 14  1996 LANGPACK
drwxr‑xr‑x  1 razvan  staff  109140 Oct 14  1996 MIPS
drwxr‑xr‑x  1 razvan  staff  108636 Oct 14  1996 PPC
drwxr‑xr‑x  1 razvan  staff     348 Oct 14  1996 SUPPORT
Razvan:Desktop razvan$ ls ‑lA /Volumes/NTWKS40A/PPC/HAL*.DLL
‑rwxr‑xr‑x  1 razvan  staff  298400 Oct 14  1996 /Volumes/NTWKS40A/PPC/HALCARO.DLL
‑rwxr‑xr‑x  1 razvan  staff  209440 Oct 14  1996 /Volumes/NTWKS40A/PPC/HALEAGLE.DLL
‑rwxr‑xr‑x  1 razvan  staff  296736 Oct 14  1996 /Volumes/NTWKS40A/PPC/HALFIRE.DLL
‑rwxr‑xr‑x  1 razvan  staff  297792 Oct 14  1996 /Volumes/NTWKS40A/PPC/HALPPC.DLL
‑rwxr‑xr‑x  1 razvan  staff  204992 Oct 14  1996 /Volumes/NTWKS40A/PPC/HALPS.DLL
‑rwxr‑xr‑x  1 razvan  staff  308704 Oct 14  1996 /Volumes/NTWKS40A/PPC/HALVICT.DLL
‑rwxr‑xr‑x  1 razvan  staff  278112 Oct 14  1996 /Volumes/NTWKS40A/PPC/HALWOOD.DLL
Razvan:Desktop razvan$ grep ‑A 25 "SourceDisksFiles.ppc]" /Volumes/NTWKS40A/PPC/TXTSETUP.SIF
[SourceDisksFiles.ppc]
ati.dll = 1,,,,,,,2,1
ati.sys = 1,,,,,,,4,1
changer.sys = 1,,,,,,,4,0,0
cs423x.dll = 1,,,,,,,2,1
cs423x.hlp = 1,,,,,,,2,1
cs423x.sys = 1,,,,,,,4,1
elnk16.sys = 1,,,,,,,4,1
halcaro.dll = 1,,,,,,_x,1,3
haleagle.dll = 1,,,,,,_x,1,3
halfire.dll = 1,,,,,,_x,1,3
halppc.dll = 1,,,,,,_x,1,3
halps.dll = 1,,,,,,_x,1,3
halvict.dll = 1,,,,,,_x,1,3
halwood.dll = 1,,,,,,_x,1,3
mga.sys = 1,,,,,,,4,1
oemnade1.inf = 1,,,,,,,2,0,0
osloader.exe = 1,,,,,,,1,3
profile.spc = 1,,,,,,,2,1,0
psidisp.dll = 1,,,,,,,2,1
psidisp.sys = 1,,,,,,_x,4,1
veneer.exe = 1,,,,,,_x,1,3
wd90c24a.dll = 1,,,,,,,2,1
wd90c24a.sys = 1,,,,,,_x,4,1
krnl386.exe = 1,,,,,,,2,0,0

Razvan Vilt said...

Here's some more info from TXTSETUP.SIF:
[Map.Computer]
sandal_up = IBM-6015
wood_up = IBM-6020
wood_up = IBM-6040
wood_up = IBM-6042
carolina_up = IBM-6070
victory_up = IBM-VICT
victory_up = IBM-7042
victory_mp = IBM-7043
victory_mp = IBM-7442
powerstack_up = PowerStack
powerstack2_up = PowerStack2
bigbend_up = "MOTOROLA-Big Bend"
powerized_up = Powerized_ES
powerized_up = Powerized_MX
powerized_up = Powerized_LX
powerized_up = Powerized_TX
powerized_mp = "Powerized_MX MP"
powerized_mp = "Powerized_LX MP"
powerized_mp = "Powerized_TX MP"

[hal]
;
; Hals to be installed based on machine type.
;
sandal_up = halppc.dll , ,hal.dll
wood_up = halwood.dll , ,hal.dll
carolina_up = halcaro.dll , ,hal.dll
victory_up = halvict.dll , ,hal.dll
victory_mp = halvict.dll , ,hal.dll
powerstack_up = haleagle.dll, ,hal.dll
powerstack2_up = halps.dll , ,hal.dll
bigbend_up = haleagle.dll, ,hal.dll
powerized_up = halfire.dll , ,hal.dll
powerized_mp = halfire.dll , ,hal.dll

(SNIP)

[Map.Display]
s3_864 = S3_864
s3_928 = S3_928
s3_964 = S3_964
weitekp9 = P9000
weitekp9 = P9100
vga = VGA
wd90c24a = WD_90C
cirrus = CL54xx
cirrus = CL5436
psidisp = "Powerized Graphics"

[Display.Load]
s3_864 = vga.sys
s3_928 = vga.sys
s3_964 = vga.sys
weitekp9 = weitekp9.sys
vga = vga.sys
wd90c24a = wd90c24a.sys
cirrus = cirrus.sys <-- we have this in QEMU
psidisp = psidisp.sys <-- custom HW, not OF Framebuffer

atar said...

Does it mean all these machines use veneer.exe, which somehow (btw, how?) determines the machine and loads the corresponding hal?

Concerning ARC floppies: there definitely were some. At least for PowerStack. Do you mean they were optional?

Razvan Vilt said...

They should definitely be optional. Veneer, will start the ARC emulation and will put the SystemClass with the value from the OFW root node based on the Company, Model.

After starting the ARC emulation, it will start setupldr which will ask you for language, keyboard, mouse, video and system (HAL). I haven't gotten that far but Andrei Warkentin seems to have gotten there: https://github.com/andreiw/prephv/blob/master/docs/halask.png.

Apparently, setting the arc-identifier = "IBM-6015" variable on the DTS root will automatically select the HAL.DLL
As such, Veneer should work out of the box on any modern PPC system including QEMU with a firmware that can load Microsoft's COFF (like OFW).
Take a look at:
https://github.com/andreiw/prephv/blob/master/prep.dts

And take another look at:
https://github.com/andreiw/prephv/commit/be90524689fd0e2c764871f3504f3a7899ab533e

I believe that we need to allow VMware and Cirrus VGA cards in QEMU for PPC. I have a VMware kext for PPC OS X and Cirrus seems to be supported properly by Windows NT.

I think that we can write a minimal Veneer loader in F-Code and put it in bootinfo.txt. It should set the appropriate properties to the root of the device tree and the /chosen and then loads Veneer, minus the 512 byte header and jump to its entry point.

Next week, when I get back to Bucharest I will do a comparison between different OF implementations to see which support MS COFF. Apple apparently supports Portable Executables through a package and it also has an nt-hack? boolean for unknown purpose. Unfortunately, MS COFF is different from X-COFF and AIX COFF. In the long run, supporting all 3 of them would be desired, but having support for them in Bootinfo.txt is desirable.

If you want to hack away at this, you need a QEMU with Cirrus support in PPC, an OpenBios with COFF, FAT/ISO9660 and MBR support compiled in. And copy the contents of the PPC directory from the ISO file to partition 1 of the MBR disk.

Razvan Vilt said...

One more detail, from: http://web.archive.org/web/20070415015339/http://playground.sun.com/1275/bindings/chrp/chrp1_8a.ps

11.4.1. CHRP PE Program-image Format Support
When the PE File Format is recognized, the Windows NT Veneer is loaded with the following environment:

• Open Firmware shall ensure that virtual address space from 0x80000000 to 0x81000000 (2GB to 2GB+16MB) is available and that any other virtual address space has not been mapped into the physical address space from 0 to 0x1000000 (0 to 16MB). If necessary, Open Firmware shall also modify the real-base configuration vari- able in such a way as to ensure that the Open Firmware image itself is not loaded into any part of the 0 to 0x1000000 physical address space.
• For CHRP Systems with greater than 256MB’s of physical memory, a suggested Open Firmware Image loading address is at the top of the first 256MB bank.
• For CHRP Systems with equal to or less than 256MB’s, a suggested Open Firmware Image loading address is at the top of physical memory.
• Open Firmware shall zero the first 256 bytes (0 to 0x100) of physical memory.
• Open Firmware shall set the little-endian? configuration variable to little endian mode. Open Firmware shall set the real-mode? configuration variable to virtual address mode.
11.4.1.1. Windows NT Veneer Implementation
The current NT Veneer implementation calls the MMU node’s standard map method through the Client Interface to map virtual address 0x80000000 to physical address 0. CHRP System’s intention is to continue using NT calling the standard map method to map virtual memory to physical memory.

atar said...

> If you want to hack away at this

Do you by any chance have a Windows NT beta for SPARC? I know it has been never sold but someone has to have it...

At the moment I'm more focused on the sun4v emulation. Hope to get back to PReP at some point...

Razvan Vilt said...

Nope. From what I remember, it was an Intergraph project and I don't think that they produced any working builds.

Simon Vannarath said...

Hi atar, I'm trying to install various NetBSD on my qemu system with your ROMs. The floppy images never worked on anything (qemu or bare metal) and I've got the 5.2.3, 6.1.2 and 7.0.1 CDs to boot in it, but I can never seem to get it to detect disks. I've set every -device and -hda configuration but the boot messages nor installer seem to detect them. Is there something I'm overlooking?

Here's my command line:

qemu-system-ppc -M prep -serial stdio -bios rom/qprepofw-vga-svn-3738.rom -cpu 604 -drive file=disk/netbsd_701.img,media=disk,if=ide,index=0 -drive file=cdrom/NetBSD-7.0.1-prep.iso,if=ide,index=2,media=cdrom

I have an IBM 7043-140 that I played around with (it's currently in storage), it only successfully booted (via CDROM) and installed NetBSD 5.2.3, openSUSE 10.0 and of course AIX. My aim is to get a SATA controller working on NetBSD, or at least as well as it did on Linux 2.6.

atar said...

You can try plugging in a scsi controller. OTOH, does NetBSD detect IDE disks on 7043-140?
And actually, under qemu, doesn't it detect the disk or the controller?

Simon Vannarath said...

I can't seem to get a SCSI interface up in qemu either, it basically displays:

Warning: Orphaned drive without device: id=scsi0-hd0,file=disk/netbsd_701.img,if=scsi,bus=0,unit=0

and NetBSD fails to detect a controller, let alone the disk. I don't think the 7043-140 has a built-in IDE controller, all my disks and CDROM are SCSI-2.

Would it be better to build my own qemu(-system-ppc) rather than installed it from a package? I'm currently running qemu on Ubuntu 16.04 amd64, installed from packages.

Simon Vannarath said...

Ahh, I it just occurred to me that you meant "install a physical SCSI controller onto my host system". It's a laptop, unfortunately.

atar said...

No, I actually meant the qemu SCSI controller, not the physical one (in theory it should be possible to pass thorough a physical one as well, but I haven't had a chance to try it). The caveat is that for any additional controller (emulated or passed through) a FCode ROM is necessary. And, afair qemu's RPeP doesn't have an easy way to plug it with just a command line in common case. There is a possibility to add an option ROM for the SCSI adapters on qemu-system-i386, so it may work for FCode, but I haven't tested it.

Concerning the QEMU version, I think the most advanced one for PReP is Hervé Poussineau branch (don't have the URL atm).

Unfortunately I don't have much time to play with PPC emulation currently, but hope to get back to it in a few weeks.