There are plenty of websites that go through the boot process (such as How Computers Boot Up). In a nutshell, its a multi-stage process that keeps building up the system a little bit at a time until it can finally start the OS processes.

It starts with the firmware on the motherboard which tries to get the CPU up and running. It then loads up the BIOS which is like a mini operating system that gets the other hardware up and running. Once that is done it looks for a boot device (disk, CD, etc) and, once found, it locates the MBR (master boot record) and loads it into memory and executes it. It’s this little piece of code that then knows how to initialize and start the operating system (or other boot loaders as things have gotten more complicated). It’s at this point that things like the kernel would be loaded and start running.

It’s pretty incredible that it works at all!

A “bare metal” operating system doesn’t run within anything. It runs the full instruction set on the physical machine, and has access to all of physical memory, all device registers and all privileged instructions, including those that control the virtual memory support hardware.

(If the operating system is running on a virtual machine, it may think it is in the same situation as above. The difference is that certain things are emulated or in some other way handled by the hypervisor; i.e. the level that runs the virtual machines.)

Anyway, while the OS might be implemented in (for example) C, it won’t have all of the normal C libraries available to it. In particular, it won’t have the normal ‘stdio’ libraries. Rather, it will implement (for example) a disk device driver that allows it to read and write disk blocks. It will implement a file system on top of the disk block layer, and on top of that it will implement the system calls that a user application’s runtime libraries make to (for example) create, read and write files … and so on.

How can an application run without being in an OS?

It needs to be a special kind of application (e.g. an operating system) that knows how to interact directly with the I/O hardware, etc.

How do you tell the computer to run, say, C, and execute these commands to the screen, if it doesn’t have an OS to run in?

You don’t.

The application (which was for the sake of argument written in C) is compiled and linked on some other machine to give a native code image. Then the image is written to the hard drive in a place that the BIOS can find it. The BIOS loads the image into memory, and executes an instruction to jump to the application’s entry point.

There (typically) isn’t any “running C and executing commands” in the application unless it is a full-blown operating system. And in that case, it is the operating system’s responsibility to implement all of the required infrastructure to make it happen. No magic. Just lots of code.

Bill’s answer covers bootstrapping which is the process in which you go from a powered-off machine to a machine in which the normal operating system is up and running. However, it is worth noting that when the BIOS completes its tasks, it (typically) gives over complete control of the hardware to the main operating system, and plays no further part — until the next system restart. The main OS is certainly not running “within” the BIOS in the conventional sense.

Does it have to do with a UNIX kernel? If so, what is a unix kernel, or a kernel in general?

Yes it does.

The UNIX kernel is the core of the UNIX operating system. It is the part of UNIX that does all of the “bare metal” stuff described above.

The idea of a “kernel” is that you try to separate the system software into core stuff (that requires physical device access, all of memory, etc), and non-core stuff. The kernel consists of the core stuff.

In reality, the distinction between kernel/core and non-kernel/non-core is more complicated than that. And there has been a lot of debate over what really belongs in a kernel, and what doesn’t. (Look up micro-kernel for example.)

In the beginning there was no power in the CPU.

And the Man said “let there be power”, and the CPU started to read from a given address in memory and execute the instruction that was present there. Then the next one and so on until the end of the power.

This was the boot up. Its task was to load another piece of software to gain access to the environment, where the main software was, and load it.

Finally, a friendly screen invited you to log on.

Sorry to be late, but I’ll describe it as such:

• The motherboard gets power.

• Timing circuits start and stabilize if necessary, based solely on their electrical characteristics. Some newer devices may actually use a very limited microprocessor or sequencer.

  It should be noted, alot[sic] of the things like “timing circuits start and stabilize if necessary” don’t really happen in hardware anymore. A tremendous amount of that work is actually extremely specialized software running on very limited subprocessors/sequencers.

  -jkerian at 5:20 on October 25

• Power is given to the CPU and the RAM.

• The CPU loads(based on its internal wiring) data from BIOS. On some machines, the BIOS may be mirrored to RAM and then executed from there but that is rare IIRC.

  When turned on, x86-compatible CPUs start at the address 0xFFFFFFF0 in the address space…

  -Micheal Steil, 17 Mistakes Microsoft Made in the Xbox Security System(archive)

• The BIOS makes calls to hardware ports and addresses used by the motherboard for disk and other hardware IO and spins up disks, gets the rest of RAM working, among other things.

• The BIOS code(by way of the CMOS settings, stored in hardware) uses low-level IDE or SATA commands to read the boot sector of each disk, in an order specified by the CMOS or a user override with a menu.

• The first disk with a boot sector gets its boot sector executed. This boot sector is Assembly that has instructions to load more data from the disk, load a larger NTLDR, later stages of GRUB, etc.

• Finally, the OS machine code is executed by the bootloader, directly or indirectly via chainloading loading a boot sector from an alternate or offset location.

You then get a friendly kernel panic, a suffocated penguin, or your disk grinds to a halt due to a head crash. =) In the alternative scenario, your kernel sets up process tables, in-memory structures, and mounts disks, loading drivers, modules, and a GUI or set of services(if on a server). Then, programs are executed as their headers are read, and their assembly is brought into memory and mapped accordingly.

There are many good answers but I wanted to add this: You mentioned you come from a Python background. Python is a ninterpreted (or “interpiled” or whatever, at least in typical CPython use cases) language. This means you have some other software (the Python interpreter) looking at the source and executing it in some way. This is a fine model and allows quite nice high level languages well abstracted from the actual hardware. Downside is you always need this interpreter software first.

Such interpreter software,typically, is written in a language which compiles to machine code, for instance C or C++. Machine code is what the CPU can handle. What a CPU can do is read some bytes from memory and depending on the byte values start a specific operation. So one byte sequence is a command to load some data from memory into a register, another sequence to add two values, another to store the value from a register back to the main memory and soon (a register is a special memory area which is part of the cpu where it can work best), most of these commands are quite low on that level. The human readable for of these machine code instructions is assembler code. This machine code, basically, is what what is stored in .exe or.com files on windows or inside Linux/Unix binaries.

Now if a computer is started up it is dumb, it has some wiring though which will read such machine code instructions. On a PC this usually (currently) is an EEPROM chip on the mainboard containing the BIOS (basicinput ouptput system), this system can’t do much, it can ease access to some hardware etc and then do a key operation: go to the boot and copy the first few bytes (aka the master boot record, MBR) into memory and then tell the CPU “here, there’s your program” the CPU will then treat those bytes there as machine code and execute it. Typically this is some operating system loader which will load the kernel with some parameters and then hand over the control to that kernel, which will then load all itsdriversto access all hardware, load some desktop or shell program or whatever and allow the user to login and use the system.

You ask “How can an application run without being in an OS”. The easy answer is “an OS is not an application”. While an OS can be created with the same tools as an application, and made out of the same raw material, they are not the same thing. An OS doesn’t have to play by the same rules as an application.

OTOH, you can think of the actual hardware and firmware as the “OS” in which the OS “application” runs. The hardware is a very simple OS — it knows how to run instructions written in machine code, and it knows that when it starts up it should look at a very specific memory address for its first instruction. So, it starts up and then immediately runs that very first instruction, followed by the second, and so on.

So, the OS is simply machine code that exists at a known location, and which can directly interact with the hardware.

Answer to your question requires the knowledge of how the native (for CPU) code looks like and how is it interpreted by CPU.

Usually whole compilation process is based of translating things you write in C, Pascal or even Python (using pypy) and C# into things CPU understands, i.e. simple instructions like “store something under [memory address]”, “add numbers stored under registers eax and ebx”, “call function foo”, “compare eax to 10”. Those instructions, executed one by one, do things you wanted to do with your code.

Now think about this: you don’t really need an OS to execute this native code! All you need is to load this code into memory and tell the CPU it’s there and you want it to be executed. Don’t be concerned too much by that, though. That’s the job BIOS should worry about – it loads your code (one and one sector only), just after the CPU starts, under physical address 0x7C00. Then CPU start to execute this one sector (512 B) of your code. And you can do whatever you imagine! Without, of course, any support from the OS. That’s because YOU are the operating system. Cool, huh? No standard library, no boost, no python, no programs, no drivers! You have to write everything by yourself.

And how do you communicate with hardware? Well, you have two choices:

1. You stay inside of “Real Mode” – CPU execution mode with only 1 MB of memory (and even less), no advanced CPU features like CPU extensions, memory protection, multitasking; 16-bit executable code, ancient addressing mode… But with some routines provided by BIOS, including simple screen output, keyboard support, disk I/O and power management. In one word, you’re back in times of MS-DOS and 16-bit CPUs.
2. You go under “Protected Mode” with all the features your CPU have, all the memory you have installed and so on. But in Protected Mode you ARE completely alone and you have to do EVERYTHING by yourself (and you communicate with hardware using “in” and “out” instructions to input/output data to I/O ports and using interrupts. And memory-mapped I/O). Do I have to say every OS since Windows 95 and very first Linux chooses this option?

Now you’re asking what kernel is.
Shortly, kernel is everything you don’t see and experience directly. It manages, along with drivers, everything, beginning from your keyboard to almost every piece of hardware inside your PC. You communicate with it by graphical shell or terminal. Or by functions inside your code, now executed, luckily, with support of the OS.

For better understanding I can give you one advice: try to write your own OS. Even if it’s gonna write “Hello world” on the screen.

There are some differences for how an operating system operates that are extremely system dependent. To be useful a system needs to have some predictable behavior on start-up, such as “start executing at address X”. For systems that have non-volatile storage (such as Flash memory) mapped into their program space this is fairly easy since you just make sure that you put start-up code at the right location within the processor’s program space. This is extremely common for microcontrollers. Some systems have to retrieve their start-up programs from some other location before executing it. These systems will have some operations hardwired (or almost hardwired) into them. There are some processors that retrieve their start-up code via i2c from another chip, so the processor does non-trivial operations without running any assembly instructions and then starts executing instructions at a predefined address.

Systems using the x86 family of processors typically use a multi-stage boot process that is fairly complex due to its evolution and backwards compatibility issues. The system executes some firmware (called BIOS – Basic Input/Output System, or similar) that is in some non-volatile memory on the motherboard. Sometimes some or all of this firmware is copied (relocated) into RAM to make it execute faster. This code was written with knowledge of what hardware would be present and usable for booting.

Start-up firmware is usually written with assumptions about what hardware will be present on the system. Years ago on a 286 machine there would probably be an assumption that there would be a floppy drive controller at I/O address X and would load sector 0 to a certain memory location if given a certain set of commands (and the code at sector 0 knows how to use the BIOS’s own functions to load more code, and eventually enough code to be an OS is loaded). On a microcontroller there may be an assumption that there is a serial port operating with certain settings that it should wait for commands (to update more complex firmware) from for X amount of time before continuing with the boot process.

The exact start-up process of a given system isn’t as important for you as knowing that it differs on different systems, but also that they all have things in common. Often within the start-up (bootstrapping) code when I/O needs to be done the I/O devices are polled rather than relying on interrupts. This is because interrupts are complex, use stack RAM (which may not be fully set up yet), and you don’t need to worry about blocking other operations when you are the only operation.

Upon first being loaded the OS kernel (the kernel is the main part of most OSes) will initially act a lot like the firmware. It will need to either be programmed with knowledge of or discover hardware present, set up some RAM as stack-space, do various tests, set up various data structures, possibly discover and mount a filesystem, and then probably start some program that is more like the programs you are used to writing (a program that relies on an OS being present).

OS code is usually written in a mixture of C and assembly. The very first code for the OS kernel is probably always in assembly and does things like set up the stack, which C code relies on, and then calls a C function. Other hand written assembly will be in there too because some operations that an OS needs to do are often not expressible in C (like context switching/swapping stacks). Often special flags have to be passed to the C compiler to tell it not to rely on the standard libraries that most C programs use and to not expect that there is an int main(int argc, char *argv[]) in the program. Additionally special linker options that most application programmers never use have to be used. These may make the kernel program expect to be loaded at a certain address or set up things to look like there are external variables at certain locations even though those variables were never declared in any C code (this is useful for memory mapped I/O or other special memory locations).

The whole operation seems like magic at first, but after you look into it and understand parts of it the magic becomes just a set of programs that require a lot more planning and system knowledge to implement. Debugging them, though, takes magic.

To understand how operating systems work, it may be helpful to split them into two categories: those which simply provide services to applications upon request, and those which use hardware features in the CPU to prevent applications from doing things they shouldn’t. MS-DOS was of the former style; all versions of Windows since 3.0 have been the latter style (at least when running anything more powerful than an 8086).

The original IBM PC running PC-DOS or MS-DOS would have been an example of the former style of “OS”. If an application wished to display a character on the screen, there would have been a few ways of doing it. It could call the routine which would ask MS-DOS to send it to “standard output”. If it did that, MS-DOS would check whether output was being redirected and, if not, it would call a routine stored in ROM (in a collection of routines IBM called the Basic Input/Output System) which would display a character at the cursor position and move the cursor (“write teletype”). That BIOS routine would then store a pair of bytes somewhere in the range 0xB800:0 to 0xB800:3999; hardware on the Color Graphics Adapter will repeatedly fetch pairs of bytes within that range, using the first byte of each pair to select a character shape and the second to select foreground and background colors. The bytes are fetched and processed into red, green, and blue signals, in a sequence which yields a legible text display.

Programs on the IBM PC could display text by using the DOS “standard output” routine, or by using the BIOS “write teletype” routine, or by storing it directly to display memory. Many programs which needed to display lots of text quickly opted for the latter approach, since it could be literally hundreds of times as fast as using the DOS routines. This wasn’t because the DOS and BIOS routines were exceptionally inefficient; unless the display was blanked, it could only be written at certain times. The BIOS routine to output a character was designed so it could be called at any time; each request thus had to start afresh waiting for the right time to perform a write operation. By contrast, application code which knew what it needed to do could organize itself around the available opportunities to write the display.

A key point here is that while the DOS and BIOS provided a means of outputting text to the display, there wasn’t anything particularly “magical” about such abilities. An application which wanted to write text to the display could do so just as effectively, at least if the display hardware worked the way the application expected (if someone had installed a Monochrome Display Adapter, which was similar to the CGA but had its character memory located at 0xB000:0000-0xB000:3999), the BIOS would automatically output characters there; an application which was programmed to work with either the MDA or CGA could do so as well, but an application which was programmed just for the CGA would be totally useless on the MDA).

On newer systems, things are a bit different. Processors have various “privilege” modes. They start out in the most privileged mode, where code is allowed to do anything it wants. They can then switch into a restricted mode, where only selected ranges of memory or I/O facilities are available. Code cannot switch directly from a restricted mode back to privilege mode, but the processor has defined privileged-mode entry points, and restricted-mode code can ask the processor to start running code at one of those entry points in privileged mode. Additionally, there are privileged-mode entry points associated with a number of operations which would be forbidden in restricted mode. Suppose, for example, that someone wanted to run multiple MS-DOS applications simultaneously, with each one having its own screen. If applications could write directly to the display controller at 0xB800:0, there would be no way to prevent one application from overwriting another application’s screen. On the other hand, an OS could run the application in restricted mode and trap on any accesses to the display memory; if it discovered that an application which was supposed to be in the “background” was trying to write 0xB800:160, it could store the data to some memory which it had set aside as a background-application screen buffer. If that application is later switched to the foreground, the buffer could then be copied to the real screen.

The key things to note are (1) although it’s often handy to have a standard set of routines to perform various standard services like displaying text, they don’t do anything that an application which was running in “privileged mode” couldn’t do if it was properly programmed to deal with the hardware that was installed; (2) although most applications running today would be prevented by their operating system from doing such I/O directly, a program which starts up in privileged mode gets to do whatever it wants, and can set up whatever rules it wants for restricted-mode programs.

As Stephen C. said, it’s not just about starting up the Operating System, it’s also about how it runs, interacts with the hardware and with the software on top of it.

I’ll just add to his answer, that you may want to take a look at “The Elements of Computing Systems”. It’s a book and some tools, that explains how a computer, operating system and compilers interact. The unique thing about it is that it gives you the tools to very quickly develop your own operating system in a simulated environment, ignoring the many details required for a real one, so that you can grasp the concepts. It does a great job of letting you see the forest instead of the trees.

If you want to go into more details about how the Operating System interacts with the hardware, then check out Minix.

You write an OS. It has to be run somehow, and that way is within another OS?

Your application is running within an OS. This Operating System provides services to your application, things like opening a file and writing bytes to it. These services are usually provided via system calls.

The Operating System is running within the hardware. The hardware provides services to the Operating System, things like setting the baud rate of a serial port and writing bytes to it. These services are usually provided via memory mapped registers or I/O ports.

To give a very simplified example of how this works:

Your application tells the operating system to write something to a file. To your application, the operating system provides concepts like files and directories.

On the hardware, these concepts do not exist. The hardware provides concepts like disks divided in fixed blocks of 512 bytes. The operating system decides which blocks to use for your file, and some other blocks for metadata like the file name, size, and location on the disk. It then tells the hardware: write these 512 bytes to the sector with this number on the disk with that number; write these other 512 bytes to the sector with this different number on the disk with that same number; and so on.

The way the operating system tells the hardware to do that varies a lot. One of the functions of an operating system is to shield the applications from these differences. For the disk example, on one kind of hardware, the operating system would have to write the disk and sector number to an I/O port, and then write the bytes one by one to a separate I/O port. On another kind of hardware, the operating system would have to copy the whole 512 bytes of a sector to an area of memory, write the location of that area of memory to a special memory location, and write the disk and sector number to yet another special memory location.

Today’s high-end hardware is extremely complicated. The manuals giving all their programming details are doorstoppers with thousands of pages; for instance, the latest Intel CPU manual is seven volumes, with a total of over 4000 pages—and that’s only for the CPU. Most other components expose blocks of memory or I/O ports, which the operating system can tell the CPU to map to addresses within its address space. Several of these components expose even more things behind a few I/O ports or memory addresses; as an example, the RTC (Real Time Clock, the component which keeps the computer’s time while it is powered off) exposes a few hundred bytes of memory behind a pair of I/O ports, and that is a very simple component dating back to the original PC/AT. Things like hard disks have whole separate processors, which the operating system talks to via standardized commands. GPUs are even more complicated.

Several people in the comments above suggested the Arduino. I agree with them, it is much simpler to understand—the ATmega328, which does everything on the Arduino Uno except exposing the USB connector as a serial port, has a manual with only a few hundred pages. On the Arduino, you run directly on the hardware, with no operating system in between; just a few small library routines, which you do not have to use if you do not want to.

Runnable examples

Technically, a program that runs without an OS, is an OS. So let’s see how to create and run some minuscule hello world OSes.

The code of all examples below is present on this GitHub repo.

Boot sector

On x86, the simplest and lowest level thing you can do is to create a Master Boot Sector (MBR), which is a type of boot sector, and then install it to a disk.

Here we create one with a single printf call:

printf '364%509s125252' > main.img
sudo apt-get install qemu-system-x86
qemu-system-x86_64 -hda main.img

Outcome:

Tested on Ubuntu 18.04, QEMU 2.11.1.

main.img contains the following:

• 364 in octal == 0xf4 in hex: the encoding for a hlt instruction, which tells the CPU to stop working.

  Therefore our program will not do anything: only start and stop.

  We use octal because x hex numbers are not specified by POSIX.

  We could obtain this encoding easily with:

  echo hlt > a.asm
  nasm -f bin a.asm
  hd a
  

  but the 0xf4 encoding is also documented on the Intel manual of course.

• %509s produce 509 spaces. Needed to fill in the file until byte 510.

• 125252 in octal == 0x55 followed by 0xaa: magic bytes required by the hardware. They must be bytes 511 and 512.

  If not present, the hardware will not treat this as a bootable disk.

Note that even without doing anything, a few characters are already printed on the screen. Those are printed by the firmware, and serve to identify the system.

Run on real hardware

Emulators are fun, but hardware is the real deal.

Note however that this is dangerous, and you could wipe your disk by mistake: only do this on old machines that don’t contain critical data! Or even better, devboards such as the Raspberry Pi, see the ARM example below.

For a typical laptop, you have to do something like:

• Burn the image to an USB stick (will destroy your data!):

  sudo dd if=main.img of=/dev/sdX
  

• plug the USB on a computer

• turn it on

• tell it to boot from the USB.

  This means making the firmware pick USB before hard disk.

  If that is not the default behavior of your machine, keep hitting Enter, F12, ESC or other such weird keys after power-on until you get a boot menu where you can select to boot from the USB.

  It is often possible to configure the search order in those menus.

For example, on my old Lenovo Thinkpad T430, UEFI BIOS 1.16, I can see:

Hello world

Now that we have made a minimal program, let’s move to a hello world.

The obvious question is: how to do IO? A few options:

• ask the firmware, e.g. BIOS or UEFI, to do if for us
• VGA: special memory region that gets printed to the screen if written to. Can be used on Protected mode.
• write a driver and talk directly to the display hardware. This is the “proper” way to do it: more powerful, but more complex.

• serial port. This is a very simple standardized protocol that sends and retrieves characters from a host terminal.

  Source.

  It is unfortunately not exposed on most modern laptops, but is the common way to go for development boards, see the ARM examples below.

  This is really a shame, since such interfaces are really useful to debug the Linux kernel for example.

• use debug features of chips. ARM calls theirs semihosting for example. On real hardware, it requires some extra hardware and software support, but on emulators it can be a free convenient option. Example.

Here we will do a BIOS example as it is simpler on x86. But note that it is not the most robust method.

main.S

.code16
    mov $msg, %si
    mov $0x0e, %ah
loop:
    lodsb
    or %al, %al
    jz halt
    int $0x10
    jmp loop
halt:
    hlt
msg:
    .asciz "hello world"

link.ld

SECTIONS
{
    . = 0x7c00;
    .text :
    {
        __start = .;
        *(.text)
        . = 0x1FE;
        SHORT(0xAA55)
    }
}

Assemble and link with:

gcc -c -g -o main.o main.S
ld --oformat binary -o main.img -T linker.ld main.o

Outcome:

Tested on: Lenovo Thinkpad T430, UEFI BIOS 1.16. Disk generated on an Ubuntu 18.04 host.

Besides the standard userland assembly instructions, we have:

• .code16: tells GAS to output 16-bit code

• cli: disable software interrupts. Those could make the processor start running again after the hlt

• int $0x10: does a BIOS call. This is what prints the characters one by one.

The important link flags are:

• --oformat binary: output raw binary assembly code, don’t warp it inside an ELF file as is the case for regular userland executables.

Use C instead of assembly

Since C compiles to assembly, using C without the standard library is pretty simple, you basically just need:

• a linker script to put things in memory at the right place
• flags that tell GCC not to use the standard library
• a tiny assembly entry point that sets required C state for main, notably:
  • the stack
  • zero out BSS

TODO: link so some x86 example on GitHub. Here is an ARM one I’ve created.

Things get more fun if you want to use the standard library however, since we don’t have the Linux kernel, which implements much of the C standard library functionality through POSIX.

A few possibilities, without going to a full-blown OS like Linux, include:

• Newlib

  Detailed example at: https://electronics.stackexchange.com/questions/223929/c-standard-libraries-on-bare-metal/223931

  In Newlib, you have to implement the syscalls yourself, but you get a very minimal system, and it is very easy to implement them.

  For example, you could redirect printf to the UART or ARM systems, or implement exit() with semihosting.

• embedded operating systems like FreeRTOS and Zephyr.

  Such operating systems typically allows you to turn off pre-emptive scheduling, therefore giving you full control over the runtime of the program.

  They can be seen as a sort of pre-implemented Newlib.

ARM

In ARM, the general ideas are the same. I have uploaded:

• a few simple QEMU baremetal examples here on GitHub. The prompt.c example takes input from your host terminal and gives back output all through the simulated UART:

  enter a character
  got: a
  new alloc of 1 bytes at address 0x0x4000a1c0
  enter a character
  got: b
  new alloc of 2 bytes at address 0x0x4000a1c0
  enter a character
  

  See also: https://stackoverflow.com/questions/38914019/how-to-make-bare-metal-arm-programs-and-run-them-on-qemu/50981397#50981397

• a fully automated Raspberry Pi blinker setup at: https://github.com/cirosantilli/raspberry-pi-bare-metal-blinker

  See also: https://stackoverflow.com/questions/29837892/how-to-run-a-c-program-with-no-os-on-the-raspberry-pi/40063032#40063032

For the Raspberry Pi, https://github.com/dwelch67/raspberrypi looks like the most popular tutorial available today.

Some differences from x86 include:

• IO is done by writing to magic addresses directly, there is no in and out instructions.

  This is called memory mapped IO.

• for some real hardware, like the Raspberry Pi, you can add the firmware (BIOS) yourself to the disk image.

  That is a good thing, as it makes updating that firmware more transparent.

Firmware

In truth, your boot sector is not the first software that runs on the system’s CPU.

What actually runs first is the so-called firmware, which is a software:

• made by the hardware manufacturers
• typically closed source but likely C-based
• stored in read-only memory, and therefore harder / impossible to modify without the vendor’s consent.

Well known firmwares include:

• BIOS: old all-present x86 firmware. SeaBIOS is the default open source implementation used by QEMU.
• UEFI: BIOS successor, better standardized, but more capable, and incredibly bloated.
• Coreboot: the noble cross arch open source attempt

The firmware does things like:

• loop over each hard disk, USB, network, etc. until you find something bootable.

  When we run QEMU, -hda says that main.img is a hard disk connected to the hardware, and

  hda is the first one to be tried, and it is used.

• load the first 512 bytes to RAM memory address 0x7c00, put the CPU’s RIP there, and let it run

• show things like the boot menu or BIOS print calls on the display

Firmware offers OS-like functionality on which most OS-es depend. E.g. a Python subset has been ported to run on BIOS / UEFI: https://www.youtube.com/watch?v=bYQ_lq5dcvM

It can be argued that firmwares are indistinguishable from OSes, and that firmware is the only “true” bare metal programming one can do.

As this CoreOS dev puts it:

The hard part

When you power up a PC, the chips that make up the chipset (northbridge, southbridge and SuperIO) are not yet initialized properly. Even though the BIOS ROM is as far removed from the CPU as it could be, this is accessible by the CPU, because it has to be, otherwise the CPU would have no instructions to execute. This does not mean that BIOS ROM is completely mapped, usually not. But just enough is mapped to get the boot process going. Any other devices, just forget it.

When you run Coreboot under QEMU, you can experiment with the higher layers of Coreboot and with payloads, but QEMU offers little opportunity to experiment with the low level startup code. For one thing, RAM just works right from the start.

Post BIOS initial state

Like many things in hardware, standardization is weak, and one of the things you should not rely on is the initial state of registers when your code starts running after BIOS.

So do yourself a favor and use some initialization code like the following: https://stackoverflow.com/a/32509555/895245

Registers like %ds and %es have important side effects, so you should zero them out even if you are not using them explicitly.

Note that some emulators are nicer than real hardware and give you a nice initial state. Then when you go run on real hardware, everything breaks.

GNU GRUB Multiboot

Boot sectors are simple, but they are not very convenient:

• you can only have one OS per disk
• the load code has to be really small and fit into 512 bytes. This could be solved with the int 0x13 BIOS call.
• you have to do a lot of startup yourself, like moving into protected mode

It is for those reasons that GNU GRUB created a more convenient file format called multiboot.

Minimal working example: https://github.com/cirosantilli/x86-bare-metal-examples/tree/d217b180be4220a0b4a453f31275d38e697a99e0/multiboot/hello-world

I also use it on my GitHub examples repo to be able to easily run all examples on real hardware without burning the USB a million times. On QEMU it looks like this:

If you prepare your OS as a multiboot file, GRUB is then able to find it inside a regular filesystem.

This is what most distros do, putting OS images under /boot.

Multiboot files are basically an ELF file with a special header. They are specified by GRUB at: https://www.gnu.org/software/grub/manual/multiboot/multiboot.html

You can turn a multiboot file into a bootable disk with grub-mkrescue.

El Torito

Format that can be burnt to CDs: https://en.wikipedia.org/wiki/El_Torito_%28CD-ROM_standard%29

It is also possible to produce a hybrid image that works on either ISO or USB. This is can be done with grub-mkrescue (example), and is also done by the Linux kernel on make isoimage using isohybrid.

Resources

• http://wiki.osdev.org is a great source for those matters.
• https://github.com/scanlime/metalkit is a more automated / general bare metal compilation system, that provides a tiny custom API