Operating System Protection

How does OS do the protection mechanism?

From Segment Descriptor side

When OS loading the segment descriptor to the segment register it will check the type in segment descriptor table

1. CS register can only load the descriptors with Executable flag
2. DS(data segment register) permits the descriptors with Readable flag
3. SS can load the descriptors with Writable flag

Permission control

If an instruction wants to write into the segment without Writable flag,

the process will be denied

Limit checking

This is for processor to not allow addresses outside the segment

We just discuss how limit does in previous post. The limit flag depends on G flag to choose which mode to use.

1. G = 0 range from 0–0xFFFFE (20 bits limit)

2. G = 1 range from 0xFFE-0xFFFFFFFE (32 bits limit)

Limit checking catches programming errors such as runaway subscripts and invalid pointer calculations. Such errors are detected when they occur, so that identification of the cause is easier. Without limit checking, such errors could corrupt other modules

Privilege Levels

By the privilege ring

0: Kernel mode

3: User mode

Privilege ring

Different privilege levels

CPL : Current Privilege Level

RPL : Requested Privilege Level

DPL : Descriptor Privilege Level

privilege level illustration

If instruction wants to load a register, it will work only when DPL is larger than MAX(CPL, RPL)

Jmp, Call instruction issue

Those instructions allow to switch to different segment. That is, we have to check the privilege for the following criteria:

1. DPL is equal to CPL
2. The conforming bit is set to 1 at target descriptor and the CPL ≥ DPL

The second criteria is used for some common library using in both of the kernel or user or the exception handling

If the conforming bit is not 1, if CPL > DPL and OS wants to execute the code in another code segment. One way can do it is through the gate (System call)

Gate Descriptors

4 types of gate descriptors:

1. Call gate

2. Trap gate

3. Interrupt gate

4. Task gate

Consider the Call gate only. Call gate allows jmp, call instructions to execute codes in different segments

format of call gate

privilege level change through call gate

How does call gate work?

1. Compare gate DPL privilege by DPL ≥ MAX(CPL, RPL)
2. Based on the instruction is JMP or CALL check the target DPL with CPL

Case JMP:

target DPL == CPL

Case CALL:

target DPL ≤ CPL

Call gates’ purposes:

• To define the entry point of the precedure
• To specify the privilege level of the entry point

call gate descriptor

call gate flow

• SELECTOR, OFFSET points to the entry point of the procedure
• Gate descriptor’s selector gives executable segment descriptor and in executable segment descriptor points to procedure base

Stack Switching

To maintain system integrity, when doing privilege level change, it should use different stack.

Solution: The process locates these stack by using Task State Segment (TSS)

When call gate changes privilege level, the process use the DPL of the target code segment to index initial stack pointer

• Initial state of the pointer in TSS is Read-Only
• To make privilege transitions transparent to the called procedure, process also copies parameters to new stack

Process steps:

• Check new stack having enough spaces for linkage and parameters. Otherwise, throw stack fault with error 0
• The old value of the stack register SS:ESP pushes to new stack
• The parameters are copied
• A pointer to the instruction after call instruction(CS:EIP) are pushed to new stack

Ret back process from notes

1. The checks are made, and CS:EIP and SS:ESP are loaded with their former values that were saved on the stack.
2. The old SS:ESP (from the top of the current stack) value is adjusted by the number of bytes indicated in the RET instruction. The resulting ESP value is not compared to the limit of the stack segment. If ESP is beyond the limit, that fact is not recognized until the next stack operation. (The SS:ESP value of the returning procedure is not preserved; normally, this value is the same as that contained in the TSS.)
3. The contents of the DS, ES, FS, and GS segment registers are checked. If any of these registers refer to segments whose DPL is greater than the new CPL (excluding conforming code segments), the segment register is loaded with the null selector (INDEX = 0, TI = 0). The RET instruction itself does not signal exceptions in these cases; however, any subsequent memory reference that attempts to use a segment register that contains the null selector will cause a general protection exception. This prevents less privileged code from accessing more privileged segments using selectors left in the segment registers by the more privileged procedure.

RPL

The process checks all selectors passed to process have RPL ≥ CPL (caller)

References

Protected mode vs. Real mode

6.828