Goals of Memory Management

• Make effective use of available memory
  • Provides increased levels of concurrency
  • Provides high utilization
  • Somewhat less important in systems where memory is cheap
  • Very important in small, portable, battery-driven systems like PalmPilot.
• Support the memory paradigms used by programming languages
  • Each programming language has a different view of memory
  • Fortan doesn't have dynamic memory -- ask only at beginning
  • Java has garbage colection -- frees memory automatically
• Safety and Robustness
  • Make causing damage more difficult for programmers and users
  • Regardless of cause, minimize damage
    • For example, keep different tasks in different address spaces.

Often these goals are in conflict with each other. Increased robustness

Memory Allocation Strategies

What makes allocation non-trivial?

available memory is one linear array

many independent requestors (processes) that are not aware of each other

unpredictable sequence of allocations and releases

the need for contiguity.

code is implicitly sequential

compilers assume data is sequential. Consider arrays, structs, I/O buffers, &c.

The need for contiguity makes fragmentation evil. For example, malloc(x) must return one chunk of memory grater then or equal to x bytes in size, not three chunks summing up to x bytes.
 

Characteristics of An Ideal Allocation Strategy

• if m bytes of memory are available, a request for m bytes or less should be satisfied immediately
• memory is never wasted by satisfying requests with more memory than is requested
• no memory or CPU are wasted with houskeeping

Satisfying these three constraints is impossible, (perhaps we can get 2 of the 3, if we know the requests in advance -- but this is not generally possible). This implies that we must make tradeoffs.
 

Usual Approach: Delegation

• OS makes coarse grain decision. It allocates large chunks of space "wholesale" to the processes
• The runtime package in each process makes fine-grained decisions. it doles out small pieces wihtin the process.
• An example is the use of the sbrk() system call to allocate large chunks of space to a process, and the use of malloc() to dole out small pieces from the large pieces supplied.
• The expectation is that the individual tasks know more about the coide and can better manage the memory

4 Typical OS Strategies

• monoprogramming
• multiprogramming with fixed partitions
• multiprogramming with variable partitions
• buddy system

Monoprogramming

The user taks area, including the stack, heap, and code segments is internally managed by the program.
 

Multiprogramming With Fixed Partitions

• Statically divide memory into a partition for each task, not necessarily fo equal size
• Partitions must be at least as large as necessary. They can be larger, but should be as tight a fit as possible.
• The excess space within a partition is a form of wastage called internal fragmentation.

Multiprogramming With Variable Paritions

• Try to keep free memory in big chunks
• Allocate only what is needed to tasks
• Coalesce adjacent chunks on return by task
• Splititing an coalescing consume CPU cycles

This approach can lead to thin slices of free memory that are too small to satisfy any processes request. This type of wastage is called external fragmentation. The solution is to try to keep free memory in chunks rather than in many small pieces. This is easier said than done.

Typical Strategies

• First-fit: List of available space is kept in an unordered list. The first available block that can satisfy the request is used. Left over space is added back to the list. Big chunks accumulate at the bottom, because they are rapidly consumed from the top.
• Best-fit: The list is kept sorted from smallest (head) to largest (tail). The first (smallest) block than can satisfy the request is used. Left over space is as small as possible and is added back to the list in the proper position. This can create many small slivers of memory that can not satisfy any request. Slower because the lsit must be sorted and searched linearly.
• Worst-fit: The list is sorted form largest (head) to smallest(tail). Allocations are always from head, no search required. Left over space is as large as possible and is added at the right position in the list. This approach aims to return large chunks of space to the list in the hope that they might be allocated. Empirical studies haven't confirmed this for most workloads.

No approach is best for all circumstances. In practice, first-fit is used because it is easy to implement and doesn't waste time sorting the list. In practice a match is found quickly, despite the unsorted list, and the match isn't too bad.

Goals of Virtual Memory

• Logical contiguity without physical contiguity
• Better support for multiprogramming
• Controlled sharing of address space
• Support of logical and physical address space of different sizes.
• Reduced fragmentation
• Protection one process from malicious or accidental damage by another (goes along with controlled sharing).
• Protection the privacy of one process data from another, as desired. (goes along with controlled sharing).

Basic Idea

The hardware and operating system cooperate to provide a mapping from a virtual address sapce to the physical address apce. This mapping should be completely transparent to the user process with only a minimal performance impact.

There are two distinct strategies:

• paging
• segmentation

Paging splits the address space into equal sized units called pages. While segmentation splits the memory into unequal units that may have sizes more meaningful or appropriate to the program. These two ideas can be combined. As a practical matter paging is easier to implement than segmentation.

Implementing Paging

1. Divide physical memory into equal sized memory units called frames. The page size should be a power of two, because this improves the performance of supporting hardware.
2. Divide the logical memory into units called pages. The pages must be the same size as the frames
3. Define a mapping from the logical (virtual) address space to the physical address space

The mapping must be fast and simple. It is typically done by a table lookup. The page size should be a power of two, because division by powers of two is trivial.

We typically consider divide a virtual address into two fields: a page number and an offset, with the offset taking the high-order bits. The example below is typically of machines with 32-bit addresses. This provides a 4K page size:

Steps In Address Translation

1. Spit virtual address
2. Lookup page number
3. Concatenate frame number with offset to produce physical address
4. Make access to physical memory

Mapping From The Virtual Address To The Physical Address

There is a mapping table, called the page table that exists in physcal memory. Each address space is independently mapped within the space of the process. Steps 1-4 are typically performed in hardware for performacne reasons -- address translation in software can crush a systems performance. Step #2 is tricky, the others are trivial.

The Translation Buffer (TB)

Since the table is stored in physical memory, lookups cause memory references. This increases the memory access latency; even with caching, address trasnlation can be very costly. The "fix" is a very specialized cache, the Tranlation Buffer (TB) also known as the Translation Lookaside Buffer (TLB).

The TLB stores the most recent mappings from virtual addresses to physical addresses. It exists in hardware and is invisible to the OS, except the OS has the ability to flush (and possibly reload) it. This is necessary since each process has its own virtual address space, the old entries must be invalidated when a context-switch occurs. Failing to flush the TB is a common mistake. Some TB's keep task ID's associated with the mappings to prevent the need to flush.

The TB operates at cache speeds and many architectures can do the TLB and cache lookups in parallel.

What If a TB Miss Occurs?

If a TB miss occurs, the translation must be found in the page table and the new mapping must be stored in the TB. On CISC machines this is typically done entirely in hardware. The table is kept in the hardware-specified format
and the harware is told its address, usually via a register. On RISC machines, the update (fill) process is handled by software. The OS is informed of the miss via a trap generated by the hardware.

Choice Of Page Size

Small page sizes yield a finer grained memory system. The mapping between the memory requested and allocated can be much tighter. But smaller pages mean more pages and that means a bigger page table. A bigger page table not only requres a bigger TB (costly) but also occupies more space in memory (greater overhead).

By constrast large page sizes yield a coarse grained mapping. This leads to more internal fragmentation, because allocations are rounded to the nearest whole page. But the TB doesn't have to be as large (cheaper) and the page table is smaller (less overhead).

Page tables are typically 1K-8K. The choice of page table size implies an engineering trade-off. Sometimes it is made in hardware, but other hardware allows a variable page size via a register.

Design of Page Tables

Major issues:

• One per task or system-wide? If the address space is system wide. This implies a model where processes actually behaive more like threads. The concept of a task is really lost.
• Is the page table itself monolithic? Single-level page tables are very large and require large amounts of contiguous memory. Multi-level page tables require less memory in the common case, but can increase the number of memory accesses required. Paradoxically, multi-level page tables can actually occupy more space in the event that a process is using almost all of the available virtual memory.

The Simplest Case: Single-level Page Tables

The offset field of the virtual address is the index into the page table, that is a 1D array. The contents of that cell, among other things, contain the physical address of the frame in memory.

Multi-level Page Tables

The biggest drawback of single-level page tables is their size. They must contain one entry for each virtual address. This is an enormous number of entries, especially considering that the virtual address space, by design, should be far larger than is ever required. Consider the large hole between the stack and the heap: there is an entry for every unused page.

Multilevel page tables are one solution to this problem. We'll consider what in practice is the most popular type: the 2-level page table. It boils down to a page table of page tables. We now divide our virtual address into three fields: the directory # (page table #1), the page number (page table #2), and the offset.

We still must create a directory entries to span our virtual address space, but these entries are much coarser in grain. Fewer entries can cover the entire address space. We only create page tables (page table #2) for those directory entries that are actually used.

In the common case -- processes with sparse memory usage -- this saves space. But it also costs time. We now must make an additional access to memory to do a page table lookup. It is also important to note that multi-level page tables consume more memory than single-level page tables if most of the address space is actually used; the difference is lost in the overhead of the additional structure.

Segmentation: Meaningful Allocation

Segmentation is a technique for allocating memory in chunks of varying and meaningful sizes instead of one arbitrary page size. Since the segment sizes are selected by the compiler, internal fragmentation is reduced. Of course, since the segments are of different sizes, we have the same problems with external fragmentation and compaction that we discussed when we talked about variable sized partions for multiprogramming. For this reason, among others, we'll find that although segmentation is supported by popular processors like those in the x86 family, paging is most often used by processors that support it, including those in the x86 family.

Meaningful Allocation

When programmers think of memory they see it as a collection of different objects, not as a linear array of bytes or pages. These objects, unlike pages vary in size. It is this view of memory they segmentation seeks to support.

A program might be broken down into segments for the code, stack and heap. In addition, separate segments might be created for statically allocated data structures such as heaps and arrays. Each of these segments is exactly the necessary size, so internal fragmentation is eliminated.

These segments are then numbered. An address in memory is specificed as a tuple consisting of the segment number and the offset within the segment: .

The Segment Table

Just as was the case for paging, we must provide a mechanism for mapping from this logical view of memory, to the linearly addressable physical memory. This mechanism is the segment table

Since the segments are defined by the compiler, the segment table can be fixed at link time. The table contains one entry for each segment. The entry contains the base and limit for the segment. The base is the physical address of the beginning of the segment in memory. The limit is the length of the segment, so that the last physical address within the segment is specified by the base plus the bound (base + bound).

Since each program enjoys its own virtual address space, each process has it's own segment table. The sement table of the current process is kept in memory and is referenced by two hardware registers, the segment-table base register (STBR) and the segment-table length register (STLR). On certain architectures, if the segment table is small enough, it can be stored entirely in registers within the CPU.

Using The Segment Table

When an address is specified as a tuple , the hardware translates it to a physical address using the segment table. The hardware checks the entry in the table specified by the offset number of find the base address of the segment. If the segment-number is out-of-bounds, the address is not valid. It then checks the limit to ensure that the offset is less than the limit. If the offset is not less than the limit, the address is invalid. If the address is valid, it is translated to physical address base+offset.

Protection and Sharing

Since segments represent logical units of a program, they are ideal for sharing. For example, it is logical for the code segment of several processes corresponding to the same program to be shared. Appropriate protection and security can be enforced by associating this information with the segment table.

When statically allocated arrays and other complex structures are associated with their own segment, bounds checking becomes almost free. It happens as a natural consequence of the lookup in the page table and requires no additional instructions.

Fragmentation

Although segmentation eliminates internal fragmentation, it does so at the expense of external fragmentation. It is possible that although a sufficient amount of memory is available, it will be partitioned into small pieces by existing segments. This results in unusuable memory and a failed request.

The impact of external fragmentation depends on the average segment size. If the segments are generally small, they can usually fit into an available space. If they are large, there is a greater chance that the available space will be partitioned into free blocks that are too small to be accomodating.

As we discussed in the context of multiprogramming with variable partitions, the placement and replacement strategies can also affect fragmentation. Depending on the policy employed, the underlying data structures and their maintenence can also add substantial overhead to memory management.

Compaction systems that are capable of relocating the segments of existing processes can ease this cost by making the allocated segments physically contiguous. This approach is very expensive because it requires extensive memory-memory copies. It also requires a runtime rebindable loader.

Supporting a Virtual Address Space Larger Than Physical Memory

Supporting a virtual address space larger than the physical address space requires what is called a backing store. A backing store is storage large enought to hold all of the data that could fit into the virtual address space. The backing store is usually a secondary storage device like disk.

When we consider memory from this perspective, we find that smaller, faster, and memore expensive storage devices are actually being used as a cache for larger, slower, and less-expensive device. This model for a system's memory is called the memory hierarchy.

For Completeness: Swapping

Swapping is another technique that involves replacing the entire address space each time a task context-switches. This technique is transparent to the user, but the I/O cost is trememndous.