Memory Management

How Much Memory?

Unlike traditional PC operating systems, Unix related systems use very sophisticated memory management algorithms to make efficient use of memory resources.  This makes the questions "How much memory do I have?" and "How much memory is being used?" rather complicated to answer.  First you must realize that there are three different kinds of memory, three different ways they can be used by the operating system, and three different ways they can be used by processes.

Kinds of Memory:

• Main - The physical Random Access Memory located on the CPU motherboard that most people think of when they talk about RAM.  Also called Real Memory.  This does not include processor caches, video memory, or other peripheral memory.
• File System - Disk memory accessible via pathnames.  This does not include raw devices, tape drives, swap space, or other storage not addressable via normal pathnames.  It does include all network file systems.
• Swap Space - Disk memory used to hold data that is not in Real or File System memory.  Swap space is most efficient when it is on a separate disk or partition, but sometimes it is just a large file in the File System.

OS Memory Uses:

• Kernel - The Operating System's own (semi-)private memory space. This is always in Main memory.
• Cache - Main memory that is used to hold elements of the File System and other I/O operations.  Not to be confused with the CPU cache or disk drive cache, which are not part of main memory.
• Virtual - The total addressable memory space of all processes running on the given machine.  The physical location of such data may be spread among any of the three kinds of memory.

Process Memory Uses:

• Data - Memory allocated and used by the program (usually via malloc, new, or similar runtime calls).
• Stack - The program's execution stack (managed by the OS).
• Mapped - File contents addressable within the process memory space.

The amount of memory available for processes is at least the size of Swap, minus Kernel.  On more modern systems (since around 1994) it is at least Main plus Swap minus Kernel and may also include any files via mapping.

Swapping

Virtual memory is divided up into pages, chunks that are usually either 4096 or 8192 bytes in size.  The memory manager considers pages to be the atomic (indivisible) unit of memory.  For the best performance, we want each page to be accessible in Main memory as it is needed by the CPU. When a page is not needed, it does not matter where it is located.

The collection of pages which a process is expected to use in the very near future (usually those pages it has used in the very near past, see the madvise call) is called its resident set.  (Some OSs consider all the pages currently located in main memory to be the resident set, even if they aren't being used.)

The process of moving some pages out of main memory and moving others in, is called swapping.  (For the purposes of this discussion, disk caching activity is included in this notion of swapping, even though it is generally considered a separate activity.)

A page fault occurs when the CPU tries to access a page that is not in main memory, thus forcing the CPU to wait for the page to be swapped in.  Since moving data to and from disks takes a significant amount of time, the goal of the memory manager is to minimize the number of page faults.

Where a page will go when it is "swapped-out" depends on how it is being used.  In general, pages are swapped out as follows:

Kernel

Never swapped out.

Cache

Page is discarded.

Data

Moved to swap space.

Stack

Moved to swap space.

Mapped

Moved to originating file if changed and shared.
Moved to swap space if changed and private.
Discarded if unchanged.

It is important to note that swapping itself does not necessarily slow down the computer.  Performance is only impeded when a page fault occurs.  At that time, if memory is scarce, a page of main memory must be freed for every page that is needed.  If a page that is being swapped out has changed since it was last written to disk, it can't be freed from main memory until the changes have been recorded (either in swap space or a mapped file).

Writing a page to disk need not wait until a page fault occurs.  Most modern UNIX systems implement preemptive swapping, in which the contents of changed pages are copied to disk during times when the disk is otherwise idle.  The page can also be kept in main memory so that it can be accessed quickly when needed.  But, if a page fault occurs, the system can instantly reclaim the preemptively swapped pages in only the time needed to read in the new page.  This saves a tremendous amount of time since writing to disk usually takes two to four times longer than reading.  Thus preemptive swapping may occur even when main memory is plentiful, as a hedge against future shortages.

Since it is extremely rare for all (or even most) of the processes on a UNIX system to be in use at once, most of virtual memory may be swapped out at any given time without significantly impeding performance.  If the activation of one process occurs at a time when another is idle, they simply trade places with minimum impact. Performance is only significantly affected when more memory is needed at once than is available.  This is discussed more below.

Mapped Files

The subject of file mapping deserves special attention simply because most people, even experienced programmers, never have direct experience with it and yet it is integral to the function of modern operating systems.

When a process maps a file, a segment of its virtual memory is designated as corresponding to the contents of the given file.  Retrieving data from those memory addresses actually retrieves the data from the file.  Because the retrieval is handled transparently by the OS, it is typically much faster and more efficient than the standard file access methods.  (See the manual page mmap and its associated system calls.)

In general, if multiple processes map and access the same file, the same real memory and swap pages will be shared among all the processes.  This allows multiple programs to share data without having to maintain multiple copies in memory.

The primary use for file mapping is for the loading of executable code.  When a program is executed, one of the first actions is to map the program executable and all of its shared libraries into the newly created virtual memory space.  (Some systems let you see this effect by using trace, ktrace, truss, or strace, depending on which UNIX you have. Try this on a simple command like ls and notice the multiple calls to mmap.)

As the program begins execution, it page faults, forcing the machine instructions to be loaded into memory as they are needed.  Multiple invocations of the same executable, or programs which use the same code libraries, will share the same pages of real memory.

What happens when a process attempts to change a mapped page depends upon the particular OS and the parameters of the mapping.  Executable pages are usually mapped "read-only" so that a segmentation fault occurs if they are written to.  Pages mapped as "shared" will have changes marked in the shared real memory pages and eventually will be written back to the file.  Those marked as "private" will have private copies created in swap space as they are changed and will not be written back to the originating file.

Some older operating systems (such as SunOS 4) always copy the contents of a mapped file into the swap space.  This limits the quantity of mapped files to the size of swap space, but it means that if a mapped file is deleted or changed (by means other than a mapping), the mapping processes will still have a clean copy.  Other operating systems (such as SunOS 5) only copy privately changed pages into swap space.  This allows an arbitrary quantity of mapped files, but means that deleting or changing such a file may cause a bus in error in the processes using it.

Because mapped files share the same address space as RAM, the total size of all mapped files will be limited if either the CPU or the memory controller uses only 32-bit addressing.  This means that files larger than two or three gigabytes cannot be mapped at all.  Such limitations are not a problem for true 64-bit systems, but remember that just because a system has a 64-bit CPU does not necessarily mean that the memory controller is also 64-bit.

So, how much memory is there?

The total real memory is calculated by subtracting the kernel memory from the amount of RAM.  (Utilities like top, yamm, Activity Monitor, or Task Manager can you show you the total real memory available.) Some of the rest may be used for caching, but process needs usually take priority over cached data.

The total virtual memory depends on the degree to which processes use mapped files.  For data and stack space, the limitation is the amount of real and swap memory.  On some systems it is simply the amount of swap space, on others it is the sum of the two.  If mapped files are automatically copied into swap space, then they must also fit into swap memory making that amount the limiting factor.  But if mapped files act as their own swap area, or if swap space is just a growable file in the file system, then the limit to the amount of virtual memory that could be mapped onto them is the amount of hard drive space available.

In practice, it is easy and cheap to add arbitrary amounts of swap space and thus virtual memory.  The real limiting factor on performance will be the amount of real memory.

Okay then, how much memory is being used?

If no programs were sharing memory or mapping files, you could just add up their resident sets to get the amount of real memory in use and their virtual memories to get the amount of swap space in use.  But shared memory means that the resident sets of multiple processes may be counting the same real memory pages more than once.  Likewise, mapped files (on OSs that use them for swapping) will count toward a process' virtual memory use but won't consume swap space.  The issue is further confused by the fact that the system will use any available RAM for disk caching. Since cache memory is low priority and can be freed at any time, it doesn't impede process performance, but is usually counted as "used".

Thus on most OSs there is no easy way to calculate how much memory is being used for what.  Some utility programs, like top yamm, Activity Monitor, or Task Manager, may be able to give you an idea, but their numbers can be misleading since its not possible distinguish different types of use. Just because top says you have very little memory free doesn't necessarily mean you need to buy more.  A better indication is to look for swap activity numbers.  These may be reported as "swap-outs", or "pageouts", and can usually be found using utilities like "vm_stat". If this number is frequently increasing, then you may need more RAM.

In Mac OS X, swap space is allocated dynamically.  In the Finder, type command-shift-g, then enter /var/vm to see these files.  If there is more than just swapfile0, it means your system ran out of memory at some point since its last reboot.  If you frequently see several swap files in here, you may need to buy more RAM.

One of the best and simplest indications of memory problems is to simply listen to the hard drive.  If there is less memory available than the total resident sets of all running processes (after accounting for sharing), then the computer will need to be continuously swapping.  This non-stop disk activity is called thrashing and is an indication that there are too many active processes or that more memory is needed.  If there is just barely enough memory for the resident sets but not enough for all virtual memory, then thrashing will occur only when new programs are run or patterns of user interaction change.  If there is more real memory than virtual memory, then there will be plenty of extra for disk caching and repeated launching of applications or access to files will produce little or no disk sounds.

Not all use is good use

Before you go out and spend money on more memory, make sure to check that the processes you are running aren't consuming more than their share of resources.  Programs like "top" can show you if an application or other process is using increasing amounts of RAM over a long period of time.  If this is occurring, especially while the program is idle, then your problem may be a memory leak in the program, and more RAM would just be filled up by the misbehaving application.  On desktop systems, web browsers are a common culprit for consuming large amounts of RAM.  Some browsers use so much memory and cause so much swapping that it ends up being quicker for them to load files off the network than out of their own cache.  If possible, limit the amount of both RAM and disk cache your browser uses (or just periodically quit and restart it) and turn off features you don't use.  On server systems which allow remote logins, look for user processes which have been left running long after the user has logged out.

Conclusion

These days you can find high-quality RAM online from a number of inexpensive sources.  So unless you are strapped for cash (or configuring a lot of machines), doubt favors the upgrade.  Just make sure the RAM you buy is from a reputable vendor and includes a lifetime warranty, as bad RAM can cause a mess of sometimes subtle system stability problems.

Check your system's specifications carefully for any limitations on the maximum RAM which can be installed.  Just because the CPU is 64-bit, doesn't mean the memory controller is!

If you are really interested in figuring out exactly how much RAM a system needs, run a monitoring program like the ones described above and listen for swapping.  If the monitoring programs show very little free memory, and the pageout count is rising, and you hear swapping when you switch focus between large applications, then you probably need more RAM.

Finally, be aware that some operating systems are much better than others at managing memory.  Closed systems like Windows or Solaris are generally much less efficient than the more modern open source systems such as Linux, *BSD, or Darwin.