Talk:Page cache

Page cache vs. buffer cache

Operating systems that didn't support paged virtual memory, such as older versions of Unix, had a "buffer cache" for file blocks read in or written to. If this article is discussing the notion of caching blocks read from or written to files, it should probably be stripped of many of its references to "pages" and discuss the concept without speaking of virtual memory systems, and perhaps even be renamed to "buffer cache".

Some systems supporting memory-mapped files use a common page cache both for blocks read from and written to files with explicit I/O calls and blocks read from files as a result of page faults (and some of them implement those explicit I/O calls by mapping part of the file in and copying to and from the mapped region); that would be a subtopic of this article. I think some systems, e.g. older versions of BSD that supported demand paging but didn't support mmap(), had a page cache for pages fetched from an executable image (demand-paged executables being the only form of memory-mapped file they supported) and a buffer cache for blocks read from files. Guy Harris (talk) 19:58, 6 February 2012 (UTC)

So should the page should be a "buffer cache" page with a separate "page cache" page (and something somewhere describing a "unified buffer cache" combining the two), or should there be separate pages discussing those two concepts with the unified buffer cache described somewhere, or what? Guy Harris (talk) 19:06, 6 December 2013 (UTC)

(Question raised because some "buffer cache" information was removed from this page as it wasn't caching that used the VM system.) Guy Harris (talk) 19:07, 6 December 2013 (UTC)

It seems to me that OS-managed caches for disk storage can be organized along several different axes. One is "who requested the data" - in this case we have explicit disk (or file) IO calls vs. page reads caused by page faults.

Another is "where does the cache storage space come from and how is it managed". (These might even be two separate axes.) The earliest disk caching software was simply allocated a fixed amount of RAM; later we got cache managers that worked along side (and sometimes fought with) the virtual memory manager; in Windows the reactive file cache, the proactive file cache (SuperFetch), and what we're calling the page cache are all using storage managed by the common virtual memory manager (and there is additional complication from ReadyDrive and ReadyDisk).

There is probably at least one more axis.

So, what about an organization that first explains this organization, then presents various examples from various OSs and shows how they fit into this organization? Only problem is, I can't think of a reference that presents a unified view of cacheing in this way (but I'm going to use it for some writing I'm doing :) ). Jeh (talk) 17:59, 7 December 2013 (UTC)

• Well, the NetBSD unified buffer cache paper [1] was required reading in some CS grad schools. And Linux implemented it a bit later (2.4) [2]. It seems however that Windows NT already had that feature (at least) 5 years before NetBSD [3]. Someone not using his real name (talk) 23:32, 20 December 2013 (UTC)

(And, as the paper notes, "The first operating system to address these problems was SunOS"; it came out in 1988, a few years before Windows NT 3.1.)

In any case, that paper might be a useful reference, if nothing else; thanks. Guy Harris (talk) 23:51, 20 December 2013 (UTC)

Correct, although "the first" seems to mean "the first published (as academic papers)" because details about some of the other commercial Unixes are lacking there (we're told they they had it too before NetBSD in some vague terms). The NetBSD paper is grad school reading list material because it's focused on this single problem and has an empirical evaluation of its impact. (The SunOS 4 papers are much more general, and don't have any benchmarks.) By the way, the book "UNIX Filesystems" seems quite useful here, although its coverage of "buffer cache" seems a little spread out. On p. 136 it says "I/O changed substantially in SVR4 with all data being read and written through pages in the page cache as opposed to the buffer cache, which was now only used for meta-data (inodes, directories, etc.)" And on p. 146 "SVR4 implemented what is commonly called the page cache through which all file data is read and written. This is actually a somewhat vague term because the page cache differs substantially from the fixed size caches of the buffer cache, DNLC, and other types of caches. The page cache is composed of two parts, a segment underpinned by the seg_map segment driver and a list of free pages that can be used for any purpose. Thus, after a page of file data leaves the cache, it is added to the list of free pages. While the page is on the free list, it still retains its identity so that if the kernel wishes to locate the same data prior to the page being reused, the page is removed from the free list and the data does not need to be re-read from disk. [...] Thus, whereas the buffer cache references file data by device and block number, the page cache references file data by vnode pointer and file offset." Someone not using his real name (talk) 01:07, 21 December 2013 (UTC)

SVR4's VM system and unified buffer cache were SunOS 4.x-derived (as part of "the Sun/AT&T deal"). Guy Harris (talk) 01:13, 21 December 2013 (UTC)

Well, in that case the SunOS 4 buffers weren't really unified, but only unified in the sense above (that they didn't store data but only metadata in the buffer cache.) I've skimmed the SunOS papers [4][5], but they aren't exactly clear on this. By the way, the aforementioned book has details on TruUNIX (p.160) and AIX (162) that the NetBSD paper didn't know much about. They pretty much copied SVR4, though AIX less so. Someone not using his real name (talk) 01:20, 21 December 2013 (UTC)

Actually, on a closer reading the 2nd paper does describe the same thing as the SVR4 had: "7.1. File Caching. Traditionally, buffers in the UNIX buffer cache have been described by a device number and a physical block number on that device. This use of physical layout information requires all file system types implemented on top of a a block device to translate (bmap) each logical block to a physical block on the device before it can be looked up in the buffer cache. In the new VM system, the physical memory in the system is used as a logical cache; each buffer (page) in the cache is described by an object name (vnode) and a (page-aligned) offset within that object. [...] 8.3. UFS Control Information. For the UFS implementation, the control information consists of the inodes, indirect blocks, cylinder groups, and super blocks. The control information is not part of the logical file and thus the control information still needs to be named by the block device offsets, not the logical file offsets. To provide the greatest flexibility we decided to retain the old buffer cache code with certain modifications for optional use by file system implementations. The biggest driving force behind this is that we did not want to rely on the system page size being smaller than or equal to the size of control information boundaries for all file system implementations. Other reasons for maintaining parts of the old buffer cache code included some compatibility issues for customer written drivers and file systems. In current versions of SunOS, what’s left of the old buffer cache is used strictly for UFS control buffers. We did improve the old buffer code so that buffers are allocated and freed dynamically. If no file system types choose to use the old buffer cache code (e.g., a diskless system), then no physical memory will be allocated to this pool. When the buffer cache is being used (e.g., for control information for UFS file systems), memory allocated to the buffer pool will be freed when demand for these system resources decreases." Someone not using his real name (talk) 01:41, 21 December 2013 (UTC)