mm: more likely reclaim MADV_SEQUENTIAL mappings

File pages mapped only in sequentially read mappings are perfect reclaim
canditates.

This patch makes these mappings behave like weak references, their pages
will be reclaimed unless they have a strong reference from a normal
mapping as well.

It changes the reclaim and the unmap path where they check if the page has
been referenced.  In both cases, accesses through sequentially read
mappings will be ignored.

Benchmark results from KOSAKI Motohiro:

    http://marc.info/?l=linux-mm&m=122485301925098&w=2

Signed-off-by: Johannes Weiner <hannes@saeurebad.de>
Signed-off-by: Rik van Riel <riel@redhat.com>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/mm/memory.c b/mm/memory.c
index 5e0e91c..99e8d5c 100644 (file)
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -767,7 +767,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
                        else {
                                if (pte_dirty(ptent))
                                        set_page_dirty(page);
-                               if (pte_young(ptent))
+                               if (pte_young(ptent) &&
+                                   likely(!VM_SequentialReadHint(vma)))
                                        mark_page_accessed(page);
                                file_rss--;
                        }

diff --git a/mm/rmap.c b/mm/rmap.c
index 53c56da..f01e922 100644 (file)
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -360,8 +360,17 @@ static int page_referenced_one(struct page *page,
                goto out_unmap;
        }
 
-       if (ptep_clear_flush_young_notify(vma, address, pte))
-               referenced++;
+       if (ptep_clear_flush_young_notify(vma, address, pte)) {
+               /*
+                * Don't treat a reference through a sequentially read
+                * mapping as such.  If the page has been used in
+                * another mapping, we will catch it; if this other
+                * mapping is already gone, the unmap path will have
+                * set PG_referenced or activated the page.
+                */
+               if (likely(!VM_SequentialReadHint(vma)))
+                       referenced++;
+       }
 
        /* Pretend the page is referenced if the task has the
           swap token and is in the middle of a page fault. */