[PATCH] __vm_enough_memory() signedness fix
authorSimon Derr <Simon.Derr@bull.net>
Fri, 5 Aug 2005 02:52:03 +0000 (19:52 -0700)
committerLinus Torvalds <torvalds@g5.osdl.org>
Fri, 5 Aug 2005 04:43:14 +0000 (21:43 -0700)
commit2f60f8d3573ff90fe5d75a6d11fd2add1248e7d6
tree173980b12459aa55586d3f4f56630abf5af1ffcc
parentb68e9f857271189bd7a59b74c99890de9195b0e1
[PATCH] __vm_enough_memory() signedness fix

We have found what seems to be a small bug in __vm_enough_memory() when
sysctl_overcommit_memory is set to OVERCOMMIT_NEVER.

When this bug occurs the systems fails to boot, with /sbin/init whining
about fork() returning ENOMEM.

We hunted down the problem to this:

The deferred update mecanism used in vm_acct_memory(), on a SMP system,
allows the vm_committed_space counter to have a negative value.

This should not be a problem since this counter is known to be inaccurate.

But in __vm_enough_memory() this counter is compared to the `allowed'
variable, which is an unsigned long.  This comparison is broken since it
will consider the negative values of vm_committed_space to be huge positive
values, resulting in a memory allocation failure.

Signed-off-by: <Jean-Marc.Saffroy@ext.bull.net>
Signed-off-by: <Simon.Derr@bull.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
mm/mmap.c
mm/nommu.c