In 2.4.19 (also 2.5.46) setrlimit code only ever makes a comparison to
check the old soft limit with the new soft limit and the new hard
limit with the old hard limit.  There is never a check to ensure the
new soft limit never exceeds the new hard limit. 
Just try "ulimit -H -m 10000" for memory limits that were not
previously set.  You end up with (hard limit = 10000) < (soft limit =
unlimited).
Fix is trivial.
--- sys.c       Sat Aug  3 10:39:46 2002
+++ edited.sys.c        Mon Nov 11 14:49:19 2002
@@ -1118,6 +1118,8 @@
 
        if (resource >= RLIM_NLIMITS)
                return -EINVAL;
+       if (new_rlim.rlim_cur > new_rlim.rlim_max)
+               return -EINVAL;
        if(copy_from_user(&new_rlim, rlim, sizeof(*rlim)))
                return -EFAULT;
        old_rlim = current->rlim + resource;
-- Kingsley - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/