RE: RtlFillMemory fails on block sizes over 0x7fffffff

["Petrovski, Roman" <RPetrovski at illumina dot com>]

Sure, in case you decide to go with the patch meanwhile, please use the one attached to this email. The original wrongly uses n instead of size for copying the memory.

As this is fairly critical issue, do you know when users should expect a  fix to become available in the binary release?

Roman.


From 3ba2e2feaf785c213d2f3db16efab74e25347b43 Mon Sep 17 00:00:00 2001
From: Roman Petrovski <rpetrovski@illumina.com>
Date: Wed, 29 Jul 2015 09:20:19 -0700
Subject: [PATCH] RtlFillMemory fails on block sizes over 0x7fffffff

---
 winsup/cygwin/miscfuncs.cc | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/winsup/cygwin/miscfuncs.cc b/winsup/cygwin/miscfuncs.cc
index 4a7a1b8..2c38de3 100644
--- a/winsup/cygwin/miscfuncs.cc
+++ b/winsup/cygwin/miscfuncs.cc
@@ -904,17 +904,35 @@ err:
 extern "C" void NTAPI RtlFillMemory (PVOID, SIZE_T, BYTE);
 extern "C" void NTAPI RtlCopyMemory (PVOID, const VOID *, SIZE_T);

+
+static const size_t RTL_MAX_SIZE = 0x7fffffff;
 extern "C" void *
 memset (void *s, int c, size_t n)
 {
-  RtlFillMemory (s, n, c);
+  char *p = (char*)s;
+  while (n)
+  {
+    size_t size = min(RTL_MAX_SIZE, n);
+    RtlFillMemory (p, size, c);
+    p += size;
+    n -= size;
+  }
   return s;
 }

 extern "C" void *
 memcpy(void *__restrict dest, const void *__restrict src, size_t n)
 {
-  RtlCopyMemory (dest, src, n);
+  char *d = (char*)dest;
+  char *s = (char*)src;
+  while (n)
+  {
+    size_t size = min(RTL_MAX_SIZE, n);
+    RtlCopyMemory (d, s, size);
+    d += size;
+    s += size;
+    n -= size;
+  }
   return dest;
 }
 #endif
--
2.4.5


-----Original Message-----
From: cygwin-patches-owner@cygwin.com [mailto:cygwin-patches-owner@cygwin.com] On Behalf Of Corinna Vinschen
Sent: 29 July 2015 17:22
To: cygwin-patches@cygwin.com
Subject: Re: RtlFillMemory fails on block sizes over 0x7fffffff

On Jul 29 14:11, Petrovski, Roman wrote:
> Hi, just ran into a problem which boils down to the following at least with Windows 7:
> 
> char *p = (char*)malloc(0x80000000UL);	//works fine, allocates memory as requested
> memset(p, 0, 0x80000000UL);			//Watch process segfault.
> 
> The RtlFillMemory either crashes or underfills the buffer depending on the size given.
> Looks like internally it treats size as a signed 4-byte integer.
> 
> Please apply the patch below or implement an alternative.

Thanks for the patch, but I'll rather be looking into an assembler alternative.  I'm planning to pull in the NetBSD implementation, with the tweaks required for MS ABI.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat