Re: malloc problem/question in 2.1.3

[Mike Corbeil <mcorbeil at netrevolution dot com>]

Kaz Kylheku wrote:

> On Thu, 1 Jun 2000, Mike Corbeil wrote:
>
> > Date: Thu, 01 Jun 2000 15:06:14 -0400
> > From: Mike Corbeil <mcorbeil@netrevolution.com>
> > Reply-To: glibc-linux@ricardo.ecn.wfu.edu
> > To: glibc-linux@ricardo.ecn.wfu.edu
> > Subject: Re: malloc problem/question in 2.1.3
> >
> > David Ronis wrote:
> >
> > > Please ignore my previous post, I'd been free'ing an unrelated (and unused)
> > > block of memory when I shouldn't have been.   I don't understand exactly
> > > why this should give a SIGSEGV, but it's fixed.
> > >
> >
> > As you probably guessed, this is because free doesn't (at least didn't in this
> > case) check if the argument is valid or not, that is, if there's actually any
> > memory attached to the pointer passed as an argument.  However, why free
> > wouldn't check for something like this leaves guess work.
>
> The answer to that is easy: it's for performance reasons. The checks would
> be a waste of time in a correct program.

That rings a bell, vaguely, from remote past, and it's a theoretically rosy idea,
but the real world continually says that there's virtually no software, especially
of significant scope, which doesn't have a bug or oversight somewhere in it.
Theory is nice, but reality is the tester.

On the other hand, for people working on non-critical applications or systems, the
theory may be adequate.  For critical mission and mission critical systems,
however, robustness is perhaps more important than a slight degradation in
performance.

You're correct about the reason; however, am not sure why such a conceptually
simple verification would significantly impact performance.  A nano or micro-second
more probably is practically insignificant, for properly developed programs.


> If you need a debugging malloc library, you can get one: ElectricFence,
> dmalloc, etc.

I've come across EF several times and haven't had the opportunity to use it yet,
however never heard of dmalloc.  I'm familiar with malloc and calloc, but wasn't
aware of dmalloc.

> > If I'm not mistaken, then I vaguely recall someone saying, numerous years ago,
> > that free couldn't check this and that this is why free isn't as reliable or
> > friendly as some or many would like.
>
> The free function is an interface specified by ANSI C. The semantics of that
> interface are such that if you try to free a pointer that did not originate
> from the allocator, the behavior is undefined. Also, any use of an
> indeterminate pointer is undefined.  Undefined behavior means that the
> implementation is not required to do anything about it, possibly leading
> to unpredictable results.

Already understood that, however, as per above, I don't know why people perceive a
significant performance degradation just because of adding a little code to verify
a pointer.  I don't write compilers, etc., and have therefore been away from such
low level technicalities pretty much since finishing school in 89.

> There is no inherent unreliability in the malloc interface. When you misuse
> something and it fails, it indicates a lack of robustness, not reliability.
> When you use something in accordance with its specification and it fails
> anyway, that indicates unreliability.

Correct, but also word play which has no benefit when a person is killed or dies
because of faulty software, regardless if the fault was "predetermined" or not.
I've been called exacting.

Although, you're technically correct as per what the CS community theorists say, or
might say, in a sense there's nonetheless a lack of robustness when there's a lack
of reliability, and vice versa.  When robustness fails, so does reliability, and
vice versa, in a manner of conceptually speaking, particularly if we apply the
concept "in the eye of the beholder".

Take government for example.  They're not robust and because of this, they're also
not reliable, and vice versa.

Also, if you're referring to the malloc function, then that's not what this was
about.  The free function is the topic and although free is part of the malloc
interface, conceptually or otherwise, they're not the same functions.  malloc
doesn't need to check for this, afaik.

That gets into similar word play as your production did, to some extent.


> Robustness can be expensive, therefore in practice it's wortwhile achieving
> it only where it is essential, like between the program and the outside world
> (guarding against bad inputs) or at major interface boundaries (e.g. process
> and kernel).

That's always as far as I'm concerned, because it's always been essential on all
jobs I've performed.  I haven't worked at developing non-critical apps like MS Word
for example.  I've only worked on critical mission systems.  Actually, I did work
on a couple of non-critical apps, but the two combined make up about one year of
experience out of over eight.  Thoroughness is about all I've been required to
produce.

> > As a short aside, I think that this could be checked for in Perl using
> > "defined" (e.g., if defined $var ...), however C is a compiled language and I
> > don't think there's any such functionality, not afaik anyway.
>
> Checking whether a pointer refers to an existing, allocated object could
> require a potentially expensive search.  For example, the allocator may have to
> keep track of all allocated objects in a tree structure, and search this
> structure whenever something is freed.  The bounds checking patches for gcc 2.7
> did something like this (but for all objects, not just dynamic ones).

I understand the overhead, however that kind of searching is lessening wrt how
expensive it is with the increasing power of CPUs positively offsetting the time
for the overhead to be performed.

Nonetheless, like someone else pointed out, one can simply verify the pointer
before calling free and only call free if the pointer is valid.  However, I've
never had to do this when calling free, because I not only run-time test code
I produce, but also do visual walkthroughs to ensure the logic is correct.

This is why I can put testing code in shell scripts, test, and then pull out this
code before submitting programs to source code control.  Sometimes, I'll leave this
code in, for protection against maintenance changes.  Some environments don't
permit this, but in those which do, I'll leave this code in, unless there's so much
of it that it renders programs much more difficult to read (sometimes the readers
aren't programmers, but instead auditors or managers).

That approach has actually allowed me to avoid much run-time testing.  Sometimes, I
run the test and it passes, completely, the first time, and some would say that
this makes a programmer less productive, but the converse is the reality.

There are visual and run-time tests, i.e., proofing, and both should always be
applied.  ISO certainly supports this, one way or another.  I just do it in forward
motion:  Write, verify visually, correct what's visually detected to be wrong, and
then run-time test.

It's all so simple children can understand this.  However, no one is a thorough
walking, living, encyclopedia on all topics one might have been, be, or get
involved in.

mike