Re: gsl Development Query

[Randall Judd <judd at nosc dot mil>]

At 03:58 PM 09/02/2000 -0600, Gerard Jungman wrote:
>Randall Judd wrote:
>> 
>> VSIPL 1.0 is actually final now. It is available on the Documents page of
>> the VSIPL site. On that page you will find something called, I think, "The
>> basics requirements document". You might want to read that instead.
>
>Yes, I just read it. It's marked "DRAFT 0.6", but I suppose I
>should ignore that. There are also the "Core" and "Core Lite" documents,
>which seem to be just tables of types and function names.
I will check it out and complain to the proper person. I wrote it
originally, but then I lost control.

The Core, and Core Lite documents are what are called profiles. No vendor,
except perhaps SKY, has completed all of VSIPL yet. But they wanted to be
able to say they are VSIPL compliant. So we defined profiles to give them a
starting point and a way to claim compliance to something. Core Lite is
only about 129 function and is a very easy starting point for most vendors.
A great many, surprisingly so, current Navy applications appear to be
mappable to just a core lite implementation. The core profile has about 500
or so functions and include more complicated things. In the future their
may be more profiles. A profile is basically a defined set of functions
that has been approved by the VSIPL forum.

>
>Anyway, as you say, it certainly seems like there is
>nothing to prevent you from interfacing with GSL in
>a natural way. I'm not even sure what we could do
>to screw that up. We're just a little closer to
>the underlying memory, which, I think, is natural
>for our purposes.
I think I agree with this. I don't necessarily think you did it the best
way though. I don't know what all design constraints the GSL forum placed
on themselves.

>
>
>> First of all in VSIPL a view is always bound to a block. The blocks in GSL
>> just seem to be a place to keep your memory storage until your ready to
>> destroy it. In VSIPL blocks are more important. To find the begining of
>> your view in GSL you store a pointer to the beginning of data. In VSIPL we
>> store an offset into the block. The vendor may do other stuff under the
>> covers, but what the user sees is an offset.
>
>I understand. Your blocks are already at a higher level
>of abstraction, whereas ours really serve at the lowest
>level, for basic heap management. Again, no problems.
>I think.
>
>
>> The vendor has a complete type definition,
>> of course, but he does not give that to the user. What the user gets is a
>> set of VSIPL defined functions to create and manipulate the block. "Views"
>> (vector views, matrix views) on the block are treated the same way.
>
>Maybe I should say explicitly that I don't think we should
>expose things like the gsl_block attributes directly either.
>The only sense in which we do is that the struct is there
>in the header file. But I have always felt that this is
>just a matter of style in C; for instance, I can go look
>up lots of things in the standard C header files which I would
>never rely on in an application program.
>
>As you say, we're writing a library, not a specification.
>So the struct defs have to go somewhere. The only reason
>I make this obvious statement is because it seems to be
>one of those things that Tisdale is ranting about (at
>least as far as I understand anything he says).
>
>
>> What VSIPL does is define a blockbind function. What blockbind does is to
>> associate some regular memory with a VSIPL block. Then both VSIPL and the
>> user can get to that memory. However VSIPL insists on owning any memory it
>> might use, so we added functions to admit the block (and memory) to VSIPL
>> and a release function to give ownership of the data back to the user.
>> Admit and release are important. VSIPL does not necessarily use the user
>> memory in the expected way. It may not use it at all. The purpose of admit
>> and release is to force data consistency between the block and the memory
>> during a state transition.
>
>Ok.
>
>
>> So if you wanted a gsl vector, and a vsipl vector on the same data space
>> you could do (I don't know much about gsl so there may be an error here)
>> 
>> gsl_vector *a_gsl_vview = gsl_vector_alloc(N);
>> vsip_block_d *a_vsipl_block = vsip_blockbind_d(
>>         a_gsl_vview->block->data, N, VSIP_MEM_NONE);
>> vsip_vview_d *a_vsipl_vview = vsip_vbind_d(a_vsipl_block, 0, 1 , N);
>
>A minor point, but maybe important. Why not use
>the gsl_vector_ptr() method to get the pointer
>to the underlying type, rather than "a_gsl_vview->block->data"?
>Anyway, that is supposed to provide the necessary abstraction
>away from the explicit struct attributes.
I guess I don't understand why this is necessary?

>[Ahah. I see that this is the point, from your
> comments below. Ok, let's move on to that then...]
>
>[Aside to Brian: Is the damn definition for gsl_vector_ptr() missing?
> I can't find the implementation anywhere...]
>
>
>> NOTE: It is not a mistake when I attach VSIPL to the data in the GSL block
>> instead of the GSL view. You don't want the same data segment attached to
>> two different VSIPL blocks. This is one of the problems I have with the way
>> GSL did this, but it appears that you can with a little care create
>> matching gsl and VSIPL views for any number of subviews on the original
>> data set.
>
>Ok. This is the part I need to understand. Let me state what I
>understand and you can tell me if I am missing the point. The problem
>is that you would like to attach VSIPL blocks to GSL views,
>but the semantics do not match because GSL views might overlap,
>and VSIPL blocks cannot. So instead you have to go directly to
>the GSL block, which breaks any encapsulation we might have hoped for.
>

Ok. Here is the main problem in your understanding. I don't attach VSIPL
blocks to GSL views. The idea is to associate (a word I prefer to attach)
VSIPL blocks with GSL blocks (not views). Then define VSIPL vector and/or
matrix views which map the block data space the same way that the GSL views
map the GSL blocks data space. 

As I understand it your GSL block contains some data space. Then you slice
this data space up. The first of your vector views creates the block, data
space, and a vector. The vector view contains a pointer to the block, plus
a pointer to the beginning of the data. When you take of subview of the
vector you set the block pointer to NULL, get a pointer to the beginning of
the subviews data space, and then set the attributes of the subview to the
proper stride and length for the new vector, which must not exceed the data
space of the block.

In VSIPL every view contains a pointer to the block. The block contains the
entire possible data space. For a GSL vector view attached to a block the
view contains a pointer to the beginning of data, a length equal to the
data space length, and a stride equal to one. The corresponding VSIPL
vector view contains an offset of zero, a stride of one, and a length equal
to the length of the block.

So when I create a block I would want the block to encompass the entire GSL
block.

Now I could create a new block to do the GSL subview, but the new block
would encompass some of the same data space as the previous VSIPL block.
This would be an error from VSIPL's point of view. What I do in VSIPL is
set an offset into the data space so my subview points to the same data
space as the GSL subview. Then the stride and length would be the same as GSL.

For example

 gsl_vector *a_gsl_vview = gsl_vector_alloc(N);
/* we assume N >> 4 */
 gsl_vector *a_gsl_subview = gsl_vector_subvector(a_gsl_vview, 4, N/2);
 gsl_vector *another_gsl_subview = gsl_vector_subviector(a_gsl_subview,2,N/4);
 vsip_block_d *a_vsipl_block = vsip_blockbind_d(
         a_gsl_vview->block->data, N, VSIP_MEM_NONE);
 vsip_vview_d *a_vsipl_vview = vsip_vbind_d(a_vsipl_block, 0, 1 , N);
 vsip_vview_d *a_vsipl_subview = vsip_vbind_d(a_vsipl_block,4,1, N/2);
 vsip_vview_d *another_vsipl_subview = vsip_vbind_d(a_vsipl_block,6,1, N/4);

/* or */
/* vsip_vview_d *a_vsipl_subview = vsip_vsubview_d(a_vsipl_vview,4,N/2); */
/* vsip_vview_d *another_vsipl_subview = vsip_vsubview_d(a
vsipl_sub_view,2,N/4); */

>If this is the problem, then the right solution is, as you do,
>to go directly to the block. So then my question is, what
>would you like to see us do with gsl_block in order to make
>this mapping easier?
Well your GSL block is fine, as far as I can tell. I don't know how you
have done complex blocks. The problem is the view. It is not a big problem,
unless for some reason you are trying to back the information out instead
of knowing it up front. If you supplied an offsets instead of a data
pointer I would have an easy map between VSIPL and GSL. Right now I have to
know the offset. I can't get it from the view. If I know the parent view I
can figure it out, but it is not something one can get from the view
itself. If you kept the block around it would be doable, but you seem to
keep the block in only one view. I think you do this to control when the
data is destroyed, but their are better ways to do this. I can't think of
any STRONG reason you need to change anything. It is not what I would have
done, but it works. 

Actually, if you just added an offset and length to the view, so the
current pointer minus the offset would point to the beginning of the block,
and the length would be the length of the original block, then you would
have all the information of the original block carried with the new view
and it would not change any of your current programs. It might be a pain
keeping the offset information current, but a minor pain.

If I were doing it from scratch I probably would have made a set of very
tightly coupled blocks and views as abstract data types with functions to
return memory pointers, lengths and strides, and also to set memory
pointers lengths and strides. The internals of the blocks and views are not
important as long as your programers can get to the underlying memory
information for whatever particular function they are working on. This
would give you a support library which could be controlled by the main
forum, and all any of the other GSL function programers would need to do is
use this well understood support library when building their function. Of
course the initial support library definition may have been hard to define.
I know how these meetings go. But once done it would give you a common API
while allowing a diverse set of programmers to do their own thing. Right
now your API is not very consistent. 

>
>Do you think gsl needs another abstraction in the middle,
>which is closer to the VSIPL block notion? That might be
>overkill, but it wouldn't be too hard.
Since you are doing a C library, I don't see any reason to do this
abstraction. You may want to add methods to allow people to attach special
memory to one of you blocks, but since your block is already public they
can do that as is.

>
>Is the real question here ownership and the possibility
>of overlaps, or is it something else?
VSIPL did the memory abstraction of blocks for portability. Their is no way
to write portable software on diverse hardware if the user manages memory
directly. The VSIPL library manages memory, and a "block" looks the same on
any system, independent of hardware. One of the major drivers of VSIPL is a
need to port code from old hardware to new hardware. The cost of the code
port is huge. With the advent of COTS (commercial of the shelf) in
government programs when the upgrade cycle is a few years instead of
decades, the software port needs to be easy. The need for portable legacy
code is the driving factor behind VSIPL.

VSIPL has some of the same type of overlap requirements as GSL except we
allow a fair amount of in-place operations.

>
>
>> VSIPL uses an offset to do this. VSIPL views are more flexible
>> than GSL views, so not any VSIPL view is mapable to a GSL view, but with
>> knowledge of both libraries it seems to be pretty reasonable to use them
>> together, at least at first glance.
>
>GSL views may be less flexible simply because we have not
>fleshed them out more. But I don't see them as fundamentally
>less flexible. Do you agree, or do you think there is a
>real flaw there?
>
You define all your matrix views as being row major, unit stride in the row
direction. I assume when your people do their programing they use this
assumption to produce higher performance code, so I would consider it to be
fundamentally less flexible. Fleshing them out more won't fix all the
underlying code.

I don't know that this is a bad thing. It's just the way you did it.

VSIPL matrix views are either row major or column major, and we allow non
unit strides in the major direction. Also all views can be coerced into
looking at any part of the block. Their is no fundamental difference
between a parent view and a subview in VSIPL. You can make a subview a
clone of the parent by setting it's attributes to be the same as the parent. 

Also, I am not sure how you do complex exactly. I see a complex block, but
I have not found where you define it, or how a view is mapped on it. VSIPL
can create (real) views of the real or imaginary parts of a complex view
(this is a little tricky because a real view is bound to type vsip_block_f
and c complex view is bound to type vsip_cblock_f). You actually view the
same data space, and if you change the data in one view the corresponding
data will change in the other view.

I don't really know enough about how your views operate. For instance I see
where you can create a block, but I don't see what you can do with it once
it's created. It seems like, the way you have designed things, you would
always create a vector or matrix first. I suppose you can always brute
force the block into a vector. You have access to all your structures.
VSIPL has functions to handle all this.

>Anyway, if I understand what you mean, then the right thing
>to do is to make it easier to do the correct
>mapping, which is blocks to blocks, with the 
>two separate view abstractions remaining in
>their own worlds. There may be no reason to
>attempt to map the two different views to
>each other in any way.
This is right. Most people will probably not need to have a one to one
mapping of views between VSIPL and GSL. In fact they may want entirely
different views of the same data space, one in VSIPL and one in GSL. My
main goal here is to see how the two libraries can be used together to
allow users the most power to produce applications. What we don't know
won't help our efforts.

           Randy Judd