<= Predictor of DVM-program performance. Detailed design.
(beginning)

Predictor of DVM-program performance.
Detailed design (continuation)
* October, 2000*

- last edited 22.05.01 -

Appendix 1. Link from field name in output HTML-file to field name in structure _IntervalResult

Field Name     Anchor inter variable
         
Efficiency     Effic Efficiency
Execution time     Exec Execution_time
Total time     Total Total_time
Productive time     Ptime Productive_time
  CPU   Ptimec Productive_CPU_time
  SYS   Ptimes Productive_SYS_time
  I/O   Ptimei IO_time
Lost time     Lost Lost_time
  Insufficient parallelism   Insuf Insuff_parallelism
    USR iuser Insuff_parallelism_sys
    SYS isyst Insuff_parallelism_usr
  Communications   comm Communication
    SYN csyn Communication_SYNCH
  Idle time   idle Idle
Load imbalance     imbal Load_imbalance
Synchronization     synch Synchronization
Time variation     vary Time_variation
Overlap     over Overlap
         
IO # op   nopi Num_op_io
  Communications   comi IO_comm
  Real synch   synchi IO_synch
  Overlap   overi IO_overlap
Reduction # op   nopr Num_op_reduct
  Communications   comr Wait_reduction
  Real synch   synchr Reduction_synch
  Overlap   overr Reduction_overlap
Shadow # op   nops Num_op_shadow
  Communications   coms Wait_shadow
  Real synch   synchs Shadow_synch
  Overlap   overs Shadow_overlap
Remote access # op   nopa Num_op_remote
  Communications   coma Remote_access
  Real synch   syncha Remote_synch
  Overlap   overa Remote_overlap
Redistribution # op   nopd Num_op_redist
  Communications   comd Redistribution
  Real synch   synchd Redistribution_synch
  Overlap   overd Redistribution_overlap

Appendix 2. Definition of auxiliary functions and classes

Below there is a description of functions and classes used in implementation of algorithms described in the previous chapter.

// Base class for most of the classes
class Space {
protected:
	long Rank; 	      	  // Number of space dimensions
	vector<long> SizeArray;	  // Size of each dimension
	vector<long> MultArray;	  // Multiplier for each dimension
public:
	Space();
	Space(long ARank, vector<long> ASizeArray, vector<long> MultArray);
	Space(long ARank, long *ASizeArray);
	Space(const Space &);
	~Space();

	Space & operator= (const Space &x);
	long GetRank();
	long GetSize(long AAxis); 
	void GetSI(long LI, vector<long> & SI); 
	long GetLI(const vector<long> & SI); 
	long GetCenterLI();
	long GetSpecLI(long LI, long dim, int shift);
	long GetLSize();
	long GetNumInDim(long LI, long dimNum);
	long GetDistance(long LI1, long LI2); 
};
GetRank – returns space rank.
GetSize – returns size of the space with number AAxis.
GetSI – calculates coordinates of SI by the linear index LI.
GetLI – calculates the linear index by coordinates in the given space.
GetCenterLI – returns the linear index of the element that is the geometric center of the space.
GetSpecLI – returns the linear index of the element moved by shift in the dimension dim from the element with linear index LI.
GetLSize – returns linear size (number of elements) of the space.
GetNumInDim – returns coordinate of the element with linear index LI in the given dimension dimNum.
GetDistance – distance between two elements of the space with linear indexes LI1 and LI2.


“Virtual machine” (“Processor system”) class.

class VM : public Space {
	int MType;	// distributed processor system type
			// 0 – net with bus organization, 1 – transputer system
	double TStart;	// Start time of exchange operation
	double TByte;	// Tyme to send one byte
public:
			// constructor
	VM(vector<long>& ASizeArray, int AMType, double ATStart, double ATByte, 
		double AProcPower);
	~VM();
	double	 getTByte();
	double	getTStart();
	int 	getMType();
 };


“Abstract machine representation” class.

class AMView : public Space  {
public:	 
	VM *VM_Dis;			// Processor system on which the template is mapped
	list<DArray *> AlignArrays;	// List of arrays aligned by the given template
	vector<DistAxis> DistRule;	// Rule by which the template is mapped
					// on the processor system
	vector<long> FillArr;		// Array containing the information about how the processor
					// system is filled with the template elements

	AMView(long ARank, long *ASizeArray);
	AMView(const AMView &); 
	~AMView();

	void DelDA(DArray *RAln_da); 
	void AddDA(DArray *Aln_da); 
	void DisAM(VM *AVM_Dis, long AParamCount, long *AAxisArray,
		long *ADistrParamArray);

	double RDisAM(long AParamCount,  long *AAxisArray, long *ADistrParamArray,
		long ANewSign);
	bool IsDistribute();
};
DelDA - removes DArray from the list of aligned arrays.
AddDA - adds DArray to the list of the aligned arrays.
DisAM - function that maps the template onto processor system. The pointer on the processor system, mapping rule and array with information about filling the processor system with template elements are initialized according to function parameters.
RdisAM - function that determines the time spent in exchanges when the template mapping is changed (template redistribution). The algorithm implemented in it is described in 3.2.
IsDistribute - checks if the template is already distributed on the processor system.


“Distributed array” class.

class DArray : public Space {
private:
	void PrepareAlign(long& TempRank, long *AAxisArray, long *ACoeffArray,
		long *AConstArray, vector<AlignAxis>& IniRule);
	long CheckIndex(long *InitIndexArray, long *LastIndexArray, long *StepArray); 
public:
	long TypeSize;			// Size of one array element in bytes.
	AMView *AM_Dis;			// Template the array is aligned by.
	vector<AlignAxis> AlignRule	// Align rule.
	int Repl;			// Criterion of fully replicated array. 

	DArray();
	DArray(long ARank, long *ASizeArray, long ATypeSize);
	DArray(const DArray &);
	~DArray();
	DArray & operator= (DArray &x);

	void AlnDA(AMView *APattern, long *AAxisArray, long *ACoeffArray,
		long *AConstArray);
	void AlnDA(DArray *APattern, long *AAxisArray, long *ACoeffArray,
		long *AConstArray);
	double RAlnDA(AMView *APattern, long *AAxisArray, long *ACoeffArray,
		long *AConstArray, long ANewSign);
	double RAlnDA(DArray *APattern, long *AAxisArray, long *ACoeffArray,
		long *AConstArray, long ANewSign);
	friend double ArrayCopy(DArray *AFromArray, long *AFromInitIndexArray, 
		long *AFromLastIndexArray, long *AFromStepArray, DArray *AToArray, 
		long *AToInitIndexArray, long *AToLastIndexArray, long *AToStepArray, 
		long ACopyRegim);
	long GetMapDim(long arrDim, int &dir); 
	bool IsAlign();
};
PrepareAlign – initializes the rule by which the array is aligned by the template.
CheckIndex – returns number of elements in the array section given in the function parameters (0 – if it is empty or array indexes are out of bounds).
AlnDA – functions that set the position (alignment) of the distributed array. In the second function, the template is set indirectly through the distributed array. Template pointer is initialized. In the first function, the fully replicated array criterion is determined. In the second, it is inherited from the array that acts as a mapping template. Also, the align rule is initialized in the first function using the PrepareAlign function, and besides, in the second function the rule is altered according to how the template is aligned (alignment superposition is used to receive the resulting alignment).
RAlnDA – function that determines the time needed for exchanges during the array realignment. Algorithm is described in 3.2.
ArrayCopy – function that determines the time needed for exchanges while loading the buffers with remote array elements. Algorithm is described in 3.5.
GetMapDim – function returns number of the processor system dimension on which the arrDim array dimension is mapped as a result. If the array dimension is replicated by all the directions of processor matrix, 0 is returned. 1 or –1 is put into dir, according to the direction of array dimension break-down.
IsAlign – checks if the array is aligned by the template.


”Bound group” class.

class BoundGroup  {
	AMView *amPtr;		// Template by which the arrays with bounds in the group
				// are aligned 
	CommCost boundCost; 	// Processor exchange information.
public:
	BoundGroup();
	virtual ~BoundGroup();
	void AddBound(DArray *ADArray, long *ALeftBSizeArray, 
		long *ARightBSizeArray, long ACornerSign);
	double StartB();
};
AddBound – inclusion of the distributed array bound in the bound group. Algorithm is described in 3.3.
StartB – function that determines the time spent in distributed array bound exchanges, with the bounds that are in the group. Algorithm is described in 3.3.


“Reduction variable” class.

class RedVar  {
public:
	long RedElmSize; 	// Size of the reduction variable–array in bytes
	long RedArrLength; 	// Number of elements in the reduction variable-array
	long LocElmSize;   	// Size of one element of the array with auxiliary information 

	RedVar(long ARedElmSize, long ARedArrLength, long ALocElmSize);
	RedVar();
	virtual ~RedVar();
	long GetSize();
};
GetSize – returns the size of the reduction variable and of the array with auxiliary information in bytes.


“Reduction group” class.

class RedGroup  {
public:
	VM *vmPtr;		    // Pointer to the processor system
	vector<RedVar *> redVars;   // Array of reduction variables
	long TotalSize;		    // Total size of reduction variables in the group with their
				    // auxiliary information, in bytes
	long CentralProc;	    // Linear index of the geometrical center of the processor
				    //system

	RedGroup(VM *AvmPtr);
	virtual ~RedGroup();	
	void AddRV(RedVar *ARedVar); 
	double StartR(DArray *APattern, long ALoopRank, long *AAxisArray);
};
void AddRV – inclusion of the reduction variable in the reduction group. Algorithm is described in 3.4.
StartR – function that returns the time spent in exchanges during the reduction operation. Algorithm is described in 3.4.


“Distribution of the array dimension” class.

class DistAxis  {
public:
	long Attr;     // Distribution type
	long Axis;     // Number of the template dimension 
	long PAxis;    // Number of the processor system dimension

	DistAxis(long AAttr, long AAxis, long APAxis);
	DistAxis();
	virtual ~DistAxis();
	DistAxis& operator= (const DistAxis&);
};

“Alignment of the distributed array by the template” class.

class AlignAxis  {
public:
	long Attr; 	// Distribution type
	long Axis;	// Number of the array dimension
	long TAxis;	// Number of the template dimension
	long A;		// Coefficient for the index variable of the array in the linear align rule of
			//the TAxis template dimension
	long B;		// Constant of the linear align rule for the TAxis template dimension
	long Bound;	// Dimension size of the array that acts as a template
			//during the partial replication of the array being aligned

	AlignAxis(long AAttr, long AAxis, long ATAxis,  long AA = 0, long AB = 0, long ABound = 0);
	AlignAxis();
	virtual ~AlignAxis();
	AlignAxis& operator= (const AlignAxis&); 

};

“Shadow edge by one distributed array dimension” class.

class DimBound  {
public:
	long arrDim;		// Array dimension number
	long vmDim;		// Processor system dimension number
	int dir;		// 1 or –1 according to the break-down direction of the array dimension
	long LeftBSize;		// Width of the left bound for the arrDim array dimension
	long RightBSize;	// Width of the right bound for the arrDim array dimension

	DimBound(long AarrDim, long AvmDim, int Adir, long ALeftBSize,  long ARightBSize);
	DimBound();
	virtual ~DimBound();
};

“Array section” class.

class Block {
	vector<LS> LSDim;   // Vector containing the corresponding linear segments for every array
			    // dimension, that describe the section
public: 
	Block(vector<LS> &v); 
	Block(DArray *da, long ProcLI);
	Block();
	virtual ~Block();
	Block & operator =(const Block & x);

	long GetRank();
	long GetBlockSize(); 
	long GetBlockSizeMult(long dim); 
	long GetBlockSizeMult2(long dim1, long dim2);
	bool IsLeft(long arrDim, long elem); 
	bool IsRight(long arrDim, long elem);
	bool IsBoundIn(long *ALeftBSizeArray,long *ARightBSizeArray);
	bool empty();
	friend Block operator^ (Block &x, Block &y); 
};
Block – creates da array section situated on the processor with ProcLI linear index.
GetRank – returns rank of the section.
GetBlockSize – number of elements in the section.
GetBlockSizeMult,GetBlockSizeMult2 – these functions return the result of multiplying sizes of the section in all the dimensions except dimensions that have been given in the function call.
IsLeft, IsRight – checks if the element elem is positioned to the left (right) of the section in the arrDim dimension.
IsBoundIn – checks if the distributed array bound is in the given section.
Empty – checks if the section has no elements.
Block operator^ – returns the intersection of the sections given in the function call.


“Linear segment” class.

class LS  {
public:	
	long Lower; 	 // Lower index value
	long Upper;	 // Upper index value

	LS(long ALower, long AUpper);
	LS();
	virtual ~LS();

	long GetLSSize(); 
	void transform(long A, long B, long daDimSize);
	bool IsLeft(long elem);
	bool IsRight(long elem);
	bool IsBoundIn(long ALeftBSize, long ARightBSize); 
	bool empty();
	LS operator^ (LS &x); 
};
GetLSSize – returns the size of the linear segment.
Transform – transforms the linear segment of the template into the linear segment of the distributed array aligned by the given template.
IsLeft, IsRight – check that the elem element is to the left(right) of the segment.
IsBoundIn – check that the given bound is in the bounds of the segment.
empty – check if there are no elements in the segment.
LS operator – segment intersection operator.


“Evaluation of interprocessor exchanges” class.

class CommCost {
public:
	Dim2Array transfer;   	// Array that contains the information about the number of bytes 
				// transferred between two processors
	VM *vm;  		// Pointer to the processor system

	CommCost(VM *Avm);
	CommCost();
	virtual ~CommCost();
	CommCost & operator =(const CommCost &);

	double GetCost();  
	void Update(DArray *oldDA, DArray *newDA);
 	void BoundUpdate(DArray *daPtr, vector<DimBound> & dimInfo, bool IsConer); 
	void CopyUpdate(DArray *FromArray, Block & readBlock);
};
GetCost – returns the time spent in interprocessor exchanges inside the system. Algorithm is described in 3.2.
Update – function that alters the transfer array according to the exchanges between the processors that occur during redistribution of the array. Algorithm implemented is described in 3.2.
BoundUpdate – function that changes the transfer array according to transfers that occur during the given distributed array bound exchange. Algorithm is described in 3.3.
CopyUpdate – function that changes the transfer array according to exchanges that occur during the replication of the readBlock section of the FromArray by all the processors.

 

Appendix 3. Main functions of time extrapolation

Constructor of the “Virtual machine” object

VM::VM( vector<long> ASizeArray, int AMType, double ATStart, double ATByte, double AProcPower );

ASizeArray – vector, element in i-th position is the size of the given processor system in dimension i + 1 (0 £ i £ ARank – 1);
AMType – type of the distributed processor system (0 – net with bus organization, 1 – transputer system);
ATStart – start time of the exchange operation;
ATByte – time to send one byte;
AProcPower – relative processor power.
     

Constructor of the “Abstract machine representation” object

AMView::AMView( vector< long> ASizeArray );

ASizeArray – vector, element in i-th position is the size of the template in dimension i+1 (0 £ i £ ARank–1).


Template mapping

void AMView::DisAM (ImLastVM *AVM_Dis, vector<long> AAxisArray,
                                        vector<long> *ADistrParamArray );

AVM_Dis – pointer to the processor system on which the template is mapped;
AAxisArray – vector, element in j-th position is the number of the template dimension which is used in the mapping rule for (j+1)-th processor system dimension;
ADistrParamArray – ignored (only two mapping rules are provided (read Lib-DVM documentation). In the first rule the block size is calculated, not taken from AdistrParamArray).
   

Task of redistributing the template on the processor system and evaluating the time of the redistribution.

double AMView::RdisAM( vector<long> AAxisArray,
                                              vector<long> ADistrParamArray, long ANewSign );

AAxisArray – vector, element in j-th position is the number of the template dimension which is used in the mapping rule for (j+1)-th processor system dimension;
ADistrParamArray – ignored (only two mapping rules are provided (read Lib-DVM documentation). In the first rule the block size is calculated, not taken from AdistrParamArray);
ANewSign – flag of updating contents of the redistributed arrays, active if value is 1.
     

Constructor of the “Distributed array” object

DArray::DArray( vector<long> ASizeArray, vector<long> AlowShdWidthArray,
                               vector<long> AhiShdWidthArray, long ATypeSize );

ASizeArray – vector, element in i-th position contains the size of the array being created, in dimension i+1 (0 £ i £ ARank–1).
AlowShdWidthArray – vector, element in i-th position contains the width of the left boundary, in dimension i+1.
AhiShdWidthArray – vector, element in i-th position contains the width of the right boundary, in dimension i+1.
ATypeSize – size of one array element in bytes.
     

Distributed array alignment

void DArray::AlnDA(AMView *APattern, vector<long> AAxisArray,
                                    vector<long> ACoeffArray, vector<long> AConstArray );

void DArray::AlnDA(DArray *APattern, vector<long> AAxisArray,
                                    vector<long> ACoeffArray, vector<long> AConstArray );

APattern – pointer to the align pattern.
AAxisArray – vector, element in j-th position contains number of the index variable (number of the dimension) of the distributed array for the linear align rule of the (j+1)-th pattern dimension.
ACoeffArray – vector, element in j-th position contains the coefficient for the index variable of the distributed array in the linear align rule of the (j+1)-th pattern dimension.
AConstArray – vector, element in j-th position contains constant for the linear align rule of the (j+1)-th pattern dimension.
     

Realignment of the distributed array. Evaluation of the time needed to perform this operation.

double DArray::RAlnDA( AMView *APattern, vector<long> AAxisArray,
                                            vector<long> ACoeffArray, vector<long> AConstArray,
                                            long ANewSign );

double DArray::RAlnDA( DArray *APattern, vector<long> AAxisArray,
                                            vector<long> ACoeffArray, vector<long> AConstArray,
                                            long ANewSign );

APattern – pointer to the align pattern (array or template).
AAxisArray – vector, element in j-th position contains number of the index variable (number of the dimension) of the distributed array for the linear align rule of the (j+1)-th pattern dimension.
ACoeffArray – vector, element in j-th position contains the coefficient for the index variable of the distributed array in the linear align rule of the (j+1)-th pattern dimension.
AConstArray – vector, element in j-th position contains constant for the linear align rule of the (j+1)-th pattern dimension.
ANewSign – flag of updating contents of the redistributed array, active if value is 1.

The function returns time of the array realignment.


Constructor of the “Parallel loop” object.

ParLoop::ParLoop( long ARank );

ARank – rank of the parallel loop.
     

Creation of the parallel loop.

void ParLoop::MapPL( AMView *APattern, vector<long> AAxisArray,
                                         vector<long> ACoeffArray, vector<long> AConstArray,
                                         vector<long> AInitIndexArray,
                                         vector<long> ALastIndexArray, vector<long> AStepArray );

void ParLoop::MapPL( DArray *APattern, vector<long> AAxisArray,
                                         vector<long> ACoeffArray, vector<long>AConstArray, vector<long>AInitIndexArray,
                                         vector<long> ALastIndexArray, vector<long>AStepArray );

APattern – pointer to the parallel loop pattern.
AAxisArray – vector, element in j-th position contains number of the index variable (number of the dimension) of the parallel loop for the linear align rule of the (j+1)-th pattern dimension.
ACoeffArray – vector, element in j-th position contains the coefficient for the index variable of the parallel loop in the linear align rule of the (j+1)-th pattern dimension.
AConstArray – vector, element in j-th position contains constant for the linear align rule of the (j+1)-th pattern dimension.
AInitIndexArray – vector, element in i-th position contains start value for the index variable of the (i+1)-th parallel loop dimension.
ALastIndexArray – vector, element in i-th position contains end value for the index variable of the (i+1)-th parallel loop dimension.
AStepArray – vector, element in i-th position contains step value for the index variable of the (i+1)-th parallel loop dimension.
     

Parallel loop mapping.

void ParLoop::ExFirst( ParLoop *AParLoop, BoundGroup *ABoundGroup)ImLast;

AParLoop – pointer to the parallel loop.
ABoundGroup – pointer to the group of bounds that must be exchanged after calculating the exported elements of the local parts of the distributed arrays.
     

Set flag of changed order of loop iteration execution.

void ParLoop::ImLast( ParLoop *AParLoop, BoundGroup *ABoundGroup)ImLast;

AParLoop – pointer to the parallel loop.
ABoundGroup – pointer to the group of bounds that must be exchanged after calculating the exported elements of the local parts of the distributed arrays. The function sets flag of changed order of loop iteration execution.
     

Calculation of the time spent for exchanges while loading buffers by remote array elements

double ArrayCopy( DArray *AFromArray, vector<long> AFromInitIndexArray,
                                 vector<long> AFromLastIndexArray,
                                 vector<long> AFromStepArray, DArray *AToArray,
                                 vector<long> AToInitIndexArray,
                                 vector<long> AToLastIndexArray,
                                 vector<long> AToStepArray, long ACopyRegim );

AFromArray – pointer to the distributed array under reading.
AFromInitIndexArray – vector, element in i-th position contains start value for the index variable of the (i+1)-th dimension of the array under reading.
AFromLastIndexArray – vector, element in i-th position contains end value for the index variable of the (i+1)-th dimension of the array under reading.
AFromStepArray – vector, element in i-th position contains step value for the index variable of the (i+1)-th dimension of the array under reading.
AToArray – header of written distributed array.
AToInitIndexArray – vector, element in j-th position contains start value for the index variable of the (j+1)-th dimension of written distributed array.
AToLastIndexArray – vector, element in j-th position contains end value for the index variable of the (j+1)-th dimension of written distributed array.
AToStepArray – vector, element in j-th position contains step value for the index variable of the (j+1)-th dimension of written distributed array.
ACopyRegim – copy mode.

The function returns required time.


Constructor of the “Edge group” object

BoundGroup::BoundGroup( );

Creation of the edge group. An empty edge group is created (does not contain any edge).


Add an array edges into the group.

void BoundGroup::AddBound( DArray *ADArray, vector<long> ALeftBSizeArray,
                                                    vector<long> ARightBSizeArray, long ACornerSign);

ADArray – pointer to the distributed array.
ALeftBSizeArray – vector, element in i-th position contains width of the low edge of (i+1)-th dimension of the array.
ARightBSizeArray – vector, element in i-th position contains width of the high edge of (i+1)-th dimension of the array.
ACornerSign – flag of including corner elements in the edge.
     

Calculation of the time spent for exchanges of distributed array edges included in the group.

double BoundGroup::StartB( );

The function returns required time.


Constructor of the “Reduction variable” object

RedVar::RedVar( long ARedElmSize, long ARedArrLength, long ALocElmSize);

AredElmSize – size of one element of the reduction variable-array in bytes.
ARedArrLength – number of elements in the reduction variable-array.
ALocElmSize – size of one element of the array with auxiliary information in bytes.
     

Constructor of the “Reduction group” object

RedGroup::RedGroup( VM *AvmPtr );

AvmPtr – pointer to the processor system.
     

Creation of the reduction group. An empty reduction group is created (does not contain any reduction variable).

Add reduction variable into the reduction group.

void RedGroup::AddRV( RedVar *ARedVar );

ARedVar – pointer to the reduction variable.
     

Calculation of the time spent for exchanges during reduction operation execution.

double RedGroup::StartR( ParLoop *AParLoop );

AParLoop – pointer to the parallel loop in which values of reduction variables of the given group are calculated.
     

Appendix 4. Trace fragments and parameters of Lib-DVM functions simulated by Predictor

CREATE AN ABSTRACT MACHINE REPRESENTATION

getamr_ 3.3 revision of pointers to element of abstract machine representation

AMRef getamr_ (AMViewRef *AMViewRefPtr, long IndexArray[]);

*AMViewRefPtr – pointer to the abstract machine representation.
IndexArray – array, i-th element contains the index value of the requested element (abstract machine) on (i+1)-th dimension. 
call_getamr_	TIME=0.000000 	LINE=6 	FILE=tasks.fdv
AMViewRefPtr=4dff90; AMViewRef=9024c0;
IndexArray[0]=0;  

ret_getamr_	TIME=0.000000 	LINE=6	FILE=tasks.fdv
AMRef=903350;

MULTIPROCESSOR SYSTEMS

genblk_ Weights of multiprocessor system elements

long genblk_(PSRef *PSRefPtr, AMViewRef *AMViewRefPtr,
                      AddrType AxisWeightAddr[], long *AxisCountPtr,
                      long *DoubleSignPtr );

*PSRefPtr – pointer to multiprocessor system, weights are set for elements of this system.
*AMViewRefPtr – pointer to the representation of multiprocessor system, weights of coordinates will be used while mapping the multiprocessor system on the given processor system.
AxisWeightAddr[] – weights of processor coordinates are defined for each dimension of processor system.
*AxisCountPtr – (nonnegative number) defines the number of elements in AxisWeightAddr array.
*DoubleSignPtr – non-zero flag of representation of processor weight coordinates as real positive numbers (double). 
call_genblk_     TIME=0.000000 LINE=7      FILE=gausgb.fdv
PSRefPtr=4d4c48; PSRef=8417d0; AMViewRefPtr=4d4c60; AMViewRef=842860; AxisCount=1; DoubleSign=0
AxisWeightAddr[0][0] = 3

ret_genblk_      TIME=0.000000 LINE=7      FILE=gausgb.fdv

crtps_ 4.2 Create subsystem of the given multiprocessor system

PSRef crtps_ (PSRef *PSRefPtr, long InitIndexArray[], long LastIndexArray[],
                         long *StaticSignPtr);

*PSRefPtr – pointer to the processor system (source), its subsystem is to be created.
InitIndexArray – array, i-th element contains the start value of the source processor system on (i+1)-th dimension.
LastIndexArray – array, i-th element contains the end value of the source processor system on (i+1)-th dimension.
*StaticSignPtr – flag of static subsystem creation. 
call_crtps_	TIME=0.000000	LINE=15	FILE=tasks.fdv
PSRefPtr=4ded68; PSRef=902450; StaticSign=0;
InitIndexArray[0]=0;  
LastIndexArray[0]=0;  

     SizeArray[0]=1;
     CoordWeight[0]= 1.00(1.00)  
ret_crtps_	TIME=0.000000 	LINE=15	FILE=tasks.fdv
PSRef=903950;

psview_ 4.3 Reconfiguration of multiprocessor system

PSRef psview_ (PSRef *PSRefPtr, long *RankPtr, long SizeArray[],
                            long *StaticSignPtr);

*PSRefPtr – pointer to the source processor system to be reconfigured.
*RankPtr – rank of resulting processor system.
SizeArray – array, i-th element contains the rank of resulting processor system on (i+1)-th dimension.
*StaticSignPtr – flag of static resulting processor system. 
call_psview_	TIME=0.000000 	LINE=6	FILE=tasks.fdv
PSRefPtr=4dff84; PSRef=901330; Rank=1; StaticSign=0;
SizeArray[0]=1;  

     SizeArray[0]=1;
     CoordWeight[0]= 1.00(1.00)  
ret_psview_	TIME=0.000000 LINE=6      FILE=tasks.fdv
PSRef=902450;

MAPPING DISTRIBUTED ARRAY

getamv_ 7.8 revision of pointer to abstract machine representation, which the given distributed array is mapped on

AMViewRef getamv_ (long * ArrayHeader);

ArrayHeader – header of the distributed array. 
call_getamv_	TIME=0.000000	LINE=16	FILE=tasks.fdv
ArrayHeader=4dfee8; ArrayHandlePtr=903530;
ret_getamv_	TIME=0.000000	LINE=16	FILE=tasks.fdv
AMViewRef=0;

PROGRAM AS AN AGGREGATE OF SUBTASKS EXECUTED IN PARALLEL

mapam_ 10.1 Mapping an abstract machine (create subtask)

long mapam_ (AMRef *AMRefPtr, PSRef *PSRefPtr );

*AMRefPtr – pointer to the abstract machine to be mapped.
*PSRefPtr – pointer to processor subsystem determining processors allocated for the abstract machine (domain of the created subtask execution). 
call_mapam_	TIME=0.000000 	LINE=51     	FILE=tsk_ra.cdv
AMRefPtr=4b3cc0; AMRef=823210; PSRefPtr=4b3ec4; PSRef=8231a0;
ret_mapam_	TIME=0.000000 	LINE=51     	FILE=tsk_ra.cdv

runam_ 10.2 Start of the subtask execution (activation, start)

long runam_ (AMRef *AMRefPtr);

*AMRefPtr – pointer to the abstract machine of the started subtask. 
call_runam_ 	 TIME=0.000000 	LINE=102   	 FILE=tsk_ra.cdv
AMRefPtr=4b3cc0; AMRef=823210;
ret_runam_	 TIME=0.000000 	LINE=102    	FILE=tsk_ra.cdv

stopam_ 10.3 End of execution of the current subtask (stop)

long stopam_ (void);

call_stopam_      TIME=0.000000 LINE=104    FILE=tsk_ra.cdv
ret_stopam_       TIME=0.000000 LINE=104    FILE=tsk_ra.cdv

REDUCTION

strtrd_ 11.5 Start of reduction group

long strtrd_ (RedGroupRef *RedGroupRefPtr);

*RedGroupRefPtr – pointer to reduction group. 
call_strtrd_      TIME=0.000000 LINE=129    FILE=tsk_ra.cdv
RedGroupRefPtr=6ffcdc; RedGroupRef=8291f0;
rf_MAX;    rt_DOUBLE; RVAddr = 6ffd24; RVVal = 7.000000
ret_strtrd_       TIME=0.000000 LINE=129    FILE=tsk_ra.cdv

waitrd_ 11.6 Waiting for the reduction completion

long waitrd_ (RedGroupRef *RedGroupRefPtr);

*RedGroupRefPtr – pointer to the reduction group. 
call_waitrd_      TIME=0.000000 LINE=129    FILE=tsk_ra.cdv
RedGroupRefPtr=6ffcdc; RedGroupRef=8291f0;
rf_MAX;    rt_DOUBLE; RVAddr = 6ffd24; RVVal = 7.000000
rf_MAX;    rt_DOUBLE; RVAddr = 6ffd24; RVVal = 7.000000

ret_waitrd_       TIME=0.000000 LINE=129    FILE=tsk_ra.cdv

DISTRIBUTED ARRAY EDGE EXCHANGE

recvsh_ 12.4 Initialization of receiving imported elements of the given edge group

long recvsh_(ShadowGroupRefPtr *ShadowGroupRefPtr);

*ShadowGroupRefPtr – pointer to edge group. 
call_recvsh_      TIME=0.000000 LINE=20     FILE=sor.fdv
ShadowGroupRefPtr=4cf6b8; ShadowGroupRef=8433c0;

ret_recvsh_       TIME=0.000000 LINE=20     FILE=sor.fdv

sendsh_ 12.5 Initialization of sending imported elements of the given edge group

long sendsh_(ShadowGroupRefPtr *ShadowGroupRefPtr);

*ShadowGroupRefPtr – pointer to edge group. 
call_sendsh_      TIME=0.000000 LINE=29     FILE=sor.fdv
ShadowGroupRefPtr=4cf6b8; ShadowGroupRef=8433c0;
 
ret_sendsh_       TIME=0.000000 LINE=29     FILE=sor.fdv

REGULAR ACCESS TO REMOTE DATA

crtrbl_ 14.1 Create buffer of distributed array remote elements

long crtrbl_(long RemArrayHeader[], long BufferHeader[], void *BasePtr,
                     long *StaticSignPtr, LoopRef *LoopRefPtr, long AxisArray[],
                     long CoeffArray[], long ConstArray[]);

RemArrayHeader – header of remote distributed array.
BufferHeader – header of the buffer for remote elements.
BasePtr – base pointer for access to the buffer of remote elements.
*StaticSignPtr – flag of static buffer creation.
*LoopRefPtr – pointer to the parallel loop, where remote array elements from the buffer are required.
AxisArray – array, i-th element contains dimension number of the parallel loop (k(i+1)), corresponding to (i+1)-th dimension of the remote array.
CoeffArray – array, i-th element contains coefficient of the index variable of linear retrieval rule for (i+1)-th dimension of the remote array A(i+1).
ConstArray   array, i-th element contains constant of linear retrieval rule for (i+1)-th dimension of the remote array B(i+1). 
call_crtrbl_	TIME=0.000000 	LINE=45	FILE=tasks.fdv
RemArrayHeader=4dfd2c; RemArrayHandlePtr=9057c0; BufferHeader=4dfd48;
BasePtr=4e1200; StaticSign=1; LoopRefPtr=4dffd0; LoopRef=906b70;
 AxisArray[0]=1;  AxisArray[1]=0;  
CoeffArray[0]=1; CoeffArray[1]=0;  
ConstArray[0]=-1; ConstArray[1]=1;  

           SizeArray[0]=8;  
      LowShdWidthArray[0]=0;  
     HiShdWidthArray[0]=0;  

     Local[0]: Lower=0 Upper=7 Size=8 Step=1
      
ret_crtrbl_	TIME=0.000000 	LINE=45	FILE=tasks.fdv
BufferHandlePtr=906e70; IsLocal=1

loadrb_ 14.2 Start of loading the buffer of distributed array remote elements

long loadrb_ (long BufferHeader[], long *RenewSignPtr);

BufferHeader – header of remote element buffer.
*RenewSignPtr – flag of repeated reloading of the buffer, which has already been loaded. 
call_loadrb_	TIME=0.000000	LINE=45	FILE=tasks.fdv
BufferHeader=4dfd48; BufferHandlePtr=906e70; RenewSign=0;
     FromInitIndexArray[0]=0; FromInitIndexArray[1]=1;  
     FromLastIndexArray[0]=7; FromLastIndexArray[1]=1;  
          FromStepArray[0]=1;      FromStepArray[1]=1;  
       ToInitIndexArray[0]=0;  
       ToLastIndexArray[0]=7;  
            ToStepArray[0]=1;  

     ResInitIndexArray[0]=0; ResInitIndexArray[1]=1;  
     ResLastIndexArray[0]=7; ResLastIndexArray[1]=1;  
          ResStepArray[0]=1;      ResStepArray[1]=1;  

     ResInitIndexArray[0]=0;  
     ResLastIndexArray[0]=7;  
          ResStepArray[0]=1;  

ret_loadrb_	TIME=0.000000 	LINE=45	FILE=tasks.fdv

waitrb_ 14.3 Waiting for completion of loading buffer of distributed array remote elements

long waitrb_ (long BufferHeader[]);

BufferHeader – header of remote element buffer. 
call_waitrb_	TIME=0.000000 	LINE=45	FILE=tasks.fdv
BufferHeader=4dfd48; BufferHandlePtr=906e70;

ret_waitrb_	TIME=0.000000 	LINE=45	FILE=tasks.fdv

crtbg_ 14.6 Create group of remote element buffers

RegularAccessGroupRef crtbg_(long *StaticSignPtr, long * *DelBufSignPtr );

*StaticSignPtr – flag of static buffer group creation.
*DelBufSignPtr – flag of deleting all buffers from the group while the group deleting. 
call_crtbg_	TIME=0.000000 	LINE=43	FILE=tasks.fdv
StaticSign=0; DelBufSign=1;
ret_crtbg_	TIME=0.000000 	LINE=43	FILE=tasks.fdv
RegularAccessGroupRef=906310;

insrb_    Insert remote element buffer in the group

long insrb_(RegularAccessGroupRef *RegularAccessGroupRefPtr, long BufferHeader[]);

*RegularAccessGroupRefPtr – pointer to the buffer group.
BufferHeader – header of the buffer to be inserted. 
call_insrb_	TIME=0.000000 	LINE=45	FILE=tasks.fdv
RegularAccessGroupRefPtr=4e1210; RegularAccessGroupRef=906310; BufferHeader=4dfd48; BufferHeader[0]=906e70
ret_insrb_	TIME=0.000000 	LINE=45	FILE=tasks.fdv

loadbg_    Start of loading remote element buffers of the given group

long loadbg_(RegularAccessGroupRef long *RegularAccessGroupRefPtr, *RenewSignPtr);

*RegularAccessGroupRefPtr – pointer to the buffer group.
*RenewSignPtr – flag of repeated reloading of the buffer group, which has already been loaded. 
call_loadbg_	TIME=0.000000 	LINE=43	FILE=tasks.fdv
RegularAccessGroupRefPtr=4e1210; RegularAccessGroupRef=906310; RenewSign=1
          FromInitIndexArray[0]=0; FromInitIndexArray[1]=1;  
          FromLastIndexArray[0]=7; FromLastIndexArray[1]=1;  
               FromStepArray[0]=1;      FromStepArray[1]=1;  
            ToInitIndexArray[0]=0;  
            ToLastIndexArray[0]=7;  
                 ToStepArray[0]=1;  
           
          ResInitIndexArray[0]=0; ResInitIndexArray[1]=1;  
          ResLastIndexArray[0]=7; ResLastIndexArray[1]=1;  
               ResStepArray[0]=1;      ResStepArray[1]=1;  
           
          ResInitIndexArray[0]=0;  
          ResLastIndexArray[0]=7;  
               ResStepArray[0]=1;  
           
          FromInitIndexArray[0]=0; FromInitIndexArray[1]=3;  
          FromLastIndexArray[0]=7; FromLastIndexArray[1]=3;  
               FromStepArray[0]=1;      FromStepArray[1]=1;  
            ToInitIndexArray[0]=0;  
            ToLastIndexArray[0]=7;  
                 ToStepArray[0]=1;  
           
          ResInitIndexArray[0]=0; ResInitIndexArray[1]=3;  
          ResLastIndexArray[0]=7; ResLastIndexArray[1]=3;  
               ResStepArray[0]=1;      ResStepArray[1]=1;  
           
          ResInitIndexArray[0]=0;  
          ResLastIndexArray[0]=7;  
               ResStepArray[0]=1;  
           
ret_loadbg_	TIME=0.010000 	LINE=43	FILE=tasks.fdv

waitbg_    Waiting for the loading completion of the remote element buffers of the given group

long waitbg_ (RegularAccessGroupRef *RegularAccessGroupRefPtr);

*RegularAccessGroupRefPt – pointer to the buffer group. 
call_waitbg_	TIME=0.000000 	LINE=45	FILE=tasks.fdv
RegularAccessGroupRefPtr=4e1210; RegularAccessGroupRef=906310;
ret_waitbg_ 	TIME=0.000000 	LINE=45	FILE=tasks.fdv


References

1. V.E.Denisov, V.N.Iliakov, N.V.Kovaleva, V.A.Krukov. Debugging DVM-program efficiency. Keldysh Institute of Applied Mathematics, Russian Academy of Science. Preprint N74, 1998.