Subsections


8. Memory issues


8.1 The 32-bit model.

The Free Pascal compiler issues 32-bit code. This has several consequences:

The fact that 32-bit code is used, means that some of the older Turbo Pascal constructs and functions are obsolete. The following is a list of functions which shouldn't be used anymore:

Seg()
: Returned the segment of a memory address. Since segments have no more meaning, zero is returned in the Free Pascal run-time library implementation of Seg.
Ofs()
: Returned the offset of a memory address. Since segments have no more meaning, the complete address is returned in the Free Pascal implementation of this function. This has as a consequence that the return type is Longint instead of Word.
Cseg(), Dseg()
: Returned, respectively, the code and data segments of your program. This returns zero in the Free Pascal implementation of the system unit, since both code and data are in the same memory space.
Ptr:
Accepted a segment and offset from an address, and would return a pointer to this address. This has been changed in the run-time library, it now simply returns the offset.
memw and mem
These arrays gave access to the DOS memory. Free Pascal supports them on the go32v2 platform, they are mapped into DOS memory space. You need the GO32 unit for this. On other platforms, they are not supported

You shouldn't use these functions, since they are very non-portable, they're specific to DOS and the ix86 processor. The Free Pascal compiler is designed to be portable to other platforms, so you should keep your code as portable as possible, and not system specific. That is, unless you're writing some driver units, of course.


8.2 The stack

The stack is used to pass parameters to procedures or functions, to store local variables, and, in some cases, to return function results.

When a function or procedure is called, then the following is done by the compiler :

  1. If there are any parameters to be passed to the procedure, they are pushed from right to left on the stack.
  2. If a function is called that returns a variable of type String, Set, Record, Object or Array, then an address to store the function result in, is pushed on the stack.
  3. If the called procedure or function is an object method, then the pointer to self is pushed on the stack.
  4. If the procedure or function is nested in another function or procedure, then the frame pointer of the parent procedure is pushed on the stack.
  5. The return address is pushed on the stack (This is done automatically by the instruction which calls the subroutine).

The resulting stack frame upon entering looks as in table (StackFrame) .

Table: Stack frame when calling a procedure
Offset What is stored Optional ?
+x parameters Yes
+12 function result Yes
+8 self Yes
+4 Return address No
+0 Frame pointer of parent procedure Yes

8.2.1 Intel x86 version

The stack is cleared with the ret I386 instruction, meaning that the size of all pushed parameters is limited to 64K.

8.2.1.1 DOS

Under the DOS targets, the default stack is set to 256Kb. This value cannot be modified for the GO32V1 target. But this can be modified with the GO32V2 target using a special DJGPP utility stubedit. It is to note that the stack size may be changed with some compiler switches, this stack size, if greater then the default stack size will be used instead, otherwise the default stack size is used.

8.2.1.2 Linux

Under LINUX, stack size is only limited by the available memory of the system.

8.2.1.3 Windows

Under WINDOWS, stack size is only limited by the available memory of the system.

8.2.1.4 OS/2

Under OS/2, stack size is determined by one of the runtime environment variables set for EMX. Therefore, the stack size is user defined.

8.2.2 Motorola 680x0 version

All depending on the processor target, the stack can be cleared in two manners, if the target processor is a MC68020 or higher, the stack will be cleared with a simple rtd instruction, meaning that the size of all pushed parameters is limited to 32K.

Otherwise on MC68000/68010 processors, the stack clearing mechanism is sligthly more complicated, the exit code will look like this:

{
  move.l  (sp)+,a0
  add.l   paramsize,a0
  move.l  a0,-(sp)
  rts
}

8.2.2.1 Amiga

Under AmigaOS, stack size is determined by the user, which sets this value using the stack program. Typical sizes range from 4K to 40K.

8.2.2.2 Atari

Under Atari TOS, stack size is currently limited to 8K, and it cannot be modified. This may change in a future release of the compiler.


8.3 The heap

The heap is used to store all dynamic variables, and to store class instances. The interface to the heap is the same as in Turbo Pascal, although the effects are maybe not the same. On top of that, the Free Pascal run-time library has some extra possibilities, not available in Turbo Pascal. These extra possibilities are explained in the next subsections.

8.3.1 The heap grows

Free Pascal supports the HeapError procedural variable. If this variable is non-nil, then it is called in case you try to allocate memory, and the heap is full. By default, HeapError points to the GrowHeap function, which tries to increase the heap.

The growheap function issues a system call to try to increase the size of the memory available to your program. It first tries to increase memory in a 1 Mb. chunk. If this fails, it tries to increase the heap by the amount you requested from the heap.

If the call to GrowHeap has failed, then a run-time error is generated, or nil is returned, depending on the GrowHeap result.

If the call to GrowHeap was successful, then the needed memory will be allocated.

8.3.2 Using Blocks

If you need to allocate a lot of small blocks for a small period, then you may want to recompile the run-time library with the USEBLOCKS symbol defined. If it is recompiled, then the heap management is done in a different way.

The run-time library keeps a linked list of allocated blocks with size up to 256 bytes8.1. By default, it keeps 32 of these lists8.2.

When a piece of memory in a block is deallocated, the heap manager doesn't really deallocate the occupied memory. The block is simply put in the linked list corresponding to its size.

When you then again request a block of memory, the manager checks in the list if there is a non-allocated block which fits the size you need (rounded to 8 bytes). If so, the block is used to allocate the memory you requested.

This method of allocating works faster if the heap is very fragmented, and you allocate a lot of small memory chunks.

Since it is invisible to the program, this provides an easy way of improving the performance of the heap manager.

8.3.3 Using the split heap

Remark: The split heap is still somewhat buggy. Use at your own risk for the moment.

The split heap can be used to quickly release a lot of blocks you allocated previously.

Suppose that in a part of your program, you allocate a lot of memory chunks on the heap. Suppose that you know that you'll release all this memory when this particular part of your program is finished.

In Turbo Pascal, you could foresee this, and mark the position of the heap (using the Mark function) when entering this particular part of your program, and release the occupied memory in one call with the Release call.

For most purposes, this works very good. But sometimes, you may need to allocate something on the heap that you don't want deallocated when you release the allocated memory. That is where the split heap comes in.

When you split the heap, the heap manager keeps 2 heaps: the base heap (the normal heap), and the temporary heap. After the call to split the heap, memory is allocated from the temporary heap. When you're finished using all this memory, you unsplit the heap. This clears all the memory on the split heap with one call. After that, memory will be allocated from the base heap again.

So far, nothing special, nothing that can't be done with calls to mark and release. Suppose now that you have split the heap, and that you've come to a point where you need to allocate memory that is to stay allocated after you unsplit the heap again. At this point, mark and release are of no use. But when using the split heap, you can tell the heap manager to -temporarily- use the base heap again to allocate memory. When you've allocated the needed memory, you can tell the heap manager that it should start using the temporary heap again. When you're finished using the temporary heap, you release it, and the memory you allocated on the base heap will still be allocated.

To use the split-heap, you must recompile the run-time library with the TempHeap symbol defined. This means that the following functions are available :

  procedure Split_Heap;
  procedure Switch_To_Base_Heap;
  procedure Switch_To_Temp_Heap;
  procedure Switch_Heap;
  procedure ReleaseTempHeap;
  procedure GetTempMem(var p : pointer;size : longint);
Split_Heap is used to split the heap. It cannot be called two times in a row, without a call to releasetempheap. Releasetempheap completely releases the memory used by the temporary heap. Switching temporarily back to the base heap can be done using the Switch_To_Base_Heap call, and returning to the temporary heap is done using the Switch_To_Temp_Heap call. Switching from one to the other without knowing on which one your are right now, can be done using the Switch_Heap call, which will split the heap first if needed.

A call to GetTempMem will allocate a memory block on the temporary heap, whatever the current heap is. The current heap after this call will be the temporary heap.

Typically, what will appear in your code is the following sequence :

Split_Heap
...
{ Memory allocation }
...
{ !! non-volatile memory needed !!}
Switch_To_Base_Heap;
getmem (P,size);
Switch_To_Temp_Heap;
...
{Memory allocation}
...
ReleaseTempHeap;
{All allocated memory is now freed, except for the memory pointed to by 'P' }
...

8.3.4 Debugging the heap

Free Pascal provides a unit that allows you to trace allocation and deallocation of heap memory: heaptrc.

If you specify the -gh switch on the command-line, or if you include heaptrc as the first unit in your uses clause, the memory manager will trace what is allocated and deallocated, and on exit of your program, a summary will be sent to standard output.

More information on using the heaptrc mechanism can be found in the Users' guide and Unit reference.

8.3.5 Writing your own memory manager

Free Pascal allows you to write and use your own memory manager. The standard functions GetMem, FreeMem, ReallocMem and Maxavail use a special record in the system unit to do the actual memory management. The system unit initializes this record with the system unit's own memory manager, but you can read and set this record using the GetMemoryManager and SetMemoryManager calls:

procedure GetMemoryManager(var MemMgr: TMemoryManager);
procedure SetMemoryManager(const MemMgr: TMemoryManager);

the TMemoryManager record is defined as follows:

  TMemoryManager = record
    Getmem      : Function(Size:Longint):Pointer;
    Freemem     : Function(var p:pointer):Longint;
    FreememSize : Function(var p:pointer;Size:Longint):Longint;
    AllocMem    : Function(Size:longint):Pointer;
    ReAllocMem  : Function(var p:pointer;Size:longint):Pointer;
    MemSize     : function(p:pointer):Longint;
    MemAvail    : Function:Longint;
    MaxAvail    : Function:Longint;
    HeapSize    : Function:Longint;
  end;

As you can see, the elements of this record are procedural variables. The system unit does nothing but call these various variables when you allocate or deallocate memory.

Each of these functions corresponds to the corresponding call in the system unit. We'll describe each one of them:

Getmem
This function allocates a new block on the heap. The block should be Size bytes long. The return value is a pointer to the newly allocated block.
Freemem
should release a previously allocated block. The pointer P points to a previously allocated block. The Memory manager should implement a mechanism to determine what the size of the memory block is 8.3 The return value is optional, and can be used to return the size of the freed memory.
FreememSize
This function should release the memory pointed to by P. The argument Size is the expected size of the memory block pointed to by P. This should be disregarded, but can be used to check the behaviour of the program.
AllocMem
Is the same as getmem, only the allocated memory should be filled with zeroes before the call returns.
ReAllocMem
Should allocate a memory block Size bytes large, and should fill it with the contents of the memory block pointed to by P, truncating this to the new size of needed. After that, the memory pointed to by P may be deallocated. The return value is a pointer to the new memory block.
MemSize
should return the total amount of memory available for allocation. This function may return zero if the memory manager does not allow to determine this information.
MaxAvail
should return the size of the largest block of memory that is still available for allocation. This function may return zero if the memory manager does not allow to determine this information.
HeapSize
should return the total size of the heap. This may be zero is the memory manager does not allow to determine this information.
To implement your own memory manager, it is sufficient to construct such a record and to issue a call to SetMemoryManager.

To avoid conflicts with the system memory manager, setting the memory manager should happen as soon as possible in the initialization of your program, i.e. before any call to getmem is processed.

This means in practice that the unit implementing the memory manager should be the first in the uses clause of your program or library, since it will then be initialized before all other units (except of the system unit)

This also means that it is not possible to use the heaptrc unit in combination with a custom memory manager, since the heaptrc unit uses the system memory manager to do all it's allocation. Putting the heaptrc unit after the unit implementing the memory manager would overwrite the memory manager record installed by the custom memory manager, and vice versa.

The following unit shows a straightforward implementation of a custom memory manager using the memory manager of the C library. It is distributed as a package with Free Pascal.

unit cmem;

{$mode objfpc}

interface

Function Malloc (Size : Longint) : Pointer;cdecl;
  external 'c' name 'malloc';
Procedure Free (P : pointer); cdecl; external 'c' name 'free';
Procedure FreeMem (P : Pointer); cdecl; external 'c' name 'free';
function ReAlloc (P : Pointer; Size : longint) : pointer; cdecl;
  external 'c' name 'realloc';
Function CAlloc (unitSize,UnitCount : Longint) : pointer;cdecl;
  external 'c' name 'calloc';

implementation

Function CGetMem  (Size : Longint) : Pointer;

begin
  result:=Malloc(Size);
end;

Function CFreeMem (Var P : pointer) : Longint;

begin
  Free(P);
  Result:=0;
end;

Function CFreeMemSize(var p:pointer;Size:Longint):Longint;

begin
  Result:=CFreeMem(P);
end;

Function CAllocMem(Size : Longint) : Pointer;

begin
  Result:=calloc(Size,1);
end;

Function CReAllocMem (var p:pointer;Size:longint):Pointer;

begin
  Result:=realloc(p,size);
end;

Function CMemSize (p:pointer): Longint;

begin
  Result:=0;
end;

Function CMemAvail : Longint;

begin
  Result:=0;
end;

Function CMaxAvail: Longint;

begin
  Result:=0;
end;

Function CHeapSize : Longint;

begin
  Result:=0;
end;


Const
 CMemoryManager : TMemoryManager =
    (
      GetMem : CGetmem;
      FreeMem : CFreeMem;
      FreememSize : CFreememSize;
      AllocMem : CAllocMem;
      ReallocMem : CReAllocMem;
      MemSize : CMemSize;
      MemAvail : CMemAvail;
      MaxAvail : MaxAvail;
      HeapSize : CHeapSize;
    );

Var
  OldMemoryManager : TMemoryManager;

Initialization
  GetMemoryManager (OldMemoryManager);
  SetMemoryManager (CmemoryManager);

Finalization
  SetMemoryManager (OldMemoryManager);
end.


8.4 Using DOS memory under the Go32 extender

Because Free Pascal is a 32 bit compiler, and uses a DOS extender, accessing DOS memory isn't trivial. What follows is an attempt to an explanation of how to access and use DOS or real mode memory8.4.

In Proteced Mode, memory is accessed through Selectors and Offsets. You can think of Selectors as the protected mode equivalents of segments.

In Free Pascal, a pointer is an offset into the DS selector, which points to the Data of your program.

To access the (real mode) DOS memory, somehow you need a selector that points to the DOS memory. The GO32 unit provides you with such a selector: The DosMemSelector variable, as it is conveniently called.

You can also allocate memory in DOS's memory space, using the global_dos_alloc function of the GO32 unit. This function will allocate memory in a place where DOS sees it.

As an example, here is a function that returns memory in real mode DOS and returns a selector:offset pair for it.

procedure dosalloc(var selector : word;
                   var segment : word;
                   size : longint);

var result : longint;

begin
     result := global_dos_alloc(size);
     selector := word(result);
     segment := word(result shr 16);
end;
(You need to free this memory using the global_dos_free function.)

You can access any place in memory using a selector. You can get a selector using the allocate_ldt_descriptor function, and then let this selector point to the physical memory you want using the set_segment_base_address function, and set its length using set_segment_limit function. You can manipulate the memory pointed to by the selector using the functions of the GO32 unit. For instance with the seg_fillchar function. After using the selector, you must free it again using the free_ldt_selector function.

More information on all this can be found in the Unit reference, the chapter on the GO32 unit.



root
2000-12-20