Errors Showing up with High Memory Use Programs
The following stems from a posting originally on the REALbasic Users Mailing List and I thought it was worth copying and publishing when I found myself writing the same answer to someone else recently.
RB2007 exercises your memory and virtual memory more than most programs used by casual users, other than video or image manipulation.
It is possible therefore that RB is hitting memory errors that your normal use of that machine doesn't - I suggest you check your memory, with the Apple Hardware Test (from install DVD) or something third-party like TechTool Pro.
Massive Swapfile Allocations
Check in /var/vm as to the number of swapfiles being created. If you have very little free space then maybe the OS is having trouble creating them as RB requires.
Note that if you look at VM in a tool like the Activity Monitor (applications/utilities) then you will see a very large figure for VM, eg: nearly 8GB on my machine at present. For a lot of programs, the bulk of their VM size is code mapping and so doesn't require an allocation on disk - it maps read-only segments directly from the code on disk. However, when RB runs up to huge VM allocations that seems to be writable memory - running RB is the only time I have ended up with 5 swapfiles allocated - a total of 1.25GB actual swap space in addition to the 1.25GB in RAM.
Swapfile Fragmentation on OS/X
I have read in my exploration of VM on OS/X that the OS refuses to allocate fragmented swapfiles, that is not actually the case and I just proved it.
For some reason, RB, deemed my plugins needed recompiling so I left the Mac going for the several hours it took to do so for PEF, Mach-O and Windows. I have MBS and Einhugur suites installed.
On my 1.25GB RAM PPC, my /var/vm folder went up to 8 swapfiles, a total of 2.5GB allocated, increasing mostly in a few minutes from the 128MB allocated prior to RB starting the plugin prep.
andeo-mac:/var/vm andydent$ ls -al
total 5242880
drwxr-xr-x 11 root wheel
374 Jan 27 17:20
.
drwxr-xr-x 24 root wheel
816 Jan 25 19:24
..
drwx--x--x 14 root wheel
476 Jan 25 17:22
app_profile
-rw------T 1 root wheel
67108864 Jan 25 17:21 swapfile0
-rw------T 1 root wheel
67108864 Jan 26 12:13 swapfile1
-rw------T 1 root wheel
134217728 Jan 27 16:12 swapfile2
-rw------T 1 root wheel
268435456 Jan 27 16:12 swapfile3
-rw------T 1 root wheel
536870912 Jan 27 16:13 swapfile4
-rw------T 1 root wheel
536870912 Jan 27 16:15 swapfile5
-rw------T 1 root wheel
536870912 Jan 27 16:33 swapfile6
-rw------T 1 root wheel
536870912 Jan 27 17:10 swapfile7
Using hfsdebug (*1), I found that the last four of these swapfiles were in two extents, rather than being contiguous. Previously, when I was very low on disk space and the free space was extremely fragmented, I had seen iDefrag report a massive number of extents for the swapfiles and my machine became utterly bogged down.
Something which is quite fascinating and also contrary to common web reports - the last three of these swapfiles later vanished after I had quit RB. Everything I've read says that OS/X won't reclaim swapfiles until you reboot. In this case it appears the only thing with a claim on those files was RB and so OS/X did in fact remove them when they weren't needed.
During the plugin preparation time, and the relatively small amount of time building the app, I continued trying to use the computer for some mild browsing activity. I was running the Activity Monitor at the time on another window and could see that the machine was flat out doing IO and with a massive amount of paging activity but certainly not CPU-bound.
I know from VMS experience (*2) that page thrashing can cause a normal user-priority process to bring a multi-tasking OS to its knees and it appears that OS/X is certainly not immune. This behaviour was like the worst I've experienced with Visual Studio compiling whilst trying to use Windows (I have no RB200x comparison possible on Windows as haven't bothered buying a Windows license for it).
These results actually give me some hope that RS can do something to improve the problem - apart from the possibility that there is a significant memory leak whilst compiling there is also the hope they can improve the locality of references made by the compiler (written in c++ so they have more control over such niceties).
hfsdebug
(*1)hfsdebug comes from http://www.osxbook.com/software/hfsdebug/
It allows you to quickly check the fragmentation of a single file, rather than using something like iDefrag which scans the entire disk.
(it also does many other reports).
eg:
andeo-mac:/var/vm andydent$ sudo hfsdebug swapfile4
path
=
MacintoshHD:/private/var/vm/swapfile4
# Catalog File Record
type
=
file
file ID
=
5153585
flags
=
0000000000000010
.
File has a thread record in the catalog.
reserved1
=
0
createDate
=
Sat Jan 27 16:13:20 2007
contentModDate =
Sat Jan 27 16:13:21 2007
attributeModDate = Sat Jan 27
16:13:21 2007
accessDate
=
Sat Jan 27 16:13:20 2007
backupDate
=
Fri Jan 1 00:00:00 1904
# BSD Info
ownerID
=
0 (root)
groupID
=
0 (wheel)
adminFlags
=
00000000
ownerFlags
=
00000000
fileMode
=
-rw------t
linkCount
=
0
textEncoding
= 0
attrBlocks
=
0
# Finder Info
fdType
=
0
fdCreator
=
0
fdFlags
=
0000000000000000
fdLocation
=
(v = 0, h = 0)
opaque
=
0
# Data Fork
logicalSize
=
536870912 bytes
totalBlocks
=
131072
fork temperature = no HFC
record in B-Tree
clumpSize
=
0
extents
=
startBlock blockCount
% of file
0x262e24
0x18662
76.25 %
0x463c2e
0x799e
23.75 %
131072
allocation blocks in 2 extents total.
65536.00
allocation blocks per extent on an average.
# Resource Fork
logicalSize
= 0
bytes
My First Experience with VM Thrashing
(*2)When I was a VMS sysadmin, about 20 years ago, we had a scientific programmer write some COBOL code. She didn't realise that FORTRAN accesses its arrays in the opposite order to COBOL and so her manipulation in arrays of about 4MB of data caused massive amounts of thrashing of pages as her code was having to page in memory every few instructions due to extremely poor locality of reference.
It brought an 80-user machine to its knees for about 2 hours as the code tried to do a series of filters and sorts in-memory. Nothing I could do with permissions or job priority control would stop this - I had to rewrite the program. The replacement as a pipeline using a series of simple filter programs and use of the Sort utility program, all on sequential files on disk, ran in a few seconds.