PostgreSQL: How to Fix "invalid memory alloc request size 4294967293" During pg_dump
When running pg_dump in PostgreSQL, you may encounter the following error:
pg_dump: dumpClasses(): SQL command failed
pg_dump: Error message from server: ERROR: invalid memory alloc request size 4294967293
pg_dump: The command was: FETCH 100 FROM _pg_dump_cursor
Initial suspicion: memory settings, but no effect
At first glance, the message "invalid memory alloc request size 4294967293" made me suspect that something was wrong with PostgreSQL's shared memory settings or that the server was running out of memory. I tried adjusting various memory-related configuration parameters, but none of these changes had any effect.
Searching for the error message online led me to the following post:
BUG #2534: ERROR: invalid memory alloc request size 4294967293
http://archives.postgresql.org/pgsql-bugs/2006-07/msg00072.php
<snip>
While executing pg_dump command, I am getting the following errors.
pg_dump: ERROR: invalid memory alloc request size 4294967293
pg_dump: SQL command to dump the contents of table "ctblankcall" failed:
PQendcopy() failed.
<snip>
This was exactly the same error I was facing, but unfortunately the thread had no replies, so it didn't help identify the root cause.
Although I never discovered the fundamental reason behind the error, I needed to complete the pg_dump, so I proceeded with the following workaround.
Step 1: Identify which table fails during pg_dump
Dump each table individually. This allows you to pinpoint the specific table that causes pg_dump to fail.
Step 2: Identify which record fails during SELECT
Once the problematic table is found, run:
SELECT * FROM table_name WHERE conditions;
This helps locate the specific record that triggers the failure.
Step 3: Identify which column contains invalid data
Next, check each column individually:
SELECT column1 FROM table_name WHERE conditions;
If this succeeds, column1 is fine.
Then try:
SELECT column2 FROM table_name WHERE conditions;
If this fails, column2 contains invalid data. Repeat this process until all problematic columns are identified.
Step 4: Update the invalid column values
Once the problematic columns are identified, update them with appropriate values. After correcting all invalid data, pg_dump will complete successfully.
Related article
This procedure is similar to what I described in an earlier article:
Additional thoughts on the possible cause
In that earlier case, the issue was caused by a single invalid value in one column of one table, likely due to a bug in the PHP program that inserted the data. After fixing the PHP code, the problem did not recur.
In the current case, multiple tables contained invalid column values. While a program bug cannot be completely ruled out, it seemed unlikely.
Another possible cause emerged when I noticed disk-related errors in the Linux syslog. It is possible that part of the PostgreSQL data files had become corrupted due to a failing hard disk. This is only speculation, but once I saw disk errors, it became difficult to ignore the possibility.
If the same error occurs again and there are no disk errors, I will investigate further at that time.