We received the following alert from our alerting system
Flash Recovery Area for FOLPRD1A has 9.97 percent of usable space left.
This is a standby database:
HOST INSTANCE STATUS VERSION STARTED UPTIME
———- ———- ———- ———— ————————- ————————————————–
xxxxxxxx INSTANCE MOUNTED 11.2.0.3.0 18-JAN-2017 18:58:18 33 days(s) 13 hour(s) 4 minute(s) 1 seconds
DB_NAME UNIQUE_NAME DB_ROLE OPEN_MODE PROTECTION_MODE PROTECTION_LEVEL
———– ———– —————— ———- ——————– ——————–
xxxxxxxx INSTANCE PHYSICAL STANDBY MOUNTED MAXIMUM AVAILABILITY MAXIMUM AVAILABILITY
Usually, when under space pressure the standby will delete archivelogs and flashback logs that it no longer needs so this alert isn’t normal for a standby. However, in this scenario, none of the space is reclaimable. Therefore, without intervention the FRA would eventually hit 100% and stop logs being transported to the standby.
NAME SPACE_LIMIT SPACE_USED SPACE_RECLAIMABLE NUMBER_OF_FILES
—————————————- ———– ———- —————– —————
+FRA 400 379.48 0 11268
FILE_TYPE USED_GB RECLAIMABLE_GB PERCENT_SPACE_USED PERCENT_SPACE_RECLAIMABLE NUMBER_OF_FILES
——————– ———- ————– —————— ————————- —————
CONTROL FILE .12 0 .03 0 1
REDO LOG 0 0 0 0 0
ARCHIVED LOG 326.12 0 81.53 0 11068
BACKUP PIECE .12 0 .03 0 1
IMAGE COPY 0 0 0 0 0
FLASHBACK LOG 53.08 0 13.27 0 197
FOREIGN ARCHIVED LOG 0 0 0 0 0
I checked the RMAN configuration suspecting it hadn’t been changed post switchover:
CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON ALL STANDBY;
Rather than being “NONE” it did look right but I did notice it was ‘ALL’. I thought we usually had it as follows:
CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY;
I assumed this wouldn’t make a difference as we only have one standby and all the logs had been applied. However, I changed it anyway. Straight away this had the desired effect and practically all the space became reclaimable.
NAME SPACE_LIMIT SPACE_USED SPACE_RECLAIMABLE NUMBER_OF_FILES
—————————————- ———– ———- —————– —————
+FRA 400 379.48 325.92 11268
FILE_TYPE USED_GB RECLAIMABLE_GB PERCENT_SPACE_USED PERCENT_SPACE_RECLAIMABLE NUMBER_OF_FILES
——————– ———- ————– —————— ————————- —————
CONTROL FILE .12 0 .03 0 1
REDO LOG 0 0 0 0 0
ARCHIVED LOG 326.12 325.88 81.53 81.47 11068
BACKUP PIECE .12 0 .03 0 1
IMAGE COPY 0 0 0 0 0
FLASHBACK LOG 53.08 0 13.27 0 197
FOREIGN ARCHIVED LOG 0 0 0 0 0
The alert subsequently cleared.
Looking at the report “Archivelog Deletion Policy – Core Production Databases” we have many databases configured with the ALL parameter. I checked another at random and it was fine. I suspected maybe it was the action of changing the parameter rather than the parameter being wrong and thought maybe dbms_backup_restore.refreshagedfiles would have done the job.
After speaking to a colleague he said this alert came out weeks ago and the trick of lowering the db_recovery_file_dest_size to force the database to be under space pressure had cleared the old logs and the alert. Therefore, the fact that this worked suggests that the space was always reclaimable, just not shown in view which the alert uses. I found a nice blog that shows the same issue and alludes to a bug.
“V$RECOVERY_AREA_USAGE is an aggregate view. If we check its definition, we see that the reclaimable size comes from x$kccagf.rectype.” It directs you to a bug (for this version) that describes that the standby “does not refresh reclaimable space automatically”.
Bug 14227959 : STANDBY DID NOT RELEASE SPACE IN FRA
The workaround is to run exec dbms_backup_restore.refreshagedfiles;
The blog also claims…..”but I’ve found another workaround: just run the CONFIGURE ARCHIVELOG POLICY from RMAN as it refreshes the reclaimable flag – even when there is no change.”
This is effectively what I did. Therefore, I’ve put the original parameter back and switched logs on the primary numerous times and the reclaimable space is being updated. For now we shall keep an eye on this as it’s not an issue anywhere else.
