Implementing Fifo (First In First Out) On Media Folder To Maintain Diskspace - Bytesized Hosting Wiki

DISCLAIMER: USE AT YOUR OWN RISK!

So I have some very media hungry family members constantly adding stuff to my media folder, so instead of constantly deleting old stuff I wanted to find a script that checks if diskspace is running low, and if so, delete the oldest files added to the folder until diskspace has been recovered to a specific percentage.

Found a nice script that does just that, and with a few tweaks by yours truly and the infamous Animazing (thanks!) its working really well!

Instructions: Copy and paste the below code into a new file called something like deleteOldFiles.sh. Then run it with the following syntax:

./deleteOldFiles.sh [PATH] [limit in percentage]

For example if you want it to run on your media folder which is located at ~/media and you want it to delete files until your diskspace is no more than 90% full, run the following:

./deleteOldFiles.sh ~/media 90

The script basically checks if your diskspace is higher than the threshold you specified, and if so recursively finds the oldest file in the directory then deletes it. It repeats this process until the diskspace is within the threshold.

WARNING: When implementing this, I would strongly advise you modify the 'MAX_CYCLES' variable to a small number - this is a built in failsafe to prevent any possible runaway scripts that may delete all your files!

WARNING2: I would advise test running this on a directory that you don't really care about - for example the Sabnzbd incomplete folder (if your queue is empty of course), before you implement it to your media folder.

If all works fine, then you can put this into a CRON job and have it run every day for example.

CODE:

#!/bin/bash
#
############################################################################### 
# Author            :  Louwrentius
# Contact           : [email protected]
# Initial release   : August 2011
# Licence           : Simplified BSD License
############################################################################### 

VERSION=1.01

#
# Mounted volume to be monitored.
#
MOUNT="$1"
#
# Maximum threshold of volume used as an integer that represents a percentage:
# 95 = 95%.
#
MAX_USAGE="$2"
#
# Failsafe mechansim. Delete a maxium of MAX_CYCLES files, raise an error after
# that. Prevents possible runaway script. Disable by choosing a high value.
#
MAX_CYCLES=1000
CYCLES=0


show_header () {

    echo
    echo DELETE OLD FILES $VERSION
    echo

}

show_header

reset () {
    OLDEST_FILE=""
    OLDEST_DATE=0
    ARCH=`uname`
}

reset

if [ -z "$MOUNT" ] || [ ! -e "$MOUNT" ] || [ ! -d "$MOUNT" ] || [ -z "$MAX_USAGE" ]
then
    echo "Usage: $0 <mountpoint> <threshold>"
    echo "Where threshold is a percentage."
    echo
    echo "Example: $0 /storage 90"
    echo "If disk usage of /storage exceeds 90% the oldest"
    echo "file(s) will be deleted until usage is below 90%."
    echo 
    echo "Wrong command line arguments or another error:"
    echo 
    echo "- Directory not provided as argument or"
    echo "- Directory does not exist or"
    echo "- Argument is not a directory or"
    echo "- no/wrong percentage supplied as argument."
    echo
    exit 1
fi

check_capacity () {
    TOTAL=`quota | tail -n1 | awk ' { print $2 }' | sed  -e 's/G//'`
    CURRENT=`quota | tail -n1 | awk ' { print $1 }' | sed  -e 's/G//'`
    let USAGE=(100*CURRENT)/TOTAL
    echo "Ani says usage is $USAGE"

    if [ ! "$?" == "0" ]    
    then
        echo "Error: mountpoint $MOUNT not found in df output."
        exit 1
    fi

    if [ -z "$USAGE" ]
    then
        echo "Didn't get usage information of $MOUNT"
        echo "Mountpoint does not exist or please remove trailing slash."
        exit 1
    fi

    if [ "$USAGE" -gt "$MAX_USAGE" ]
    then
        echo "Usage of $USAGE% exceeded limit of $MAX_USAGE percent."
        return 0
    else
        echo "Usage of $USAGE% is within limit of $MAX_USAGE percent."
        return 1
    fi
}

check_age () {

    FILE="$1"
    if [ "$ARCH" == "Linux" ]
    then
        FILE_DATE=`stat -c %Z "$FILE"`
    elif [ "$ARCH" == "Darwin" ]
    then
        FILE_DATE=`stat -f %Sm -t %s "$FILE"`
    else
        echo "Error: unsupported architecture."
        echo "Send a patch for the correct stat arguments for your architecture."
    fi

    NOW=`date +%s`
    AGE=$((NOW-FILE_DATE))
    if [ "$AGE" -gt "$OLDEST_DATE" ]
    then
        export OLDEST_DATE="$AGE"
        export OLDEST_FILE="$FILE"
    fi
}

process_file () {

    FILE="$1"

    #
    # Replace the following commands with wathever you want to do with 
    # this file. You can delete files but also move files or do something else.
    #
    echo "Deleting oldest file $FILE"
    rm -f "$FILE"
}

while check_capacity
do
    echo $CYCLES
    if [ "$CYCLES" -gt "$MAX_CYCLES" ]
    then
        echo "Error: after $MAX_CYCLES deleted files still not enough free space."
        exit 1
    fi

    reset

    FILES=`find "$MOUNT" -type f`

    IFS=$'\n'
    for x in $FILES
    do
        check_age "$x"
    done

    if [ -e "$OLDEST_FILE" ]
    then
        #
        # Do something with file.
        #
        process_file "$OLDEST_FILE"
    else
        echo "Error: somehow, item $OLDEST_FILE disappeared."
    fi
    echo "Increasing Counter"
    CYCLES=$[CYCLES + 1]
done
echo```


Enjoy :)