The results and ramblings of research

phewww!

Deleting Files older than x days

leave a comment »

So, I have written a script to download tons of streaming social media data, specifically JSON files each about 70KB every 5 minutes. Unfortunately, social media data is freaking huge and disk quotas on university machines aren’t too large. As usual within 15-20 days, I end up with a disk quota exceeded error. However, since my processing is about 4 days behind, I cant just delete the folder. Fortunately, unix is a beast. ‘find’ is an amazing command. I can use ‘find’ to delete from the folder all JSON files which are say ‘7’ days in the past. As an example,

$ find ~/datafolder/* -mtime +7 -exec rm {} \;

will delete all files that are 7 days in the past. More interesting I can run this in a weekly cron job. Problem solved :).

Advertisements

Written by anujjaiswal

October 11, 2011 at 8:34 am

Posted in Linux

Tagged with , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

%d bloggers like this: