How to Backup the Tomato Firmware's Bandwidth Logs (Rstats) to a FTP Server

Discussion in 'Networking Guides' started by InsaneNutter, Dec 7, 2008.

  1. InsaneNutter

    InsaneNutter Resident Nutter Staff Member

    Joined:
    Jun 1, 2007
    Messages:
    12,252
    Likes Received:
    3,724
    Location:
    Yorkshire, England
    If your router is running the Tomato firmware you will probably be aware the router can log your bandwidth usage, the bandwidth log could be useful if you have a limit each month or just want to keep track of your internet usage.

    In its default configuration the router will lose the logs when its powered down or re started, this guide will show you how to automatically backup the logs to a remote ftp server every 12 hours.

    Before you begin you will need some ftp space on a remote server, if you have some free webspace on your isp’s server this will be perfect.

    1] First log on to your router then go to Administration > Scripts or http://192.168.1.1/admin-scripts.asp

    2] In the shutdown box add:
    Code:
    if [ ! -s /tmp/restore-rstats ] ; then  {make sure any previous restore was successful, or we will overwrite the backup with nothing...}
    /tmp/backup-rstats
    fi
    
    3] In the wan up box add:

    Code:
    USER="FTP Username"
    PASS="FTP Password"
    PORT=21
    SERVER="IP or Hostname of the FTP server"
    RPATH="/rstats.tgz"
    RPATH_DATE="/rstats\`date +%Y.%m.%d-%H.%M.%S\`.tgz"
    LPATH="/tmp/rstats.tgz"
     
    if [ ! -s /tmp/backup-rstats ] ; then
        echo -e "#!/bin/sh\nkillall -1 rstats\nsleep 1\ntar -czf \"$LPATH\" /tmp/var/lib/misc/rstats-*\nftpput -u \"$USER\" -p \"$PASS\" -P $PORT $SERVER \"$RPATH\" \"$LPATH\"\nftpput -u \"$USER\" -p \"$PASS\" -P $PORT $SERVER \"$RPATH_DATE\" \"$LPATH\"\nrm \"$LPATH\"" > /tmp/backup-rstats
        chmod 777 /tmp/backup-rstats
        echo -e "#!/bin/sh\nservice rstats stop\nftpget -u \"$USER\" -p \"$PASS\" -P $PORT $SERVER \"$LPATH\" \"$RPATH\"" > /tmp/restore-rstats
        echo -e "if [ \$? != 0 ] ; then\n  logger RStats Restore Failed... will retry\n  cru a rstats \"*/5 * * * * /tmp/restore-rstats\"\n  return 1\nfi\ntar -xzf \"$LPATH\" -C /\nrm \"$LPATH\"\nservice rstats start\ncru a rstats \"1 */12 * * * /tmp/backup-rstats\"" >> /tmp/restore-rstats
        echo -e "logger RStats Data Restored\nrm /tmp/restore-rstats" >> /tmp/restore-rstats
        chmod 777 /tmp/restore-rstats
        /tmp/restore-rstats
    fi
    
    Modify your FTP username, password, port and hostname as appropriate.

    Thats it, your good to go!

    You can test this works by surfing the net for a bit or downloading something, then use the shutdown button on the routers web interface. It should upload the bandwidth logs then download them again when the router re starts. (it can take a minute or two after the router loading up for the logs to appear again)

    I take no credit for creating the script used in this guide, I found it on a random website after spending a long while looking for a solution to backup the logs to a remote server.
     
    Dark Scyth and xzKinGzxBuRnzx like this.
  2. xzKinGzxBuRnzx

    xzKinGzxBuRnzx The Feature Man

    Joined:
    Aug 9, 2008
    Messages:
    1,875
    Likes Received:
    1,245
    Location:
    Space
    I was just working on doing this the other day. Nimrod was asking why was I even bothering and to just use some program. I couldn't find anything useful on the net. Though tonight I was searching on Google very hard for a good script and I was amazed at what site I finally was returned to...DIGIEX!!! ;) I'm glad you already have a guide for this.

    Quick qustion though, is there anyway to change it to back up sooner then every 12 hours? What if your router dies, electric dies, a tornado come through ;) and you want that data to be preserved.

    I have a ftp server running on my PC at the moment InsaneNutter but do you think it would be better to back this data up to my webspace here?
     
  3. InsaneNutter

    InsaneNutter Resident Nutter Staff Member

    Joined:
    Jun 1, 2007
    Messages:
    12,252
    Likes Received:
    3,724
    Location:
    Yorkshire, England
    I think it would be better to backup to a remote server such as your Digiex webspace. A dedicated server should be online 24/7, at some point your pc is not going to be on when your router wants to backup the bandwidth logs. (especially as you like installing new Windows 7 builds)

    The advantage of doing the bandwidth monitoring with your router is that it will log the bandwidth used by every device on your network and not just a pc running an application to do so, this means the bandwidth used by the 360 and other games consoles will also be logged.

    To change when the logs are backed up I think you would find "1 */12 * * * /tmp/backup-rstats" and modify the 12 to 1 for example if you want the backup to run every hour.

    The script has been working perfectly for the last 1 and a half years now.
     

    Attached Files:

    • logs.jpg
      logs.jpg
      File size:
      83.4 KB
      Views:
      2,967
  4. xzKinGzxBuRnzx

    xzKinGzxBuRnzx The Feature Man

    Joined:
    Aug 9, 2008
    Messages:
    1,875
    Likes Received:
    1,245
    Location:
    Space
    Finally got it working fully today. Of course this picture is mostly showing all your rstats since you sent me yours. Though its backing up once every hour to my user webspace here. which is behind a webpage so no one should be able to see it inless they access it via a ftp client which to my knowledge doesn't work without a user and pass.

    Thanks for helping me today on MSN. I turned out leaving the shutdown code as it was and changing the wan up script a little to this. so it replaces the old file on the server instead on creating a new one each time.

    Code:
    USER="[B]PUT FTP USERNAME HERE[/B]"
    PASS="[B]PUT FTP PASSWORD HERE[/B]"
    PORT=[B]PUT FTP PORT HERE[/B]
    SERVER="[B]PUT FTP HOST HERE IP OR HOSTNAME[/B]"
    RPATH="/Router/rstats.tgz" WHERE IT SAYS ROUTER YOU CAN CHANGE THAT TO BE ANYTHING. LETS SAY IF IT SAYS "/ROUTER/RSTATS.TGZ" THEN IT TELLS THIS SCRIPT TO MAKE THE FILE RSTATS.TGZ IN THE REMOTE PATH /ROUTER WHICH IS A FOLDER I HAVE CREATED ON MY WEBSPACE. (DELETE THIS IF YOU INTEND TO USE IT)
    LPATH="/tmp/rstats.tgz"
     
    if [ ! -s /tmp/backup-rstats ] ; then
        echo -e "#!/bin/sh\nkillall -1 rstats\nsleep 1\ntar -czf \"$LPATH\" /tmp/var/lib/misc/rstats-*\nftpput -u \"$USER\" -p \"$PASS\" -P $PORT $SERVER \"$RPATH\" \"$LPATH\"\nftpput -u \"$USER\" -p \"$PASS\" -P $PORT $SERVER \"$LPATH\"\nrm \"$LPATH\"" > /tmp/backup-rstats
        chmod 777 /tmp/backup-rstats
        echo -e "#!/bin/sh\nservice rstats stop\nftpget -u \"$USER\" -p \"$PASS\" -P $PORT $SERVER \"$LPATH\" \"$RPATH\"" > /tmp/restore-rstats
        echo -e "if [ \$? != 0 ] ; then\n  logger RStats Restore Failed... will retry\n  cru a rstats \"*/5 * * * * /tmp/restore-rstats\"\n  return 1\nfi\ntar -xzf \"$LPATH\" -C /\nrm \"$LPATH\"\nservice rstats start\ncru a rstats \"1 */1 * * * /tmp/backup-rstats\"" >> /tmp/restore-rstats
        echo -e "logger RStats Data Restored\nrm /tmp/restore-rstats" >> /tmp/restore-rstats
        chmod 777 /tmp/restore-rstats
        /tmp/restore-rstats
    fi
    
    As you can see it updated the file on the server at 11:01!!!!! Thanks again! ;)
     

    Attached Files:

  5. InsaneNutter

    InsaneNutter Resident Nutter Staff Member

    Joined:
    Jun 1, 2007
    Messages:
    12,252
    Likes Received:
    3,724
    Location:
    Yorkshire, England
    Glad you got it working :)

    I think the idea of having it creating a separate file is so if an upload was to fail or the file got corrupted you could just restore another backup and not lose everything.
    As the files are only a few kb i left it be.

    No idea who created the script but a big thanks to them!
     
  6. mathieumg

    mathieumg New Member

    Joined:
    Aug 2, 2009
    Messages:
    3
    Likes Received:
    0
    Hi,

    I found this thread when Googling to find a way to backup my bandwidth stats on my Tomato (v1.25 running on WRT54GL v1.1). I put the appropriate scripts both in Shutdown and WAN up but when I reboot my router (even multiple times) it won't do anything. When I navigate to Daily and Weekly stats pages, I get empty tables and the Monthly page gives me a "The rstats program is not responding or is busy. Try reloading after a few seconds." error. I've made sure all the FTP info was correct and tested it with Filezilla. Also, no file gets uploaded to the ftp :( I would like to know if someone has a clue as to what the problem could be.

    Thank you very much in advance!

    Edit: Nevermind, I uploaded the stats file I had backed up from the router interface to the FTP and renamed it rstats.tgz and it seems to have done the trick! :)
     
  7. digitaldr

    digitaldr New Member

    Joined:
    Feb 8, 2011
    Messages:
    4
    Likes Received:
    0
    Hi,

    sorry to resurrect an old thread with my first post but I can't get this working on Tomato 1.28. First off backups seemed to be saved as .gz files with a file name tomato_rstats_001217d1af49.gz

    First I followed InsaneNutters instructions exactly as above i.e using rstats.tgz. When this didn't work I tried using rstat.gz and finally tomato_rstats_001217d1af49.gz all to no avail. Each time I uploaded the appropriate file to my FTP space. I got the same error every time "The rstats program is not responding or is busy. Try reloading after a few seconds." and nothing was written to my ftp folder when I checked with Filezilla.

    I'm a totally newbie when it comes to this stuff so was wondering if I was correct to leave the quotation marks around my username, password and server.

    Any help greatly appreciated although I'm not to hopeful of getting a reply as thread is so old! In the meantime I'm going to save weekly to vRam - Does the "Total / Free Memory30.02 MB / 19.55 MB (65.14%)" refer to vRam?

    Thanks
     
  8. mathieumg

    mathieumg New Member

    Joined:
    Aug 2, 2009
    Messages:
    3
    Likes Received:
    0
    Did you try to upload the stats file backed up from the router interface to the FTP and rename it rstats.tgz?

    That's what made it work for me back when I set it up (see my previous post) and it still works fine today.
     
  9. digitaldr

    digitaldr New Member

    Joined:
    Feb 8, 2011
    Messages:
    4
    Likes Received:
    0
    I tried that but no luck. I'm beginning to think that it may not work on Tomato 1.28. Are you still on 1.25?
     
  10. mathieumg

    mathieumg New Member

    Joined:
    Aug 2, 2009
    Messages:
    3
    Likes Received:
    0
    Yeah, same exact setup on 1.25.
     
  11. digitaldr

    digitaldr New Member

    Joined:
    Feb 8, 2011
    Messages:
    4
    Likes Received:
    0
    Thanks for that Might try downgrading to 1.25 tomorrow - checked the changelog and I don't think that I'll be missing much if I do.
     
  12. digitaldr

    digitaldr New Member

    Joined:
    Feb 8, 2011
    Messages:
    4
    Likes Received:
    0
    Finally got this working:) I downgraded to 1.25 but don't think it helped in the end. I think the problem was that Filezilla doesn't automatically refresh directory listings on reconnect so I thought I still has a rstats.tgz file on my ftp space when I had actually deleted it!

    Just one more query - being a newbie I'm not too clear on the difference between tgz and gz files. It seems that the script saves tgz files to ftp. These cannot be restored in the tomato bandwidth configuration page. I guess this isn't too important as the script restores them on reboot?
     
  13. temp1

    temp1 New Member

    Joined:
    Feb 3, 2012
    Messages:
    3
    Likes Received:
    0
    late to the game..need help

    I've read this post a couple of times and I copied the scripts but I can't get it to work. I using ftp.4shared.com as my ftp server and I can upload with Filezilla, but the scripts don't work for me.

    for the username and password do I leave the " " in the code? I'm using tomato firmware 1.28? Any help would be appreciated.;)
     
  14. InsaneNutter

    InsaneNutter Resident Nutter Staff Member

    Joined:
    Jun 1, 2007
    Messages:
    12,252
    Likes Received:
    3,724
    Location:
    Yorkshire, England
    Yup leave the " so the user for example might be: USER="admin"

    I dont think this works in later tomato versions, i could never get it to work in 1.28 sadly.

    In later versions can also save the logs to NVRAM, I have enabled this and told it to save them once a week, as my router is never usually switched off this has been working ok for me

    [​IMG]

    Ill have to see if i can get it to save them via ftp again sometime.
     
  15. temp1

    temp1 New Member

    Joined:
    Feb 3, 2012
    Messages:
    3
    Likes Received:
    0
    Thanks for quick response. I will see if the file uploads later tonight before I give up. Can I ask what do you with the backed up file. How to view its contents with the bandwidth usage? Do you have to restore the file in tomato to view the usage? Does this action remove the current stats from the router?
     
  16. InsaneNutter

    InsaneNutter Resident Nutter Staff Member

    Joined:
    Jun 1, 2007
    Messages:
    12,252
    Likes Received:
    3,724
    Location:
    Yorkshire, England
    I only only ever restored it to tomato, it has never really bothered me enough to try view it in anything else. The routers web interface tells me everything i could want to know.
     
    temp1 likes this.
  17. temp1

    temp1 New Member

    Joined:
    Feb 3, 2012
    Messages:
    3
    Likes Received:
    0
    I got it to work. I followed what mathieumg did by uploading a file to the FTP site and now I get a file backed up every 12 hours. Thanks for the help :)
     
  18. Hakim Sheriff

    Hakim Sheriff New Member

    Joined:
    Mar 24, 2012
    Messages:
    1
    Likes Received:
    0
    Did it work for version 1.28? It doesn't seem to work for me.
     

Share This Page