198
edits
Changes
→Introduction
The script is to be run on the computer: bahamas
The script is to be run with the user: backup
=== Dependencies ===
The following software should be installed before using:
<pre>
yum install gcc
yum install autoconf
yum install python-pip
pip install pycrypto
pip install pysftp
pip install ecdsa
</pre>
== Goals ==
- Script resides on a single computer (complete)
- Do not run multiple backups using the same hard drive (complete)
- Check space requirements before performing a backup on source and destination (in progresscancelled)- Emails out daily reports on success or fail (not completework around with mail command)
- Logs all information /var/log/smart-bk/ (complete)
- Easy(ish) to add a new backup schedule (complete)
- Can view all the backups in the queue to run (complete)
- Can view all the schedules that are added (complete)
- Records a record of all previously run backups (not complete)- Website to view status of currently running backups (not complete)
</pre>
At this time a website can now view the information from the database, and the database can now keep a record of all previously run backups. Email was a issue, so, the output of a report is piped to a mail command running in the crontab. Checking space requirements was a silly goal for now, since trying to do this on the source and destination would increase the backup time by way too much(recursively looking through directories, which contain 100G+ data). <s>At this time, not all of these goals have been completed, but I would like them to be sooner or later. Right now I'm setting up a little documentation on how it currently works, what it's missing, and what my next steps will be.</s>
== Scheduler System ==
Run the following sql statements to create the proper tables within the database
<pre>
sqlite> > .schema CREATE TABLE Logs(scheduleid INTEGER, status TEXT, errors TEXT, start_date TEXT, start_time TEXT, end_date TEXT, end_time TEXT);
CREATE TABLE Queue(scheduleid INTEGER, queuetime TEXT, FOREIGN KEY(scheduleid) REFERENCES Schedule(id));
CREATE TABLE Running(scheduleid INTEGER, starttime TEXT, FOREIGN KEY(scheduleid) REFERENCES Schedule(id));
CREATE TABLE Schedule(id INTEGER PRIMARY KEY, day TEXT, time TEXT, type TEXT, source_host TEXT, dest_host TEXT, source_dir TEXT, dest_dir TEXT, source_user TEXT, dest_user TEXT, desc TEXT);
</pre>
Add a new schedule:
<pre>
[backup@bahamas ~]$ sbk --add --time="11:00 " --backup-type=archive --source-host=japan --dest-host=bahamas --source-dir=/etc/ --dest-dir=/data/backup/japan/etc/ --source-user=backup --dest-user=backup--desc "archive of japan /etc-> bahamas"
</pre>
=== Logging ===
[UPDATE]
A new logging section has been added to the schedule.db. This allows for specific logging events to be saved with dates and success or failure, for more accurate logging.
I could not figure out the format for the logging. Too many options. I went with a procedure where it makes a new log file each time the program is run. This could be a problem if you run the script too frequently, since it will make so many log files. I think the best idea would be to log to a single file, or to log into the sqlite3 database. I have not had time to change this yet.
# backup user created on all computers
# backup user must be able to ssh without a password from any computer to any other as backup user
# backup user must have sudo access with the nopasswd option on the rsync program and tar program(Security risk! Giving rsync sudo access allows backup user to modify any file.)visudo: backup ALL=(ALL) NOPASSWD: /usr/bin/rsync, /bin/tar
# root user must be able to ssh to all backup users from any computer(This is annoying, trying to find a way around this.)
# add custom users such as koji to work with ssh no password to all backup users, give root access to koji user in the same way
# WARNING, make sure you disable the passwords on all these backup accounts, that way they can't log in and get access to root without a private key
# IMPORTANT - visudo on each machine you connect would like to connect to:<pre>Defaults:backup !requiretty</pre>
otherwise sudo will complain about not having a tty
This list of configurations, that need to be done to each computer, is annoying and could be done better. Currently looking for ways to change it. After these configurations are made, you can use this host in any backup schedule.