530 likes | 537 Views
Learn various methods to transfer data home securely, including FTP, tapes, and troubleshooting tips for compatibility issues. Improve your workflow and avoid distractions.
E N D
Getting data home ? FTP
Getting data home ? FTP tapes
Getting data home ? FTP: not secure tapes: incompatible
Getting data home ? FTP: not secure tapes: incompatible distracting
Getting data home ? http://bl831.als.lbl.gov/~jamesh/beamline/moving_data.html
Getting data home ? http://bl831.als.lbl.gov/~jamesh/beamline/moving_data.html http://bcsb-web.als.lbl.gov/beamline/Sector8/DataBackup.htm
Getting data home ? http://bl831.als.lbl.gov/~jamesh/beamline/moving_data.html http://bcsb-web.als.lbl.gov/beamline/Sector8/DataBackup.htm http://ucxray.berkeley.edu/~jamesh/elves/manual/tricks.html
Can you login? ssh user@home
Universal directory-transfer tar cf - . | ssh user@home “cd /dir ; tar xvf -”
Universal directory-transfer tar cf - . | ssh user@home “cd /dir ; tar xvf -”
Universal directory-transfer tar cf - . | ssh user@home “cd /dir ; tar xvf -” Password: Ctrl-z bg
Making it faster tar cf - . | compress | ssh user@home “cd /dir ; uncompress | tar xvf -”
The easy way http://bl831.als.lbl.gov/~jamesh/elves/scripts/sendhome sendhome . user@home:/dir
Real Time http://bl831.als.lbl.gov/~jamesh/elves/scripts/sendhome sendhome first_001.img user@home:/dir
Through a firewall tar cf - . | ssh user@firewall ‘ssh fileserver “cd /dir ; tar xvf -”’
The other way ssh user@home cd /dir ssh user@beamline “cd /data ; tar cf - .” | tar xvf –
The other way, faster ssh user@beamline “cd /dir ; tar cf - . | compress” | uncompress | tar xvf –
The easy other way ssh user@beamline “download yourname” | tar xvf –
What if it stops? ssh user@beamline “download yourname frame_046.img” | tar xvf –
What if it stops? http://bl831.als.lbl.gov/~jamesh/elves/scripts/sendhome sendhome frame_046.img user@home:/dir
What if I’m confused? 4 steps
1) Generate a filename list find .-name ‘*.img’ >! here.log
2) Generate a “done” list ssh user@home “cd /dir ; find .” >! there.log
3) Make the “todo” list diffthere.loghere.log | awk ‘/^>/{print $2}’ >! 2bsent.log
4) Feed “still left” list to tar cat 2bsent.log | tar cTf-- | ssh user@home “cd /dir ; tar xvf -”
What if I can’t login? ssh user@home warning: Connecting to home failed: Connection Refused
“SneakerNet” Tape Firewire DVD
Realtime Firewire 1)plug in
Realtime Firewire • plug in • share it
Realtime Firewire • plug in • share it • smbclient
Realtime Firewire tar cf - . | smbclient //laptop/share –U user%password" -N -Tx -
Realtime Firewire find . | tar cTf - - | smbclient //laptop/share –U user%password" -N -Tx -
The easy way http://bl831.als.lbl.gov/~jamesh/beamline/backup_firewire.com.txt backup_firewire.com .
HOWTO make a DVD mkisofs -r -J . >! image.iso
HOWTO make a DVD mkisofs -r -J . >! image.iso dvdrecord dev=0,0,0 –data image.iso
The easy way http://bl831.als.lbl.gov/~jamesh/beamline/backup_local.com.txt backup_local.com .
The easy way http://bl831.als.lbl.gov/~jamesh/beamline/backup_local.com.txt backup_local.com . http://bl831.als.lbl.gov/~jamesh/beamline/autolabel.com.txt autolabel.com