Getting data home

1. Getting data home ?

2. Getting data home ? FTP

3. Getting data home ? FTP tapes

4. Getting data home ? FTP: not secure tapes: incompatible

5. Getting data home ? FTP: not secure tapes: incompatible distracting

6. Getting data home ? http://bl831.als.lbl.gov/~jamesh/beamline/moving_data.html

7. Getting data home ? http://bl831.als.lbl.gov/~jamesh/beamline/moving_data.html http://bcsb-web.als.lbl.gov/beamline/Sector8/DataBackup.htm

8. Getting data home ? http://bl831.als.lbl.gov/~jamesh/beamline/moving_data.html http://bcsb-web.als.lbl.gov/beamline/Sector8/DataBackup.htm http://ucxray.berkeley.edu/~jamesh/elves/manual/tricks.html

9. Can you login? ssh user@home

10. Universal directory-transfer tar cf - . | ssh user@home “cd /dir ; tar xvf -”

11. Universal directory-transfer tar cf - . | ssh user@home “cd /dir ; tar xvf -”

12. Universal directory-transfer tar cf - . | ssh user@home “cd /dir ; tar xvf -” Password: Ctrl-z bg

13. Making it faster tar cf - . | compress | ssh user@home “cd /dir ; uncompress | tar xvf -”

14. The easy way http://bl831.als.lbl.gov/~jamesh/elves/scripts/sendhome sendhome . user@home:/dir

15. Real Time http://bl831.als.lbl.gov/~jamesh/elves/scripts/sendhome sendhome first_001.img user@home:/dir

16. Through a firewall tar cf - . | ssh user@firewall ‘ssh fileserver “cd /dir ; tar xvf -”’

17. The other way ssh user@home cd /dir ssh user@beamline “cd /data ; tar cf - .” | tar xvf –

18. The other way, faster ssh user@beamline “cd /dir ; tar cf - . | compress” | uncompress | tar xvf –

19. The easy other way ssh user@beamline “download yourname” | tar xvf –

20. What if it stops?

21. What if it stops? ssh user@beamline “download yourname frame_046.img” | tar xvf –

22. What if it stops? http://bl831.als.lbl.gov/~jamesh/elves/scripts/sendhome sendhome frame_046.img user@home:/dir

23. What if I’m confused?

24. What if I’m confused? 4 steps

25. 1) Generate a filename list find .-name ‘*.img’ >! here.log

26. 2) Generate a “done” list ssh user@home “cd /dir ; find .” >! there.log

27. 3) Make the “todo” list diffthere.loghere.log | awk ‘/^>/{print $2}’ >! 2bsent.log

28. 4) Feed “still left” list to tar cat 2bsent.log | tar cTf-- | ssh user@home “cd /dir ; tar xvf -”

29. What if I can’t login? ssh user@home warning: Connecting to home failed: Connection Refused

30. “SneakerNet”

31. “SneakerNet” Tape Firewire DVD

32. FireWire

33. Realtime Firewire 1)plug in

34. Realtime Firewire • plug in • share it

35. Realtime Firewire • plug in • share it • smbclient

36. Realtime Firewire tar cf - . | smbclient //laptop/share –U user%password" -N -Tx -

37. Realtime Firewire find . | tar cTf - - | smbclient //laptop/share –U user%password" -N -Tx -

38. The easy way http://bl831.als.lbl.gov/~jamesh/beamline/backup_firewire.com.txt backup_firewire.com .

39. DVD

40. DVD

41. HOWTO make a DVD mkisofs -r -J . >! image.iso

42. HOWTO make a DVD mkisofs -r -J . >! image.iso dvdrecord dev=0,0,0 –data image.iso

43. The easy way http://bl831.als.lbl.gov/~jamesh/beamline/backup_local.com.txt backup_local.com .

44. The easy way http://bl831.als.lbl.gov/~jamesh/beamline/backup_local.com.txt backup_local.com . http://bl831.als.lbl.gov/~jamesh/beamline/autolabel.com.txt autolabel.com

46. DVD data archive

47. DVD data archive

48. DVD data archive

49. DVD data archive

50. DVD data archive