Lesson 17 Reading and Writing Files

1. Lesson 17Reading and Writing Files Python Mini-Course University of Oklahoma Department of Psychology Python Mini-Course: Lesson 17

2. Lesson objectives • Open files for reading, writing, or appending data • Write data to a text file • Use the os module to manipulate paths and pathnames • Use the pickle module to store complex data types Python Mini-Course: Lesson 17

3. Files in Python • Files are objects • http://www.python.org/doc/2.5.2/lib/bltin-file-objects.html • Python file methods are wrappers for the standard C stdio package Python Mini-Course: Lesson 17

4. File types • Text file • Contains ACSII or Unicode characters • Can be created and read by most applications • Text editors (Notepad, SimpleText, etc.) • IDEs (IDLE, SPE, Eclipse, etc.) • Word processors (MS Word, etc.) • Spreadsheet programs (Excel, etc.) • Other apps (SAS, SPSS, R, Mathmatica, etc.) Python Mini-Course: Lesson 17

5. File types • Binary file • Contain data coded in other formats • Examples: • JPEG images • Audio or video clips • Packed binary data from FORTRAN • Matlab data files (.m files) Python Mini-Course: Lesson 17

6. The open statement • Returns a file object for access with file methods • Syntax fid = open(filename, mode) where fid is the name of the file object Python Mini-Course: Lesson 17

7. The filename argument • Should be a string containing the complete name of the file, including the file extension • NB: In MS Windows, most file extensions are hidden in Windows Explorer • Can include a partial or complete path • Default path is the folder containing the main script (.py file) Python Mini-Course: Lesson 17

8. File modes: reading a file • 'r' read (text file) • 'rb' read (binary file) • Can read file contents but cannot change file • If file does not exist, raises exception Python Mini-Course: Lesson 17

9. File modes: writing • 'w' write (text file) • 'wb' write (binary file) • Create a new file • Overwrites existing file if there is one Python Mini-Course: Lesson 17

10. File modes: append • 'a' append (text file) • 'ab' append (binary file) • Append data to (the end of) a file • If file does not exist, creates a new file Python Mini-Course: Lesson 17

11. File modes: mixed modes • 'r+' read and write existing file • If file does not exist, raises exception • 'a+' read and write existing file • Creates new file if one does not exist • 'w+' read and write a new file • Overwrites file if it already exists Python Mini-Course: Lesson 17

12. Note • Data transferred between files and your programs is represented as Python strings, even if it is binary data. • String objects can contain character bytes of any value Python Mini-Course: Lesson 17

13. End-of-line translations • Unix and Linux (and Mac OS X) • Use newline: \n • DOS and Windows • Use return + newline: \r\n • Old Mac OSs • Use return: \r Python Mini-Course: Lesson 17

14. End-of-line translations • Python automatically translates Windows EOLs when reading and writing files on Windows platforms • When in text mode • Not in binary mode Python Mini-Course: Lesson 17

15. Example: eol.py, win.txt, mac.txt text_mode = [open('win.txt','r').read(), open('mac.txt','r').read()] print 'Text mode:' print text_mode binary_mode = [open('win.txt','rb').read(), open('mac.txt','rb').read()] print '\nBinary mode:' print binary_mode Python Mini-Course: Lesson 17

16. File read methods • file.read() • Read all data until EOF is reached and return as a string object • file.readline() • Read one entire line from the file (keeps the trailing newline character) and return as a string object • file.readlines() • Read until EOF using readline() and return a list containing the lines thus read Python Mini-Course: Lesson 17

17. Example: read.py fin = open('win.txt', 'r') print fin.read() fin.seek(0) print fin.readline() fin.seek(0) print fin.readlines() fin.close() Python Mini-Course: Lesson 17

18. File write methods • file.write(str) • Write a string to the file • NB: Due to buffering, the string may not actually show up in the file until the flush() or close() method is called • file.writelines(sequence) • Write a sequence of strings to the file • NB: Does not add line separators, but this can be done using the string join operator Python Mini-Course: Lesson 17

19. Example: randnums.py import random fout = open('rand.txt', 'w') fout.write('Number\n') seq = [] for i in range(10): s = '%2.4f' % (random.random()) seq.append(s) fout.writelines('\n'.join(seq)) fout.close() Python Mini-Course: Lesson 17

20. Example: randnums2.py import random fout = open('rand.txt', 'w') fout.write('Index\tNumber\n') seq = [] for i in range(10): s = '%d\t%2.4f' % (i, random.random()) seq.append(s) fout.write('\n'.join(seq)) fout.close() Python Mini-Course: Lesson 17

21. The os module • Provides generic operating system (OS) support and a standard, platform-independent OS interface • Includes tools for environments, processes, files, shell commands, and much more • http://www.python.org/doc/2.5.4/lib/module-os.html Python Mini-Course: Lesson 17

22. File and directory commands • os.getcwd() • Returns the name of the current wording directory as a string • os.chdir(path) • Changes the current working directory for this process to path, a directory name string Python Mini-Course: Lesson 17

23. File and directory commands • os.listdir(path) • Returns a list of names of all the entries in the directory path Python Mini-Course: Lesson 17

24. Portability constants • os.curdir() • String for the current directory • os.pardir() • String for the parent directory • os.sep() • String used to separate directories • os.linesep() • String used to terminate lines Python Mini-Course: Lesson 17

25. The pickle module • Used to serialize and de-serialize a Python object structure http://www.python.org/doc/2.5.4/lib/module-pickle.html Python Mini-Course: Lesson 17

26. The pickle module • Pickling • the process whereby a Python object hierarchy is converted into a byte stream • Unpickling • the inverse operation, whereby a byte stream is converted back into an object hierarchy Python Mini-Course: Lesson 17

27. The pickle module • pickle.dump(obj, file) • Write a pickled representation of obj to the open file object file • pickle.load(file) • Read a string from the open file object file and interpret it as a pickle data stream, reconstructing and returning the original object hierarchy Python Mini-Course: Lesson 17

28. Example: pickling.py import random, pickle seq = [] for i in range(10): s = '%d\t%2.4f' % (i, random.random()) seq.append(s) print seq f = open('temp.pk', 'w') pickle.dump(seq, f) f.close() Python Mini-Course: Lesson 17

29. Example: pickling.py seq = [] print seq f = open('temp.pk', 'r') seq = pickle.load(f) print seq Python Mini-Course: Lesson 17