Census 2000: File Transfer Protocol for The Redistricting Summary File PL 94-171

1. Census 2000: File Transfer Protocol for The Redistricting Summary File (PL 94-171)

2. FTP Process FTP is File Transfer Protocol Transfers a complete state file from the Census Bureau computers to the user The FTP, UnZIP, spreadsheet / database process is for experienced data and software users We have several objectives for this presentation: --Explain what FTP means and how to do it --Reinforce that this process is for the experienced user --Reinforce the use of the README file in the FTP directory. File Transfer Protocol is the means by which users can directly download data files to a personal computer (PC). For the Census 2000 data, the FTP process provides for downloading of complete state files (all tables, all geographies). Users wanting to work with a subset of the data are better served by either the CD-ROM or American FactFinder. There are multiple steps in the FTP process. Because of the complexity of the file manipulation, we definitely recommend that it be done only by experienced data users. In the words of the sports car commercials, “Professional driver, don’t try this at home!” Public Law 94-171 data files available for downloading are ASCII files. In this presentation we will walk through actual steps a user follows when using the FTP to send a state file to a PC. We have several objectives for this presentation: --Explain what FTP means and how to do it --Reinforce that this process is for the experienced user --Reinforce the use of the README file in the FTP directory. File Transfer Protocol is the means by which users can directly download data files to a personal computer (PC). For the Census 2000 data, the FTP process provides for downloading of complete state files (all tables, all geographies). Users wanting to work with a subset of the data are better served by either the CD-ROM or American FactFinder. There are multiple steps in the FTP process. Because of the complexity of the file manipulation, we definitely recommend that it be done only by experienced data users. In the words of the sports car commercials, “Professional driver, don’t try this at home!” Public Law 94-171 data files available for downloading are ASCII files. In this presentation we will walk through actual steps a user follows when using the FTP to send a state file to a PC.

3. Census 2000 FTP Information Files are ZIPPED (compressed) to speed up transfer time No software is provided with the file Files are structured to provide easy import to the spreadsheet / data base software of choice Due to the large size of many of these files, they have been compressed in order to speed the downloading process. The compression ratio varies from about 80% to 95%. The README file in the FTP directory provides estimated file sizes for each state, both zipped and unzipped. It’s also important to remember that there is no software provided with the FTP files. No actual software is provided with the file. We will talk later in the presentation about the software needed to unzip the files. Even with the zipped files, a download of an average state of the redistricting data (at 56K bps, typical for a home computer with a modem) will take approximately 2 hours. Larger states, of course, will take longer. Our estimate for an FTP download for California is approximately 8 hours at the 56K bps speed. If you have a modem/telephone line link to the Internet, we do not recommend using the FTP option. The Redistricting data are segmented into 3 files. This is done to accommodate easily moving the data into the spreadsheet/database of choice. No segment has more than 255 cells.Due to the large size of many of these files, they have been compressed in order to speed the downloading process. The compression ratio varies from about 80% to 95%. The README file in the FTP directory provides estimated file sizes for each state, both zipped and unzipped. It’s also important to remember that there is no software provided with the FTP files. No actual software is provided with the file. We will talk later in the presentation about the software needed to unzip the files. Even with the zipped files, a download of an average state of the redistricting data (at 56K bps, typical for a home computer with a modem) will take approximately 2 hours. Larger states, of course, will take longer. Our estimate for an FTP download for California is approximately 8 hours at the 56K bps speed. If you have a modem/telephone line link to the Internet, we do not recommend using the FTP option. The Redistricting data are segmented into 3 files. This is done to accommodate easily moving the data into the spreadsheet/database of choice. No segment has more than 255 cells.

4. FTP from American FactFinder This is the FactFinder main page where we can begin the ftp process. We can also go directly to the ftp site. I’ll show you that address in a few minutes. We start from the Data Set area on the left side of the main page. This is the FactFinder main page where we can begin the ftp process. We can also go directly to the ftp site. I’ll show you that address in a few minutes. We start from the Data Set area on the left side of the main page.

5. Data Sets in the Decennial Census Column Next we go to the Data Sets page of American FactFinder. The first column is for data relating to the decennial census. Next we go to the Data Sets page of American FactFinder. The first column is for data relating to the decennial census.

6. Scroll to the Bottom We scroll to the bottom of the Data Sets column to see the link for the FTP site. We scroll to the bottom of the Data Sets column to see the link for the FTP site.

7. ftp://ftp2.census.gov This is the FTP directory page. Notice that the address for direct access to the FTP directory is on this slide. These files are set up in the tree structure. We will begin by opening the Census_2000 file area.This is the FTP directory page. Notice that the address for direct access to the FTP directory is on this slide. These files are set up in the tree structure. We will begin by opening the Census_2000 file area.

8. Select Datasets Within the Census 2000 file area, we select Datasets so we can move on the the listing of data sets available for FTP.Within the Census 2000 file area, we select Datasets so we can move on the the listing of data sets available for FTP.

9. Select 0Test for Testing FTP Download Select the OTEST for FTP Download test file. (It’s named 0test so it can appear at the top!) The other two directories will contain data for redistricting and block data. If we were to select one of these directories, we would find (already there) a directory listing each state. When the redistricting/block data are released the three files for each state will appear in the state directory under the redistricting directory and the state directory under the block data directory.Select the OTEST for FTP Download test file. (It’s named 0test so it can appear at the top!) The other two directories will contain data for redistricting and block data. If we were to select one of these directories, we would find (already there) a directory listing each state. When the redistricting/block data are released the three files for each state will appear in the state directory under the redistricting directory and the state directory under the block data directory.

10. Select File for Downloading A critical file is the README file. It is here in three different formats: WordPerfect, Word, and text. The files are identical but we added the word processing files, since the text has lots of bold text for emphasis. This emphasis is, of course, lost in the text version. Moving on to the data files (*.zip), the data in this test file, in the redistricting/unadjusted block data, and in other Census 2000 summary files are segmented. This is done so that individual files do not have more than 255 fields, facilitating exporting into spreadsheet software. In short, to get the complete data set for the redistricting data/block counts files, users must FTP all three files in the state directory. In the slide above we see that the Geographic Header file is named tfgeo.uPL.zip. The next two files are tf0001.uPL.zip and tf0002.uPL.zip. The tf indicates test file. A critical file is the README file. It is here in three different formats: WordPerfect, Word, and text. The files are identical but we added the word processing files, since the text has lots of bold text for emphasis. This emphasis is, of course, lost in the text version. Moving on to the data files (*.zip), the data in this test file, in the redistricting/unadjusted block data, and in other Census 2000 summary files are segmented. This is done so that individual files do not have more than 255 fields, facilitating exporting into spreadsheet software. In short, to get the complete data set for the redistricting data/block counts files, users must FTP all three files in the state directory. In the slide above we see that the Geographic Header file is named tfgeo.uPL.zip. The next two files are tf0001.uPL.zip and tf0002.uPL.zip. The tf indicates test file.

11. README File is Critical Again, a plug for the README file. It contains information on: The FTP application itself Other sources of the data File information Segmentation of the data File record layout Estimated file sizes for each state Now we’ll continue on with the ftp demonstration.Again, a plug for the README file. It contains information on: The FTP application itself Other sources of the data File information Segmentation of the data File record layout Estimated file sizes for each state Now we’ll continue on with the ftp demonstration.

12. Save the File to Disk Here we see the usual Windows File Download screen that users should be familiar with. We save the file to disk in this case. Here we see the usual Windows File Download screen that users should be familiar with. We save the file to disk in this case.

13. Select Where the File is Saved Now simply select the folder you wish to download the Zipped file to.Now simply select the folder you wish to download the Zipped file to.

14. File Download Information Here we see the completion of the download. NOTE: The largest zipped file is 24 megabytes, the smallest < 1 megabyte.Here we see the completion of the download. NOTE: The largest zipped file is 24 megabytes, the smallest < 1 megabyte.

15. Download is Complete Now that the download is complete, we will open and unzip (decompress) the file with standard unzip software. Now that the download is complete, we will open and unzip (decompress) the file with standard unzip software.

16. UnZIP – Select Windows Explorer Through your Windows Explorer tool, locate the folder where the Zipped files were downloaded.Through your Windows Explorer tool, locate the folder where the Zipped files were downloaded.

17. Go to File Storage Location on Your Computer – Select File Select the file you want to decompressSelect the file you want to decompress

18. No UnZIP Software on Your Computer? See Your Network Administrator Available for Download at www.winzip.com , www.pkunzip.com and other sites If you don’t have unzip software on your computer, it is easy to obtain. Users on a network computer, might check with your Local Area Network (LAN) staff or system dministrator is. Otherwise the decompression software can be downloaded at no cost from the internet at the sites shown on the screen. If you don’t have unzip software on your computer, it is easy to obtain. Users on a network computer, might check with your Local Area Network (LAN) staff or system dministrator is. Otherwise the decompression software can be downloaded at no cost from the internet at the sites shown on the screen.

19. No Licensing Problem to UnZIP In this case we are using WinZip to decompress the file. As you can see in order to proceed with the download, users will be asked to Agree to the license agreement. After reviewing the agreement, Click on ‘I Agree’ to proceed. The software to ZIP files is sold; the software to unzip files is free. In this case we are using WinZip to decompress the file. As you can see in order to proceed with the download, users will be asked to Agree to the license agreement. After reviewing the agreement, Click on ‘I Agree’ to proceed. The software to ZIP files is sold; the software to unzip files is free.

20. Using WinZip - Select Next The WinZip wizard will take you through each step. At this point click on the Next button and continue on. The WinZip wizard will take you through each step. At this point click on the Next button and continue on.

21. Identify Storage Location, Then UnZIP Now, select the folder to which you want the Zipped file decompressed into. It’s a good practice to select a different directory that where you stored the zipped version. It makes it easier to see what you have. Once you have done this, click on the Unzip Now button.Now, select the folder to which you want the Zipped file decompressed into. It’s a good practice to select a different directory that where you stored the zipped version. It makes it easier to see what you have. Once you have done this, click on the Unzip Now button.

22. UnZIP File in Progress The file will unzip and move to the folder previously designated.The file will unzip and move to the folder previously designated.

23. Close or UnZIP Another File Users will receive confirmation of the successful unzip. After receiving this confirmation please continue with the other files, or close if you don’t want to unzip more files. Users will receive confirmation of the successful unzip. After receiving this confirmation please continue with the other files, or close if you don’t want to unzip more files.

24. UnZIPPED File Looks Like This PL, CA,0,1,1,377741,357428,185478,58443,11270,56877,2388,42972,20313,18599,2539,2827,2867,275,3514,978,521,110,822,337,1445,167,894,757,546,1545,444,134,15,208,135,30,58,122,148,20,40,25,23,25,23,2,34,4,2,53,132,33,14,42,10,2,3,11,3,0,2,8,1,0,2,1,27,21,2,0,3,1,0,10,10,377741,77320,300421,287125,161201,56172,9180,55259,2008,3305,13296,12209,2232,2002,2414,234,496,869,462,105,274,242,1291,29,840,287,432,999,355,115,12,101,84,15,21,103,35,13,31,18,12,21,6,0,16,0,2,39,72,21,10,22,7,0,2,5,0,0,0,1,1,0,2,1,12,6,2,0,3,1,0,4,4 This is an example of what the unzipped data file looks like. This is the comma-delimited format, with commas separating individual data items or data cells. It does not look like a pretty table! Users who want to see a single table for an area, in a nice presentation format, should utilize the table download (as opposed to file download or FTP) feature of American FactFinder or use the CD-ROM This is an example of what the unzipped data file looks like. This is the comma-delimited format, with commas separating individual data items or data cells. It does not look like a pretty table! Users who want to see a single table for an area, in a nice presentation format, should utilize the table download (as opposed to file download or FTP) feature of American FactFinder or use the CD-ROM

25. UnZIPPED File in Spread Sheet This is an example of what the data looks like when it is opened into a spreadsheet or data base package. In this case, we are using EXCEL. This is an example of what the data looks like when it is opened into a spreadsheet or data base package. In this case, we are using EXCEL.

26. Next Steps for User Import file into spreadsheet or database software of choice Data dictionary is needed See README for file information and Internet location of data dictionary Once the file has been successfully loaded into your software of choice, it needs to be formatted according to the data dictionary specifications. The dictionary is available in the Census Bureau web site and is cited in README. This is not a trivial task and is one of the reasons we recommend that only experienced users use the FTP option. In summary, we’ve stressed three things in this presentation: --FTP process is developed for the experienced user --FTP provides for downloading all tables and all geography for a state; no subsets --README file is an essential tool.Once the file has been successfully loaded into your software of choice, it needs to be formatted according to the data dictionary specifications. The dictionary is available in the Census Bureau web site and is cited in README. This is not a trivial task and is one of the reasons we recommend that only experienced users use the FTP option. In summary, we’ve stressed three things in this presentation: --FTP process is developed for the experienced user --FTP provides for downloading all tables and all geography for a state; no subsets --README file is an essential tool.