Instructions: Downloading BINARY(compressed) Data and Renaming the Files


Downloading
First off, make sure you have the correct file uncompression software. If the file name ends in ".gz" then this file was compressed using the software package "gzip" which was adopted by the GNU project. For more information, go to the web site for the GNU project, or the gzip web site to download this utility.

Most problems we've encountered relating to data downloads are caused by web browsers uncompressing files and not renaming them properly, or renaming files without uncompressing them. In either case, it will cause problems for you. These are issues that we have no control over, yet as users of software must find a workaround solution. If you have some type of ftp(file transer protocol) tool available, use it for data downloads. When ftp is used properly, I've never had downloading problems(e.g set the transfer mode as binary for compressed files rather than letting the software determine the type automatically). If you still want to use a web browser, then read on.

Web browsers are powerful tools for data download if all works properly. If not, you must decipher what was done incorrectly. All the problems I've encountered are due to combinations of file compression/decompression and file naming, or lack thereof. There are 4 possible combinations described below.

This table is a quick summary. Assuming that you are having trouble, look at the name of the file that was downloaded. The two cells below that are colored red most likely contain the solution to problems caused by the web browser. Assume we have some generic filename <filename> and it's compressed version would be <filename>.gz. In reality, that filename could be something like "precip.e00" or "us_ppt_1971_2000.01" and the compressed names would be "precip.e00.gz" or "us_ppt_1971_2000.01.gz", respectively. The actual name is of less importance than what the web browser does with the ".gz" part of the name.

was file uncompressed on the fly?
was file renamed? yes no
yes Downloaded file name = <filename>

Hint : the file size should have grown considerably larger than the original file size.

Do nothing as download and uncompression worked ok.

Downloaded file name = <filename>

Hint : the file size should have stay approximately the same size

Look at the data in a text editor. If the data is not ASCII characters (readable text and numbers), then rename the file to <filename>.gz and manually run uncompression software.

no Downloaded file name = <filename>.gz

Hint : the file size should have grown considerably larger than the original file size.

Look at the data in a text editor. If the data is ASCII characters, then the data has been uncompressed already so rename the file to <filename>.

Downloaded file name = <filename>.gz

Hint : the file size should have stay approximately the same size

Manually run uncompression software on data file

For a more detailed explanation of what actually happened during download, read on.

Assume the data file resides on the computer "server" and the name is <filename>.gz. You will then download the data to computer "client" and the uncompressed version of the file is <filename>.

  1. Compressed data is uncompressed "on the fly" (e.g. by the web browser before saving) and the file is renamed correctly.

    In this case, the web browser correctly uncompressed the data and renamed the file. If this happens, you probably are not reading this document. So the file transfer was as follows :

    server : <filename>.gz

    The data is transferred and uncompressed. It is found on your computer as

    client : <filename>

  2. Compressed data is NOT uncompressed and the file is NOT renamed.

    In this case, the web browser never uncompressed the data or renamed the file. If this happens, you will have to uncompress the file on your computer using the proper uncompression software. So the file transfer was as follows :

    server : <filename>.gz

    The data is transferred and the file is saved on your computer as

    client : <filename>.gz

    You will need to manually run uncompression software on data file.

  3. Compressed data is NOT uncompressed and the file is renamed as though it was uncompressed.

    In this case, the web browser did not uncompressed the data but renamed the file. This is not correct. So the file transfer was as follows :

    server : <filename>.gz

    The data is transferred and renamed but the data was not uncompressed on the fly as it should have been. It is found on your computer as

    client : <filename>

    To fix the problem, rename the file <filename>.gz and uncompress using the correct software.

  4. Compressed data is uncompressed but the file name is NOT changed.

    In this case, the web browser uncompressed the data but but did not rename the file. This is not correct. So the file transfer was as follows :

    server : <filename>.gz

    The data is transferred, uncompressed, but it was not renamed and this is incorrect. It is found on your computer as

    client : <filename>.gz

    To fix the problem, rename the file <filename>.

Renaming
This discussion is specific to ESRI's GIS software. Most of ESRI's software tools follow the MS-DOS FAT file naming convention (8 character for the base part of the name and 3 character for the file extension with a period (".") separating the base part from the file extension) while some follow the long name convention (not limited to 8 character for the base part of the name). If you don't follow these conventions, you most likely will run into issues with the software working properly.

If the file extension is e00, you shouldn't need to rename the file. If you do need to rename it, you should only need to rename the base part of the name. For example, if the file you downloaded is named us_ppt_1971_2000.e00, rename it to ppt_avg.e00

If you downloaded an ASCII format file and it doesn't have an extension of txt or asc, you will need to rename it. Let me emphasize that the file extension must be either txt or asc. If you don't follow this convention, you will have problems converting the ARCINFO ASCII GRID format to Raster (e.g. ASCII to Raster tool). For example, if the file you downloaded is named us_ppt_1971_2000.01, rename it to ppt_01.txt

Further information on file name specifications can be found on the Microsoft web site.


Wayne Gibson, The PRISM Climate Group, Oregon State Univeristy
Copyright © 20XX by , The PRISM Climate Group, Oregon State University