Transferring data
Contents
Before anything, read first about the storage and login. We will assume you are logged in potato.wurnet.nl server.
Copy programs to use
This machine supports several commands to copy files but for non privileged users only 3 are recommended (rsync, scp, cp).
Rsync and scp can both be used by users to copy their data from a distant machine or from one partition to another.
Rsync program is more versatile and has the ability to only copy files that are different or missing.
On the server itself the command cp can be used to copy files and folders.
The definition of those command is always the same: cmd [options] source destination
Data to analyze
Let's assume the following username alberto tries to copy sequence data from micro4 server to this server to run an assembly.
We also assume that on micro4 server files are located in /home/alberto/all_my_data/illumina/sample_1/
First create a folder for your run in the /work partition.
For example:
The data can now be copied with the following command:
To copy only files do:
1 rsync -avztP alberto@micro4:/home/alberto/all_my_data/illumina/sample_1/sample_1_R1.fastq.gz /work/alberto/assembly/raw_data/
To copy entre folder do (please notice that the trailing / in the source folder path has been removed):
1 rsync -avztrP alberto@micro4:/home/alberto/all_my_data/illumina/sample_1 /work/alberto/assembly/raw_data/
You are now to run you analyses.
Raw sequencing data
Data coming from sequencing company or machine (so called raw data) should be placed in a specific place /seq/sequencing_runs. Only admins can/should add this to avoid wrong manipulation of the data. Please contact the local admin, <bioinfo DOT mib AT wur DOT nl>
This rule also applies to data received from collaborators, stored in /seq/collaborators_seqs/.
WARNINGS
Clean your data to make sure the servers are running fine for everybody. Take the habit of moving your data to a more appropriate place. /work partition should only be reserved for running data. It will not be backed up.