|
Size: 2008
Comment:
|
← Revision 17 as of 2023-04-25 11:28:41 ⇥
Size: 2897
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from transfering data <<Anchor(top|scroll to the top of the page)>> |
|
| Line 2: | Line 4: |
| ---- | <<TableOfContents(2)>> |
| Line 4: | Line 6: |
| Before anything read first about the storage and login. We will assume you are logged in potato.wurnet.nl server. | Before anything, read first about the storage and login. We will assume you are logged in potato.wurnet.nl server. |
| Line 8: | Line 10: |
| This machine support several software to copy files but for users only 3 are recommended (rsync, scp, cp). | This machine supports several commands to copy files but for non privileged users only 3 are recommended (rsync, scp, cp). |
| Line 10: | Line 12: |
| '''''Rsync''''' and '''''scp''''' can both be used by users to copy their data from a distant machine or from one partition to another one. | '''''Rsync''''' and '''''scp''''' can both be used by users to copy their data from a distant machine or from one partition to another. |
| Line 21: | Line 23: |
| Let's assume the following username '''''alberto''''' tries to copy sequenced data from micro4 server to this server to run an assembly. We also assume that on micro4 server files are located in /home/alberto/all_my_data/illumina/sample_1/ |
Let's assume the following username '''''alberto''''' tries to copy sequence data from micro4 server to this server to run an assembly. We also assume that on '''''micro4''''' server files are located in /home/alberto/all_my_data/illumina/sample_1/ |
| Line 49: | Line 52: |
| == Raw sequencing data == Data coming from sequencing company or machine (so called raw data) should be placed in a specific place '''''/seq/sequencing_runs'''''. Only admins can/should add this to avoid wrong manipulation of the data. Please contact the local admin, <<MailTo(bioinfo DOT mib AT wur DOT nl)>> This rule also applies to data received from collaborators, stored in '''''/seq/collaborators_seqs/'''''. |
|
| Line 52: | Line 63: |
| '''Clean your data''' To make sure the servers are running fine for everybody, take the habit of moving your data to a more appropriate place. /work partition should only be reserved for running data. |
'''Clean your data''' to make sure the servers are running fine for everybody. Take the habit of moving your data to a more appropriate place. /work partition should only be reserved for running data. It will not be backed up. |
| Line 56: | Line 66: |
== Quick links == 1. [[Computers description]] 2. [[Request an account]] 3. [[storage description]] 4. [[First login]] 5. [[Receiving sequencing data]] 6. [[transferring data]] 7. [[Submitting jobs]] 8. [[Monitor jobs]] 9. [[Install tools]] 10. [[Space management]] [[#top]] |
Transferring data
Contents
Before anything, read first about the storage and login. We will assume you are logged in potato.wurnet.nl server.
Copy programs to use
This machine supports several commands to copy files but for non privileged users only 3 are recommended (rsync, scp, cp).
Rsync and scp can both be used by users to copy their data from a distant machine or from one partition to another.
Rsync program is more versatile and has the ability to only copy files that are different or missing.
On the server itself the command cp can be used to copy files and folders.
The definition of those command is always the same: cmd [options] source destination
Data to analyze
Let's assume the following username alberto tries to copy sequence data from micro4 server to this server to run an assembly.
We also assume that on micro4 server files are located in /home/alberto/all_my_data/illumina/sample_1/
First create a folder for your run in the /work partition.
For example:
The data can now be copied with the following command:
To copy only files do:
1 rsync -avztP alberto@micro4:/home/alberto/all_my_data/illumina/sample_1/sample_1_R1.fastq.gz /work/alberto/assembly/raw_data/
To copy entre folder do (please notice that the trailing / in the source folder path has been removed):
1 rsync -avztrP alberto@micro4:/home/alberto/all_my_data/illumina/sample_1 /work/alberto/assembly/raw_data/
You are now to run you analyses.
Raw sequencing data
Data coming from sequencing company or machine (so called raw data) should be placed in a specific place /seq/sequencing_runs. Only admins can/should add this to avoid wrong manipulation of the data. Please contact the local admin, <bioinfo DOT mib AT wur DOT nl>
This rule also applies to data received from collaborators, stored in /seq/collaborators_seqs/.
WARNINGS
Clean your data to make sure the servers are running fine for everybody. Take the habit of moving your data to a more appropriate place. /work partition should only be reserved for running data. It will not be backed up.