<?xml version="1.0" encoding="utf-8"?><!DOCTYPE article  PUBLIC '-//OASIS//DTD DocBook XML V4.4//EN'  'http://www.docbook.org/xml/4.4/docbookx.dtd'><article><articleinfo><title>transferring data</title><revhistory><revision><revnumber>17</revnumber><date>2023-04-25 11:28:41</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>16</revnumber><date>2023-04-25 11:26:26</date><authorinitials>fhoma</authorinitials><revremark>typo</revremark></revision><revision><revnumber>15</revnumber><date>2023-04-25 09:53:11</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>14</revnumber><date>2021-10-29 14:53:55</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>13</revnumber><date>2021-10-29 14:36:20</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>12</revnumber><date>2021-10-29 14:28:46</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>11</revnumber><date>2021-10-29 09:34:03</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>10</revnumber><date>2021-10-29 09:23:35</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>9</revnumber><date>2021-10-28 15:44:12</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>8</revnumber><date>2020-05-13 09:48:31</date><authorinitials>fhoma</authorinitials><revremark>typo</revremark></revision><revision><revnumber>7</revnumber><date>2020-05-13 09:47:32</date><authorinitials>fhoma</authorinitials><revremark>typo</revremark></revision><revision><revnumber>6</revnumber><date>2020-04-17 17:02:57</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>5</revnumber><date>2020-04-17 17:01:39</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>4</revnumber><date>2020-04-17 16:22:42</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>3</revnumber><date>2020-04-14 08:51:13</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>2</revnumber><date>2020-04-14 08:44:51</date><authorinitials>fhoma</authorinitials></revision><revision><revnumber>1</revnumber><date>2020-04-14 08:43:38</date><authorinitials>fhoma</authorinitials></revision></revhistory></articleinfo><para><anchor id="top|scroll to the top of the page"/> </para><section><title>Transferring data</title><para>Before anything, read first about the storage and login. We will assume you are logged in potato.wurnet.nl server. </para><section><title>Copy programs to use</title><para>This machine supports several commands to copy files but for non privileged users only 3 are recommended (rsync, scp, cp). </para><para><emphasis><emphasis role="strong">Rsync</emphasis></emphasis> and <emphasis><emphasis role="strong">scp</emphasis></emphasis> can both be used by users to copy their data from a distant machine or from one partition to another. </para><para><emphasis><emphasis role="strong">Rsync</emphasis></emphasis> program is more versatile and has the ability to only copy files that are different or missing. </para><para>On the server itself the command <emphasis><emphasis role="strong">cp</emphasis></emphasis> can be used to copy files and folders. </para><para>The definition of those command is always the same: cmd [options] source destination </para></section><section><title>Data to analyze</title><para>Let's assume the following username <emphasis><emphasis role="strong">alberto</emphasis></emphasis> tries to copy sequence data from micro4 server to this server to run an assembly. </para><para>We also assume that on <emphasis><emphasis role="strong">micro4</emphasis></emphasis> server files are located in /home/alberto/all_my_data/illumina/sample_1/ </para><para>First create a folder for your run in the <emphasis><emphasis role="strong">/work</emphasis></emphasis> partition. </para><para>For example: </para><programlisting format="linespecific" language="bash" linenumbering="numbered" startinglinenumber="1"><![CDATA[mkdir -p /work/alberto/assembly ]]><lineannotation><![CDATA[#Folder for data and results]]></lineannotation>
<lineannotation></lineannotation><![CDATA[mkdir /work/alberto/assembly/raw_data ]]><lineannotation><![CDATA[#Folder where to send your raw data]]></lineannotation>
<lineannotation></lineannotation>
</programlisting><para>The data can now be copied with the following command: </para><para>To copy only files do: </para><programlisting format="linespecific" language="bash" linenumbering="numbered" startinglinenumber="1"><![CDATA[rsync -avztP alberto@micro4:/home/alberto/all_my_data/illumina/sample_1/sample_1_R1.fastq.gz /work/alberto/assembly/raw_data/]]>
</programlisting><para>To copy entre folder do (please notice that the trailing / in the source folder path has been removed): </para><programlisting format="linespecific" language="bash" linenumbering="numbered" startinglinenumber="1"><![CDATA[rsync -avztrP alberto@micro4:/home/alberto/all_my_data/illumina/sample_1 /work/alberto/assembly/raw_data/]]>
</programlisting><para>You are now to run you analyses. </para></section><section><title>Raw sequencing data</title><para>Data coming from sequencing company or machine (so called raw data) should be placed in a specific place <emphasis><emphasis role="strong">/seq/sequencing_runs</emphasis></emphasis>. Only admins can/should add this to avoid wrong manipulation of the data. Please contact the local admin, <code>&lt;bioinfo DOT mib AT wur DOT nl&gt;</code> </para><para>This rule also applies to data received from collaborators, stored in <emphasis><emphasis role="strong">/seq/collaborators_seqs/</emphasis></emphasis>. </para></section><section><title>WARNINGS</title><caution><para><emphasis role="strong">Clean your data</emphasis> to make sure the servers are running fine for everybody. Take the habit of moving your data to a more appropriate place. /work partition should only be reserved for running data. It will not be backed up. </para></caution></section><section><title>Quick links</title><orderedlist numeration="arabic"><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/Computers%20description#">Computers description</ulink> </para></listitem><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/Request%20an%20account#">Request an account</ulink> </para></listitem><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/storage%20description#">storage description</ulink> </para></listitem><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/First%20login#">First login</ulink> </para></listitem><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/Receiving%20sequencing%20data#">Receiving sequencing data</ulink> </para></listitem><listitem><para><link linkend="">transferring data</link> </para></listitem><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/Submitting%20jobs#">Submitting jobs</ulink> </para></listitem><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/Monitor%20jobs#">Monitor jobs</ulink> </para></listitem><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/Install%20tools#">Install tools</ulink> </para></listitem><listitem><para><ulink url="https://platecarpus.wur.nl/wiki.potato/transferring%20data/wiki.potato/Space%20management#">Space management</ulink> </para></listitem></orderedlist><para><link linkend="top">#top</link> </para></section></section></article>