Introduction

Since, you probably has data on our former servers and maybe on other machines (workstations or so), On this page you can find some help on how to transfer data from other servers to valhalla. Please ensure your account is set and secured on valhalla before going further.

preparation

Before copying files over to the new server, we recommend that you sort and organise your files and folders. This will minimise network usage and make the copying process easier to follow. Sorting the data needs to be done in '/work/username/' if you are working on potato server, or on a local disk if you are working on a workstation. Create the folder structure shown below to contain all the files and folders that should be moved to the new server.

/work/username/
movingToValhalla/
    Home/
    Archive/
        project1/
        project2/
    Current/
        project1/
        project2/
    Scripts/
        script1/script1.R
        script2/script2.py
        util.py

• Move any bioinformatics output from your home folder to the relevant projects folder.
• Transfer any relevant files from your home folder to '/work/username/movingToValhalla/Home/'.
• All sequencing runs should already be available on the new server. If not, contact bioinfo.mib@wur.nl.
• Place files that should be kept for a period of 10+ years in the relevant project folder in Archive/. Bioinformatics files should be organised by project, with each project folder containing a README file.
  • the goal of the project
  • the workflow (a diagram is often easier to read)
  • the commands used to generate each file. Delete any files that are no longer needed. File deletion should be done using a relative path, never an absolute path. In case of doubt, keep the files until all doubts are cleared.

Ensure that the home folder contains no bioinformatics output.

Use the command below to move a file/folder from its current location to another location. {{{code: id="move" lang="sh" linenumbers="False" mv source destination }}} we discourage the use of "cp" as it duplicates files. Use links instead of 'cp'.

Transfer files

The recommended command to remote copy files between servers is, rsync

1. Login to the server where your data is (e.g. ssh username@potato)
2. transfer data to valhalla, rsync -avz /folder/to/transfer user@valhalla:/work/username/

Valhalla users

The new infrastructure runs on a separate network and can only be accessed from specific locations.

• Potato
• Helicoprion
• Snellius
• Yoda
• WUR VPN.

Any file transfers to the new MIB servers (Valhalla) must be made from the machines and network mentioned above. Any data you have elsewhere should first be transferred there.