https://crwiki.cent.uw.edu.pl/ 2026-04-24T15:33:06+00:00 https://crwiki.cent.uw.edu.pl/ https://crwiki.cent.uw.edu.pl/lib/exe/fetch.php?media=wiki:logo.png text/html 2023-11-04T07:28:57+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_desc&rev=1699082937&do=diff Anthill ver 0.3 Hardware Software ---------- ---------- ---------- Before sending request to install new software on computing nodes : * use slurm interactive session and check ( it is probably already installed ) * if it is not installed yet, but is is not available in standard repositories consider installing it from source in user space. You can find some examples how to do it bellow. text/html 2024-07-18T11:40:56+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_hardware_v03&rev=1721302856&do=diff node name(s) # CPU RAM GPU /tmp /tmp2 network ant602-ant609 8 1x Intel(R) Core(TM) i9-7900X CPU @ 3.30GHz 16 GB 2x GeForce GTX 1080 Ti (11GB) ~ 390 GB (ssd) ~4 TB (sata) 1 Gbps ant101-ant105 5 2x Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_hardware&rev=1690844884&do=diff nodes (#2) : ant607, ant608 * cpu : 1x Intel(R) Core(TM) i9-7900X CPU @ 3.30GHz * gpu : 2x GeForce GTX 1080 Ti (11GB) * mem : 16 GB * scratch : 4 TB (sata 7.2k) * net : 1 Gb ethernet nodes (#1) : ant009 * cpu : 2x Intel(R) Xeon(R) CPU E5-2670 v0 @ 2.60GHz text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_soft_v03&rev=1690844884&do=diff Available on nodes : OS * based on ubuntu 22.04 LTS SLURM * slurm 21.08.3 GCC * 11.2.0 * 10.3.0 * 9.4.0 clang * clang : 14.0.0 python * python : 3.10.4 * python : 2.7.18 CUDA * Driver Version: 510.73.05 CUDA Version: 11.6 text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_soft&rev=1690844884&do=diff Available on nodes : OS * ubuntu 18 LTS SLURM * slurm 18.08.6-2 GCC * gcc-8 : 8.3.0 * gcc-7 : 7.4.0 * gcc-6 : 6.5.0 * gcc-4 : 4.8.5 clang * clang : 6.0.0 python * python 2.7.15rc1 * python 3.6.7 * python 3.8.2 CUDA * text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri001&rev=1690844884&do=diff description CD-HIT is a fast program for clustering and comparing large sets of protein or nucleotide sequences. Accelerated for clustering the next generation sequencing data. Software page : <http://cd-hit.org> . software version CD-HIT , from Ubuntu 18.04 repo (CD-HIT version 4.7 (built on Jul 1 2017)) . text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri002&rev=1690844884&do=diff FIXME description ... software version ... sbatch example ... ... performance tests ... ... ---------- software can be downloaded from : https://github.com/BrainTech/matching-pursuit software instalation @anthill23 If you wold like to use mp5 software you are required to install it by yourself. Firstly login to anthill23.cent1.uw.edu.pl and open interactive session : srun -p test --pty bash -l Then run following commands ( it will take less than 2 minutes ) : module load c… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri003&rev=1690844884&do=diff description Bedtools is a powerful toolset for genome arithmetic. About: developed in the quinlanlab.org and by many contributors worldwide. Docs: <http://bedtools.readthedocs.io/> Code: <https://github.com/arq5x/bedtools2> software version Version: v2.26.0 sbatch example ... #!/bin/bash -l #SBATCH --partition=long #SBATCH --ntasks=1 # <--- bedtool uses only one core #SBATCH --mem 2048M #SBATCH --job-name="bedtools test" #test filenames #https://s3.amaz… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri004&rev=1690844884&do=diff FIXME description ... software version ... sbatch example ... ... performance tests ... ... ---------- available descriptions see NAMD ver 2.12 issues error: No processors available for load balancing! With increase of number of allocated cores (>~200) you can see following error in NAMD's output file : ------------- Processor 0 Exiting: Called CmiAbort ------------ Reason: FATAL ERROR: No processors available for load balancing! [0] Stack Traceback: [0:0] CmiAbort+0x39 … text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri005&rev=1690844884&do=diff FIXME description ... software version ... sbatch example ... ... performance tests ... ... ---------- FIXME Anthill NAMD 212 preparation Below information allow you to prepare NAMD software on Anthill computing cluster. download NAMD source Source tarball NAMD_2.12_Source.tar.gz can be downloaded from : http://www.ks.uiuc.edu/Development/Download/download.cgi?PackageName=NAMD . compile NAMD from source By following bellow you will create NAMD in ~/anthill23_NAMD/source/NAMD… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri006&rev=1690844884&do=diff FIXME description ... software version ... sbatch example ... ... performance tests ... ... ---------- FIXME local storage All computing nodes use central NFS shared directores ( /home, /workspace ). When running computation which require frequent storage access it maybe beneficial first to copy your input data to local disk on node ( to i.e /tmp ) and running computation on local files. Bellow few comparation : mp5 compare In this test each node was occupied only by one job.… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri007&rev=1690844884&do=diff FIXME description ... software version ... sbatch example ... ... performance tests ... ... ---------- FIXME openMP OpenMP is used for parallelism within a (multi-core) node. program file MXM computes a matrix-matrix product. The source code mxm.c comes from https://people.sc.fsu.edu/~jburkardt/c_src/openmp/openmp.html . Source file is modified, matrix size was changed from 500 to 5000. < l = 500; < m = 500; < n = 500; --- > l = 5000; > m = 5000; > n = 5000; ba… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri008&rev=1690844884&do=diff FIXME description ... software version ... sbatch example ... ... performance tests ... ... ---------- FIXME software RSEM is a software package for estimating gene and isoform expression levels from RNA-Seq data. The RSEM package provides an user-friendly interface, supports threads for parallel computation of the EM algorithm, single-end and paired-end read data, quality scores, variable-length reads and RSPD estimation. In addition, it provides posterior mean and 95% credi… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri009&rev=1690844884&do=diff FIXME description ... software version ... sbatch example ... ... performance tests ... ... ---------- FIXME software Spliced Transcripts Alignment to a Reference © Alexander Dobin, 2009-2018 https://www.ncbi.nlm.nih.gov/pubmed/23104886 Software and its description is available at : https://github.com/alexdobin/STAR software instalation @anthill23 Before you can use STAR software at anthill you need to install it. Firstly login to anthill23.cent1.uw.edu.pl and open an inte… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri010&rev=1690844884&do=diff FIXME description ... software version ... sbatch example ... ... performance tests ... ... ---------- FIXME software Tools (written in C using htslib) for manipulating next-generation sequencing data. The original samtools package has been split into three separate but tightly coordinated projects: htslib: C-library for handling high-throughput sequencing data samtools: mpileup and other tools for handling SAM, BAM, CRAM bcftools: calling and other tools for han… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri011&rev=1690844884&do=diff description Multivariate Analysis of Transcript Splicing (MATS). MATS is a computational tool to detect differential alternative splicing events from RNA-Seq data. Software page : <http://rnaseq-mats.sourceforge.net/> . software version Below release of rMATS 4.0.2 ( 04/25/2018 ) is described and used. text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri012&rev=1690844884&do=diff description File <http://rmaps.cecsresearch.org/STAR/STARindex.tgz> referenced in rMATS is no longer available. So to be able to test fast examples we need to generate an index of the reference genome ourselfs <http://people.duke.edu/~ccc14/duke-hts-2018/bioinformatics/genome_prep.html> . Use bellow to install STAR software : @anthill23:~$srun -J star_compile --pty bash -l cd ~/anthill23_rmats/ wget https://github.com/alexdobin/STAR/archive/2.7.1a.tar.gz tar xzf 2.7.1a.tar.gz cd STAR-2.7.1a c… text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri013&rev=1690844884&do=diff description This article shows how to download in parallel hundreds of BAM files using Anthill cluster. It can also be used as a template when using the Anthill cluster for similar tasks . file list Assume you have file download.list that has lines location of files that you would like to download with text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri014&rev=1690844884&do=diff description ... Need to be described. - DIRAC: <http://diracprogram.org> - ReSpect: <http://www.respectprogram.org> Some details may be find in email “Re: Oprogramowanie naukowe używane na klastrach CeNTowych”. file list ... text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri015&rev=1690844884&do=diff description If you download data from NCBI, National Center for Biotechnology Information (<https://www.ncbi.nlm.nih.gov/>) you probably use prefetch command. This article shows how to use prefetch (with ascp as transport layer) in batch job. Use of ascp is advised because it is more efficent (shorter download times) than default prefetch protocol ( https ). text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri016&rev=1690844884&do=diff description AdapterRemoval searches for and removes adapter sequences from High-Throughput Sequencing (HTS) data and (optionally) trims low quality bases from the 3' end of reads following adapter removal. Software page : <https://github.com/MikkelSchubert/adapterremoval>, software usage examples <https://adapterremoval.readthedocs.io/en/latest/examples.html> . text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri017&rev=1690844884&do=diff description The preseq package is aimed at predicting the yield of distinct reads from a genomic library from an initial sequencing experiment. Software page : <https://github.com/smithlabcode/preseq>. software version Bellow description covers preseq version 2.3.1 . prepare software text/html 2023-11-07T06:33:29+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_sri018&rev=1699338809&do=diff description GLIMPSE2 is a set of tools for low-coverage whole genome sequencing imputation. GLIMPSE2 is based on the GLIMPSE model and designed for reference panels containing hundreads of thousands of reference samples, with a special focus on rare variants. text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=anthill_todos&rev=1690844884&do=diff Topic that need to be tested and described, than put to anthill_desc how to use bedtools, more... FIXME how to use NAMD, more... FIXME see NAMD ver 2.12, more... FIXME should I use nfs share or local storage ? more... FIXME openMP tests more... FIXME how to prepare and use RSEM more... FIXME how to prepare and use STAR more... FIXME how to prepare and use samtools more... FIXME how to use matching-pursuit (mp5), more... FIXME text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=hades_desc&rev=1690844884&do=diff Hades Hardware Software ---------- ---------- text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=hades_hardware&rev=1690844884&do=diff cn nodes (#25) : * cpu : 2x Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz(cache : 30 MB, Cores : 12, Turbo Frequency : 3.3GHz) (disablled HT) * gpu : none * mem : 64 GB * hdd : /home, /lustre at LUSTRE * net : Infiniband ,56Gb tcn nodes (#2) : text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=hades_soft&rev=1690844884&do=diff Available on nodes : OS * CentOS release 6.6 (Final), x86_64 SLURM * slurm 15.08.2 ICC * intel64 GCC * gcc4.8.1 * gcc4.9.3 * gcc5.3.0 * gcc8.1.0 CUDA * cuda5.5 * cuda7.5 python * python/2.7.11 * python/3.5.1 * python/3.8.2 text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=howtouseslurm_000&rev=1690844884&do=diff You can use slurm in two 'modes'. Batch mode Most computation is done with batch mode. We prepare batch script (job). This script contains declaration of resources we require, and launch of computation we would like to run on them. Then we commit this batch script with text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=howtouseslurm_001&rev=1690844884&do=diff Slurm Slurm - to system zarzadzania zasobami. Dzieki jego uzyciu nie dopuszcza sie do sytuacji kiedy wiele procesow obliczeniowych tego samego lub roznych uzytkownikow / uzytkowniczek probuje uzyc tych samych zasobow ( rdzenie, pamiec, dysk, ... ) co moze doprowadzic do znacznego wydluzenia czasu ich wykonania, lub wrecz ich przerwania. Inne podobne programy to LSF, PBS/Torque, SGE ich porownanie mozna znalezc [tu.] text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=howtouseslurm_002&rev=1690844884&do=diff Nodes can have features assigned to them by the Slurm administrator. Users can specify which of these features are required by their job using the constraint option. Only nodes having features matching the job constraints will be used to satisfy the request. text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=howtouseslurm_003&rev=1690844884&do=diff Check number of CPUs reserved by jobs squeue -o"%.7i %.9P %.8j %.8u %.2t %.10M %.6D %C" See additional info ( mem, Socket/Cores/Threats, tmp_disk, ..) of nodes sinfo -p test -l --Node text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=howtouseslurm_004&rev=1690844884&do=diff Using dependencies in Slurm Slurm has a fairly robust set of dependencies you can use. These are set when you submit the job and can be used for setting up pipelines. A job can depend on more than one other job as well. To use dependencies, submit the job with the following switch. If using multiple dependency types, they should be comma seperated: text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=howtouseslurm&rev=1690844884&do=diff SLURM, batch and interactive mode SLURM, basic concepts SLURM, how to use constraints SLURM, how to build pipelines SLURM, good to know SLURM Documentation text/html 2024-07-18T11:39:10+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=jupyterhub_desc&rev=1721302750&do=diff TLJH was pernamently shutdown, service is not available. Jupyter Hub (TLJH) The Littlest JupyterHub (TLJH) simple JupyterHub distribution for a small (0-100) number of users on a single server (<https://tljh.jupyter.org/en/latest/index.html>). access / login Jupyter Hub (TLJH) is available only from internal CeNT networks ( or with active CeNT VPN connection ). text/html 2024-07-18T06:12:47+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=news_arch&rev=1721283167&do=diff News archives 2023 2023.03.10 : * With 10th April 2023 TCR cluster will be permanently turned off. 2022 2022.09.22 : * Anthill's computing node ant101 was upgraded. 2022.09.21 * Power maintenance in CeNT has finished. Anthill is online . Hades is text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=obsolete&rev=1690844884&do=diff ---------- TCR ver 0.1 old resources : TCR webpage TCR wiki ---------- ---------- ---------- ---------- ---------- text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=sidebar&rev=1690844884&do=diff * item 1 * item 2 * item 3 text/html 2024-07-18T11:46:18+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=start&rev=1721303178&do=diff This site describe Computing Resources available in CeNT. ---------- News 2024.07.18 : * Anthill's computing nodes ant101 - ant105 were updated, now each has 384 GB RAM. 2024.07.12 : * new computing nodes : ant104, ant105 in Anthill. more... ---------- SLURM How to use SLURM (f.k.a Simple Linux Utiliy for Resource Management) ---------- text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=tcr_desc&rev=1690844884&do=diff TCR ver 0.2 Hardware Software ---------- Before sending request to install new software on computing nodes : * use slurm interactive session and check ( it is probably already installed ) * if it is not installed yet, but is is not available in standard repositories consider installing it from source in user space. You can find some examples how to do it bellow. text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=tcr_hardware&rev=1690844884&do=diff x86_64 nodes : cnodesNNN, gnodesNNN, mnodesNNN * CPU: 2x Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz (all) * MEM: 64GB1 (cnodes/gnodes) * MEM: 512GB2 (mnodes) * GPU: Tesla K20m ( gnodes ) * NET: ethernet 1Gb, infiniband3 (all) power7 nodes : power001 text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=tcr_soft&rev=1690844884&do=diff Available on nodes : OS * centOS 7 x86_64/ppc SLURM * slurm 19.05.5 GCC * gcc : 4.8.5 (default) * gcc : 5.5.0 * gcc : 9.3.0 * gcc : 10.4.0 * gcc : 11.3.0 clang * clang : 3.4.2 python * python 2.7.5 * python 3.6.8 NVIDIA (gnodes) text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=tcr_sri001&rev=1690844884&do=diff FIX ME FIX ME FIX ME GROMACS 2020.1 description GROningen MAchine for Chemical Simulations (GROMACS) is a molecular dynamics package mainly designed for simulations of proteins, lipids, and nucleic acids. Software page : <http://www.gromacs.org/> . software version GROMACS 2020.1, text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=tcr_sri002&rev=1690844884&do=diff Quantum Espresso 6.5 description Quantum ESPRESSO is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials. ( text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=tcr_sri003&rev=1690844884&do=diff LAMMPS version 29 Oct 20 description LAMMPS is a classical molecular dynamics code with a focus on materials modeling. It's an acronym for Large-scale Atomic/Molecular Massively Parallel Simulator. ( <https://lammps.sandia.gov/index.html> ) . software version LAMMPS version 29Oct20, text/html 2023-07-31T23:08:04+00:00 Anonymous (anonymous@undisclosed.example.com) https://crwiki.cent.uw.edu.pl/doku.php?id=tcr_sri004&rev=1690844884&do=diff Quantum Espresso 7.0 description Quantum ESPRESSO is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials. (