iRODS with Ansible

The story behind : My group ask me if I can do a Ansible install for iRODS.

But what is Ansible ? 
Ansible is an open source, powerful automation software for configuring, managing and deploying software applications on the nodes without any downtime just by using SSH. Today, most of the IT Automation tools runs as a agent in remote host, but ansible just need a SSH connection and Python (2.4 or later) to be installed on the remote nodes to perform it’s action.

Prerequisites
Operating System: RHEL/CentOS/Fedora and Ubuntu/Debian/Linux Mint
Jinja2: A modern, fast and easy to use stand-alone template engine for Python.
PyYAML: A YAML parser and emitter for the Python programming language.
parmiko: A native Python SSHv2 channel library.
httplib2: A comprehensive HTTP client library.
sshpass: A non-interactive ssh password authentication.

Source : http://www.tecmint.com/install-and-configure-ansible-automation-tool-in-linux/

My quick install (centOS) :

sudo rpm -iUvh http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-8.noarch.rpm

yum -y install ansible

####Make the trust###

ssh-keygen -t rsa -b 4096 -C "root@192.168.xxx.147"

ssh-copy-id root@192.168.xxx.148

ssh-copy-id root@192.168.xxx.149

### please note IP is the MAIN note IP###

To do the automate installation we use Ansible from www.ansible.com , Ansible-Tower and some ideas from EUDATs Github.  On a Redhat/Centos 7.x.  

Please note : You can easy re-use the shellscript with Ansible because it only need a ssh-login.

My RPMbuilder and installer: 

https://raw.githubusercontent.com/sunadtudk/irods_2_figshare/master/Irods-install.sh
Update : Our Ansible script is online at : https://github.com/sunadtudk/ansible_irods
Shell script and help : https://github.com/EUDAT-B2STAGE/docker-images

We just change the playbook from : https://github.com/aabdulwahed/iRODS/

So... do a clone and "hack" your own version ;-)

More IRODS :
http://bigdatadk.blogspot.dk/2016/08/update-irods-from-41x-to-42-with-yum.html
http://bigdatadk.blogspot.dk/2016/08/building-irods-419-from-source-on.html

Tnx for Leaked Mirai Source Code Jerry Gamblin!

"Source code of Mirai botnet responsible for Krebs On Security DDoS released online
Now anyone can use the IoT-based botnet for their own destructive purposes."
Source :
http://www.zdnet.com/article/source-code-of-mirai-botnet-responsible-for-krebs-on-security-ddos-released-online/

Some days ago I got a tweet from a friend about the source code to a IoT BotNet.
The source can be found on :   https://github.com/jgamblin/Mirai-Source-Code
Backup : http://pastebin.com/raw/8bgjhYQa

The files real file name (google for links!) :
loader.src.zip 
dlr.src.zip  
mirai.src.zip

     Pix. Source : http://blog.malwaremustdie.org/2016/08/mmd-0056-2016-linuxmirai-just.html

Read the full text :
https://raw.githubusercontent.com/jgamblin/Mirai-Source-Code/master/ForumPost.txt

Configuring CNC
Code: apt-get install mysql-server mysql-client
CNC requires database to work. When you install database, go into it and run following commands:
http://pastebin.com/86d0iL9g

This will create database for you. To add your user,
Code: INSERT INTO users VALUES (NULL, 'anna-senpai', 'myawesomepassword', 0, 0, 0, 0, -1, 1, 30, '');
Now, go into file ./mirai/cnc/main.go
Edit these values
Code:
const DatabaseAddr string   = "127.0.0.1"
const DatabaseUser string   = "root"
const DatabasePass string   = "password"
const DatabaseTable string  = "mirai"

To the information for the mysql server you just installed

Setting Up Cross Compilers
Cross compilers are easy, follow the instructions at this link to set up. You must restart your system or reload .bashrc file for these changes to take effect. http://pastebin.com/1rRCc3aD

Source : http://hackforums.net/showthread.php?tid=5420472

Thx for a nice RedHat Forum

Thx for a really great RedHat Forum In Copenhagen @Docken.

https://www.redhat.com/en/about/events/red-hat-forum-denmark-2016

Lets do it agian nex year ;-) 

                                   
                                              Plz. note  Microsoft.... at Redhat Forum!

Microsoft Ignite.. as a sofa scener :-(

September 26–30, 2016, Atlanta, GA  : https://ignite.microsoft.com 

Twitter : 

https://twitter.com/MS_Ignite  

https://twitter.com/hashtag/MSIgnite?src=hash

Alot of live streaming... and stuff ;-)

Donation €4,000,000.00.... That's nice - if ever true ;-)

What a great story!!.....   just saying : don't bullshit a bullshitter ;-)

send from mellisa lewis : xxxx@hotmail.com   (one hotmail and one Yahoo???  Come on!!!)

My Donation €4,000,000.00
My Dear

With warm hearts I offer my friendship, and my greetings to you in the name of our lord God, and I hope this letter meets you in good time, I propose with my free mind and as a person of integrity from God, I know that this message will appear as a surprise to you that we barely know but the grace of God directed me to you and I wish you read this message and be blessed in name of the Lord.

I have a brain tumor, I suffer terribly at the moment. My doctor just informed me that my days are numbered because of my health therefore condemned to certain death. Currently, I have exhausted all my savings for my medical care. But I do have some funds for my charity project, these funds are deposited in a fixed deposit in a local bank here in Benin. purposed for charitable foundation My marital status is such that I'm single because I lost my Husband for over 9 years now and unfortunately we have not had a child together, which I am no one to leave my legacy . Therefore, to release my funds I would like to make a donation so that there is no stiff tax on my money.

To this I would so graceful and in order to help the poor to give what amounts to said legacy worth Four million euros (€4,000,000.00) (Four Million Euro) to enable you to establish a charitable foundation in my memory so that the grace of God be with me until my last home so I can receive an honorable place with the Lord our father. I have no fear because before contacted you, i have for several nights prayed for the Lord God to give me the contact of a trusted person to whom I can entrust this matter and it is the result of why I am doing research that allowed me to contact you through this site.

Know that you can keep half of the money for yourself and the rest will be used to create a charitable foundation in my memory and a federation in the fight against cancer and also build orphanages. I would have the following information: Your name and first name, your exact address and your permanent telephone contact so I can forward them to my lawyer to appear that you perform the procedure under deal. I count on your goodwill and especially on the proper use of these funds have something I do not doubt because I have great confidence in you that God may guide me toward you. My private E-mail: xxxxxxxxx@yahoo.com

Awaiting your prompt reply, receive my cordial and fraternal greetings.

Yours Sincerely
Mrs Mellisa Lewis
My Private email: xxxxxx@yahoo.com

Update iRODS from 4.1.x to a 4.2 with YUM (Repo)

There was a note on the google chat for iRODS fans about updating to 4.2 via YUM,Zypper or APT.

-----

renci-irods-unstable/7/x86_64/signature                  |  819 B     00:00

Retrieving key from https://unstable.irods.org/irods-unstable-signing-key.asc

Importing GPG key 0x9086C490:

 Userid     : "iRODS Consortium - Unstable <info@irods.org>"

 Fingerprint: 5909 3cb7 0f18 8588 25ac 778f 5054 912c 9086 c490

 From       : https://unstable.irods.org/irods-unstable-signing-key.asc

Is this ok [y/N]: y

----Just say Y

Dependencies Resolved

================================================================================

 Package                            Arch   Version   Repository            Size

================================================================================

Installing:

 irods-icommands                    x86_64 4.2.0-1   renci-irods-unstable 313 k

     replacing  irods-icat.x86_64 4.1.9-0

 irods-server                       x86_64 4.2.0-1   renci-irods-unstable 1.7 M

     replacing  irods-icat.x86_64 4.1.9-0

Updating:

 irods-database-plugin-postgres     x86_64 4.2.0-1   renci-irods-unstable 895 k

 irods-runtime                      x86_64 4.2.0-1   renci-irods-unstable 3.3 M

Installing for dependencies:

 irods-externals-avro1.7.7-0        x86_64 1.0-1     renci-irods-unstable 1.5 M

 irods-externals-boost1.60.0-0      x86_64 1.0-1     renci-irods-unstable  13 M

 irods-externals-clang-runtime3.8-0 x86_64 1.0-1     renci-irods-unstable 347 k

 irods-externals-jansson2.7-0       x86_64 1.0-1     renci-irods-unstable 159 k

 irods-externals-libarchive3.1.2-0  x86_64 1.0-1     renci-irods-unstable 1.4 M

 irods-externals-zeromq4-14.1.3-0   x86_64 1.0-1     renci-irods-unstable 549 k

Transaction Summary

================================================================================

Install  2 Packages (+6 Dependent packages)

Upgrade  2 Packages

Total download size: 23 M

Is this ok [y/d/N]: Y

-----------------------

The Note says:

This server hosts both APT and YUM repositories for the iRODS Consortium.

Installation

Add this server to your package manager's list of repositories to check for new software.

APT

Install the public key and add the repository:

wget -qO - https://unstable.irods.org/irods-unstable-signing-key.asc 

| sudo apt-key add -
echo "deb [arch=amd64] https://unstable.irods.org/apt/ $(lsb_release -sc) 

main" | sudo tee /etc/apt/sources.list.d/renci-irods-unstable.list
sudo apt-get update

YUM

Install the public key and add the repository:

sudo rpm --import https://unstable.irods.org/irods-unstable-signing-key.asc
wget -qO - https://unstable.irods.org/renci-irods-unstable.yum.repo 

| sudo tee /etc/yum.repos.d/renci-irods-unstable.yum.repo

ZYPPER

Install the public key and add the repository:

sudo rpm --import https://unstable.irods.org/irods-unstable-signing-key.asc
wget -qO - https://unstable.irods.org/renci-irods-unstable.zypp.repo 

| sudo tee /etc/zypp/repos.d/renci-irods-unstable.zypp.repo

Source : https://groups.google.com/forum/#!topic/irod-chat/5cg24SHQ79w

Doing a FreeIPA for the server

What is FreeIPA : https://www.freeipa.org/page/Main_Page

Turn off SeLinux :    setenforce 0

in the file  :  nano /etc/sysconfig/selinux
SELINUX=permissive
#SELINUX=enforcing

Install with :  
yum -y install ipa-server bind-dyndb-ldap ipa-server-dns
ipa-server-install --setup-dns

The log file for this installation can be found in /var/log/ipaserver-install.log
==============================================================================
This program will set up the IPA Server.

This includes:
  * Configure a stand-alone CA (dogtag) for certificate management
  * Configure the Network Time Daemon (ntpd)
  * Create and configure an instance of Directory Server
  * Create and configure a Kerberos Key Distribution Center (KDC)
  * Configure Apache (httpd)
  * Configure DNS (bind)

To accept the default shown in brackets, press the Enter key.

Enter the fully qualified domain name of the computer
on which you're setting up server software. Using the form
<hostname>.<domainname>
Example: master.example.com.

Server host name [labipa.example.com]:

Please see ; https://www.youtube.com/watch?v=8wc4MO3LXQI

Please note :  If you get the error : "Your system is running out of entropy"
Please install haveged [ http://www.issihosts.com/haveged/ ] or just wait some time.... ;-)

Look at : http://serverfault.com/questions/406038/server-suddenly-running-out-of-entropy

I need to change my passwd on a .pl site?

We got Spammed agian.... with a "we need your password on a .pl site" 

Come on guys!!  It's Amateur Night Agian.....

-------------------------------------------The Mail---------------

Kære bruger ,

Vi har bemærket nogle usædvanlige aktiviteter i din postkasse , og det er midlertidigt låst. Vær venlig at validere din e-mail- adresse ved hjælp af linket nedenfor for at gøre os i stand genaktivere din e-mail- konto . Hvis dette ikke gøres , vil føre til permanent blokering , og du vil være i stand du sende og modtage post .

Tak for dit samarbejde

Hilsen,

Help Desk

Copyright © 2016 Tekniske Universitet, All Rights Reserved.

ENGLISH 

Dear user,

We have noticed some unusual activities in your mail box and it has been temporarily locked. Please kindly validate your email address using the link below to enable us reactivate your email account. Failure to do this will lead to permanent blocking and you will be unable you send and receive mail.

Thank you for your cooperation

Regards,

Help Desk

Copyright © 2016 Tekniske Universitet, All Rights Reserved.

The site the (removed) link pointed at :

Getting NodeJS 6.x.x to run on CentOS7

On one of the servers we got an error running the new NodeJS script.

How was it installed ?? yum install nodejs -y

[root@server1 ~]# node -v
v0.10.42
-- How to fix???
[root@server1 ~]# rpm -qa |grep node
---cut----------
nodejs-inherits-2.0.0-4.el7.noarch
nodejs-fstream-0.1.24-1.el7.noarch
nodejs-slide-1.1.5-1.el7.noarch
nodejs-tar-0.1.18-1.el7.noarch
nodejs-abbrev-1.0.4-6.el7.noarch
nodejs-mime-1.2.11-1.el7.noarch
nodejs-cryptiles-0.2.2-1.el7.noarch
nodejs-cmd-shim-1.1.0-3.el7.noarch
nodejs-delayed-stream-0.0.5-5.el7.noarch
nodejs-read-1.0.5-1.el7.noarch
nodejs-npmconf-0.1.3-1.el7.noarch
nodejs-ctype-0.5.3-3.el7.noarch
nodejs-json-stringify-safe-5.0.0-1.el7.noarch
nodejs-normalize-package-data-0.2.1-1.el7.noarch
nodejs-minimatch-0.2.12-2.el7.noarch
nodejs-read-installed-0.2.4-1.el7.noarch
nodejs-asn1-0.1.11-3.el7.noarch

---cut----------

rpm -e nodejs-inherits-2.0.0-4.el7.noarch

error: Failed dependencies:

        npm(inherits) >= 2.0.0 is needed by (installed) nodejs-fstream-0.1.24-1.el7.noarch

        npm(inherits) < 2.1 is needed by (installed) nodejs-fstream-0.1.24-1.el7.noarch

        npm(inherits) >= 2.0.0 is needed by (installed) nodejs-block-stream-0.0.7-1.el7.noarch

        npm(inherits) < 2.1 is needed by (installed) nodejs-block-stream-0.0.7-1.el7.noarch

        npm(inherits) >= 2 is needed by (installed) nodejs-tar-0.1.18-1.el7.noarch

---cut----------

 Uninstall all NodeJS and fix the error....

[root@server1 ~]#  rpm -qa |grep node >  awk 'BEGIN{ORS=","}1' > node.txt

Please node : Node.txt is a  full list of NodeJS files...   

Just add "rpm -e " at the beginning of the file and it will uninstall all NodeJS files.

[root@server1 ~]#  echo 'rpm -e' | cat - node.txt | tee node.txt  

-and run your uninstall bash script with  sh ./node.txt

Then do: 

curl --silent --location https://rpm.nodesource.com/setup_6.x | bash - ; yum -y install nodejs ; yum install gcc-c++ make

[root@server1 ~]# node -v

v6.3.1

Get Julia to run on CentOS 7

What is Julia???
"Julia is a high-level, high-performance dynamic programming language for technical computing, with syntax that is familiar to users of other technical computing environments. " 

Info about the language : https://julialang.org/

Install Info from : https://copr.fedorainfracloud.org/coprs/nalimilan/julia/
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -ivh epel-release-latest-7.noarch.rpm

cd /etc/yum.repos.d

wget https://copr.fedorainfracloud.org/coprs/nalimilan/julia/repo/epel-7/nalimilan-julia-epel-7.repo


The Repo file :

[nalimilan-julia]
name=Copr repo for julia owned by nalimilan
baseurl=https://copr-be.cloud.fedoraproject.org/results/nalimilan/julia/epel-7-$basearch/
skip_if_unavailable=True
gpgcheck=1
gpgkey=https://copr-be.cloud.fedoraproject.org/results/nalimilan/julia/pubkey.gpg
enabled=1
enabled_metadata=1

Or you can :  (Source : http://julialang.org/downloads/platform.html  )

sudo dnf copr enable nalimilan/julia
or
sudo dnf copr enable nalimilan/julia-nightlies

yum -y install julia

yum -y install mc nano vim libXp openmotif openmotif22 python.x86_64 numpy* ncurses-devel gcc gcc-c++ autoconf automake; yum -y groupinstall "Development Tools"

julia 

That was easy ;-)  Next comes the cluster setup ....

Building iRODS 4.1.9 from source on CentOS7

##  Install from a .gz file ( ftp://ftp.renci.org/pub/irods/releases/4.1.9/  )
yum -y install gcc-c++ make python-devel rpm-build help2man unixODBC fuse-devel curl-devel
yum -y install --enablerepo=centosplus openssl-devel pam-devel bzip2-devel zlib-devel pam-devel openssl-devel libxml2-devel perl-JSON
yum -y install rsync unixODBC-devel epel-release net-tools wget nano mc mlocate ImageMagick python-devel rpm-build help2man
yum -y install bzip2-devel libxml2-devel unixODBC pam-devel openssl-devel wget nano mc tomcat tomcat-webappsl tomcat-admin-webapps
yum -y install tomcat-docs-webapp tomcat-javadoc net-tools postgresql postgresql-serverl postgresql-contrib postgresql-libs
yum -y install postgresql-odbc fuse-libs.x86_64 perl-JSON* python-requests.noarch python-jsonschema python-psutil lsof
yum -y install authd postgresql postgresql-libs postgresql-plperl postgresql-plpython postgresql-server python-jsonschema python-psutil python-requests postgresql-odbc
###NodeJS 6.x install

curl --silent --location https://rpm.nodesource.com/setup_6.x | bash - ; yum -y install nodejs ; yum install gcc-c++ make

yum -y groupinstall "Development Tools"
yum -y install python-pip git gettext-devel openssl-devel perl-CPAN perl-devel zlib-devel
##############
git clone https://github.com/DICE-UNC/irods-cloud-browser.git
git clone https://github.com/sunadtudk/irods_2_figshare.git
##############
pip install git+git://github.com/iPlantCollaborativeOpenSource/pycommands.git
pip install --upgrade pip
pip install --upgrade git+git://github.com/irods/python-irodsclient.git
pip install pigshare
#pip install --upgrade pigshare
###############################

#############Build iRODS server###############
wget ftp://ftp.renci.org/pub/irods/releases/4.1.9/irods-4.1.9.tar.gz
tar -zxvf irods-4.1.9.tar.gz
cd irods-4.1.9/packaging/
./build.sh  icat postgres
./build.sh -s resource
./build.sh -s icommands
###############Install###############
cd ~/irods-4.1.9/build/
rpm -ivh irods-icat-4.1.9-64bit-centos7.rpm
rpm -ivh irods-database-plugin-postgres-1.9-centos7.rpm
rpm -ivh irods-dev-4.1.9-64bit-centos7.rpm
rpm -ivh irods-runtime-4.1.9-64bit-centos7.rpm
#rpm -ivh irods-database-plugin-postgres93-1.9-centos7.rpm
#rpm -ivh irods-resource-4.1.9-64bit-centos7.rpm
postgresql-setup initdb
systemctl start postgresql
systemctl enable postgresql
######config iRODS server#######
sudo /var/lib/irods/packaging/setup_irods.sh

# rpm -ivh irods-icommands-4.1.9-64bit-centos7.rpm

Cracking my Technicolor TG788vn open

A look inside my Technicolor TG788vn.

With a RaspBerry  PI you can do funny things ;-)

YES!!! I just won a lot of money ;-)

I Got this mail some days ago : 

Attn:

Congratulation!!!

Your name and your e-mail address has won £2,050,000GBP in the VW Motors Email Lottery.

Your winning No. is (VJETTA-UK-VP990003-M4GA).

Contact (Mr. Elvis Bowie) elvis.bowie@yahoo.com.hk. Tel.: +44-703-590-5301 immediately; and forward your winning No.

(VJETTA-UK-VP990003-M4GA) to him to claim your prize.

Congratulation once again.

Thank you. Patricia Wesley.

--------------

Danm....

First clue... (a google search...)

The Winning no. is already in use....  http://www.419scam.org/emails/2015-10/01/00981949.473.htm

From: "Mrs. Patricia Wesley " <pat.vwltt@gmail.com>
Reply-To: info.sipclaims@aol.com
Date: Thu, 01 Oct 2015 10:53:55 +0800

Second clue...

A YhaooMail in Hongkong used by VM motor? Really guys ?

Come on guys!!  It's Amateur Night Agian.....

Get CentOS 7 on FUJITSU PRIMERGY RX300

I just got 8 retired FUJITSU  PRIMERGY RX300 with 196 Gb of RAM and 256GB SSD.

My Idea was run a MatLab Cluster on them,
for that I will use Ansible. info on how to install :
http://www.tecmint.com/install-and-configure-ansible-automation-tool-in-linux/

But first updates and the drivers...

Updates...
Just find your server and download the DVD : http://www.fujitsu.com/fts/support/
And boot from the iRMCA interface... "FUJITSU ServerView® iRMC S2 Web Server"

Then find the drivers and Repo...  from http://patches.ts.fujitsu.com/linux/pldp/

Looks like this on CentOS 7 ( edit : /etc/yum.repos.d/  )

# Save this file under /etc/yum.repos.d to enable the yum repository

[PRIMERGY_Drivers_RHEL7_rhel7]
name=PRIMERGY_Drivers_RHEL7_rhel7
baseurl=http://patches.ts.fujitsu.com/linux/pldp/RHEL7/rhel7
enabled=1
gpgcheck=1
gpgkey=http://patches.ts.fujitsu.com/linux/pldp/gpg-pubkey-79444536-46837bd0.asc

# Save this file under /etc/yum.repos.d to enable the yum repository

[PRIMERGY_Drivers_RHEL7_rhel7-u1]
name=PRIMERGY_Drivers_RHEL7_rhel7-u1
baseurl=http://patches.ts.fujitsu.com/linux/pldp/RHEL7/rhel7-u1
enabled=1
gpgcheck=1
gpgkey=http://patches.ts.fujitsu.com/linux/pldp/gpg-pubkey-79444536-46837bd0.asc

# Save this file under /etc/yum.repos.d to enable the yum repository [PRIMERGY_Drivers_RHEL7_rhel7-u2] name=PRIMERGY_Drivers_RHEL7_rhel7-u2 baseurl=http://patches.ts.fujitsu.com/linux/pldp/RHEL7/rhel7-u2 enabled=1 gpgcheck=1 gpgkey=http://patches.ts.fujitsu.com/linux/pldp/gpg-pubkey-79444536-46837bd0.asc

Then just do :  yum update ; yum install prime*

Setting up a MatLab cluster...

I just got 8 retired VMware server with 196 Gb of RAM and 256GB SSD.

And I was wondering if the servers can be re-used as a MatLab cluster...

More info about MatLab setup : 

http://se.mathworks.com/products/parallel-computing/index.html

The Design looks like this :

The pdf file :
http://se.mathworks.com/support/product/DM/installation/ver_current/Files/mdcs-mjs-quickstart-guide.pdf

The pdf file :  http://se.mathworks.com/help/pdf_doc/mdce/mdce.pdf


The Admin center...

Error 37 and 11 on a MYSQL (CentOS) after we moved data to nfs.

InnoDB: Unable to lock ./ibdata1, error: 11

Solution: 

1) killed the lingering mysql process

2) mv (move) ibdata1 file to ibdata1.bad

3) cp -a ibdata1.bad ibdata1

4) restart the db server or just type: reboot

InnoDB: Unable to lock ./ibdata1, error: 37

We faced the “error  InnoDB: Unable to lock ./ibdata1, error: 37 “ while moving my data directory from a SAN to a NFS on a local disk , when we restarted the mysql , the restart failed and we got the error in the error log , all permissions are set properly.

Solution: 

1) Add nolock option into nfs mount entry in /etc/fstab and just type : reboot

-optional-
2) /etc/init.d/portmap restart

3) /etc/init.d/nfslock restart

Audit user passwords : John the Ripper Max Speed with more cores!

I was asked how to audit users passwords on our HPC cluster...
well... crack the password with John the ripper from http://openwall.com/john/

Please note : 
More tools can be found on : http://www.darknet.org.uk/

More about using wordlists : 
http://blog.thireus.com/look-back-on-2012s-famous-password-hash-leaks-wordlist-analysis-and-new-cracking-techniques/

The Wordlist:
https://wiki.skullsecurity.org/Passwords
http://www.openwall.com/passwords/wordlists/

first I did :  yum install john
and run : john --test

                        As you see here it runs on 1 core....  But I have 4 cores.... No Cuda :-(

After reading abit about the multicore options...
please note!  There is alot of info on : http://openwall.info/wiki/john/parallelization

I did:
1. Grabed latest patched jumbo version that includes lots of goodies:

$ wget http://openwall.com/john/g/john-1.7.9-jumbo-7.tar.bz2
2. Extract

$ tar xvf john-1.7.9-jumbo-7.tar.bz2
3. Change to the /src directory

$ cd john-1.7.9-jumbo-7/src/
4. Edit the Makefile to enable multi-threaded capabilities.

$ nano Makefile
Uncomment these lines as follows:

OMPFLAGS =
# gcc with OpenMP
OMPFLAGS = -fopenmp
OMPFLAGS = -fopenmp -msse2

5. For this next step, you will want to specify the target system. To see available targets, type
$ make linux-x86-64-native

The options are:
linux-x86-64-native      Linux, x86-64 'native' (all CPU features you've got)
linux-x86-64-gpu         Linux, x86-64 'native', CUDA and OpenCL (experimental)
linux-x86-64-opencl      Linux, x86-64 'native', OpenCL (experimental)
linux-x86-64-cuda        Linux, x86-64 'native', CUDA (experimental)
linux-x86-64-avx         Linux, x86-64 with AVX (2011+ Intel CPUs)
linux-x86-64-xop         Linux, x86-64 with AVX and XOP (2011+ AMD CPUs)
linux-x86-64[i]          Linux, x86-64 with SSE2 (most common)
linux-x86-64-icc         Linux, x86-64 compiled with icc
linux-x86-64-clang       Linux, x86-64 compiled with clang
linux-x86-gpu            Linux, x86 32-bit with SSE2, CUDA and OpenCL (experimental)
linux-x86-opencl         Linux, x86 32-bit with SSE2 and OpenCL (experimental)
linux-x86-cuda           Linux, x86 32-bit with SSE2 and CUDA (experimental)
linux-x86-sse2[i]        Linux, x86 32-bit with SSE2 (most common, 32-bit)
linux-x86-native         Linux, x86 32-bit, with all CPU features you've got (not necessarily best)
linux-x86-mmx            Linux, x86 32-bit with MMX (for old computers)
linux-x86-any            Linux, x86 32-bit (for truly ancient computers)
linux-x86-clang          Linux, x86 32-bit with SSE2, compiled with clang

...and more....

6. When complete, binaries are available under the /run directory.

$ cd ../run
To run a benchmark on your system, type:

$ ./john --test
Cracking MD5 via brute force is now done 24 x faster!

I did run : cd ../run/ ; john test.txt   (test.txt is a md5 username + password)

Loaded 1 password hash (FreeBSD MD5 [128/128 AVX intrinsics 12x])
guesses: 0  time: 0:00:04:57 0.00% (3)  c/s: 633654  trying: mikmpit1 - mikmpl91

Bluemix and BigData at Roskilde Festival

When one of Denmark's smallest cities erupts into one of the largest as part of one of the world's most eminent music festivals, how does the town adapt? IBM, Roskilde Music Festival and The Copenhagen Business School (CBS) came together to gather and understand terabytes of data generated by the event to help make it one of the most efficient, sustainable, and enjoyable festivals in the world.

Ten days of infinite unknowns

The Roskilde Festival is "the perfect microcosm of a complex modern city." It generates immense amounts of data, both structured and unstructured, generated from geospatial, social media and a multitude of other sources. 

Enter the data laboratory

To help gather, process, store, and analyze the Festival's data:

• The IBM Bluemix platform on the SoftLayer infrastructure—a cloud-based analytics environment—provided the backbone.
• dashDB, a managed data warehousing platform, stored the data.
• SPSS, an intuitive visual modeler, cleaned and merged the data.
• SoftLayer’s AutoScale service automatically scaled the data.
• IBM Watson Analytics offered real-time analytics for non-technical users.

A gateway of geospatial data

Festival goers opted-in via more than 60,000 smartphones to share information about their activities in real-time, and the behavior of the crowds was visualized in polygons and color-coded, time-lapsed animations—91 million rows of data was used to create 60 seconds of time-lapse.

Data in motion

Information took shape: The sale of food and average over time, the popularity of the various artists, the usage of bathroom facilities and parking, and so on created a comprehensive view of what was happening at the festival.

Real-time, operational decisions

By making this wealth of data accessible to non-data scientists, the Festival gained insights enabling better operational decisions that will minimize food waste, improve security, and optimize the festival experience.

"For one week each year, a small town called Roskilde on the outskirts of Copenhagen temporarily becomes Denmark’s fourth largest city. The renowned Roskilde Festival attracts 130,000 visitors annually, who camp out and attend more than 170 live music and cultural performances. During the ten-day festival, the crowds buy and consume approximately 200 tons of food and generate 300 tons of waste. There is growing international recognition that mankind needs to make drastic changes to meet the challenges we face around climate change, a point underscored at the 2012 “Rio+20” United Nations conference on sustainable development. Inspired by this conference, researchers at the Copenhagen Business School (CBS) wanted to do more than just talk about sustainability.  "

Source : http://www.ibmbigdatahub.com/infographic/cloud-data-rocks-out-roskilde-festival

Never Win10... Want Linux or Win7

How existing Windows users can refuse Microsoft's Windows 10 update:
http://www.zdnet.com/article/how-existing-windows-users-can-refuse-microsofts-windows-10-update/

"If you don't want Windows 10 to install automatically on your Windows 7/8 PC or tablet, for whatever reason, here's what to remember":

Note!  Grc.com got a tool to turn off Win10:   https://www.grc.com/never10.htm

The irods to FigShare design

The idea : 

Copy files from iRODS to Figshare.

To make it easy for the user(s) this will be done via a Web-GUI,

where there the user will select the file and right click to get a file-menu.

           The code from : https://github.com/sunadtudk/irods_2_figshare
Please note!!  Update the api.py from https://github.com/makkus/pigshare

# please note!  
# please do before playing with figshare: 
# yum update ; yum -y install python-pip
# pip install git+git://github.com/iPlantCollaborativeOpenSource/pycommands.git
# pip install --upgrade pip
# pip install --upgrade git+git://github.com/irods/python-irodsclient.git
# pip install pigshare
#  Do a "pigshare -h"  to see the API ;-)
# irods_2_figshare
# By Sune Andersen
# version 0.1b   
import fileinput
import requests
import os
from irods import *
from irods.session import iRODSSession
from irods.models import Collection, User, DataObject
sess = iRODSSession(host='localhost', port=1247, user='irodsuser', password='XxxxxxxA', zone='MyZone')
coll = sess.collections.get("/MyZone/home/irodsuser")
fileout = open('figshare_test.txt', 'w')
objf = sess.data_objects.get("/MyZone/home/irodsuser/figshare_test.txt")
with objf.open('r+') as f:
   for line in f:
     fileout.write(line)
#   Test : 
#BASH #   pigshare create_article --article '{"title": "irods test", "custom_fields": {"key1": "from_irods"}}'
#{
# "location": "https://api.figshare.com/v2/account/articles/3393214"    <----  This number is the article ID for upload
#}
#BASH #    pigshare upload_new_file --id 3393214 figshare_test.txt

Today kick off : eScience @ DTU - Pictures...

Today kick off : eScience @ DTU.dk - Some pictures

Really? a new login on a Outlook in Polen?

Oh!!.....  I need to update my login on a page in poland... Most be armature night agian...

The mail :

Kære bruger ,

På grund af den igangværende ændring i vores login side , vil du blive brug for at validere din konto login , så vi integrere din konto til den nye login-siden . 

Klik på Single Sign -On linket nedenfor for at validere din konto ved at logge ind :

http://xxxxxx/mail.win.dtu.dk/owa/auth/

BEMÆRK : ENTER dit brugernavn og password , som forkert brugernavn og adgangskode VIL AUTOMATISK deaktivere din konto . Undladelse af at gøre dette vil resultere i permanent begrænset adgang til din postkasse .

Tak fordi du valgte DTU .

Hilsen,

DTU Webmail verifikationstjenester

DTU ITS Help Desk | Copyright © 2016 Danmarks Tekniske Universitet . Alle rettigheder forbeholdes.

The site looks like this:

ImageMagic Bug... Update your iRODS - CVE-2016–3714

There are multiple vulnerabilities in ImageMagick, a package commonly used by web services to process images. One of the vulnerabilities can lead to remote code execution (RCE) if you process user submitted images. The exploit for this vulnerability is being used in the wild.

If you use ImageMagick or an affected library, we recommend you mitigate the known vulnerabilities by doing at least one of these two things (but preferably both!):

• Verify that all image files begin with the expected “magic bytes” corresponding to the image file types you support before sending them to ImageMagick for processing. (see FAQ for more info)
• Use a policy file to disable the vulnerable ImageMagick coders. The global policy for ImageMagick is usually found in “/etc/ImageMagick”. The below policy.xml example will disable the coders EPHEMERAL, URL, MVG, and MSL

Source :  http://www.jupiterbroadcasting.com/99566/getting-inside-the-oryx-pro-las-416/          

Info : https://imagetragick.com/

PoC : https://www.exploit-db.com/exploits/39767/

HD crash on NAS and NASA is using iRODS!!!

Crap!! ... the NAS where 11 TB data is living on crashed on one disk. Lucky we are running Raid 6!!
got a new disk and let it rebuild the Raid pool.

For Real NASA ?? Just got in contact with a guy working at NASA, They use iRODS too!

Proteomics is an area in which the EDRN Informatics Center is investing resources to provide processing pipelines of proteomics datasets. This is an effort in collaboration with Vanderbilt University Medical Center Mass Spectrometry Bioinformaticist David Tabb. Proteomics is a project currently under-development.

Source :
http://cancer.jpl.nasa.gov/documents/applications/laboratory-catalog-and-archive-service-labcas/proteomics

Videos and stuff from Linux Fest NorthWest 2016

                                            Picture source : https://twitter.com/ChrisLAS
                                         
After a great Linux NorthWest 2016(Bellingham, WA • April 23rd & 24th)
A lot of pictures and Videos comes online... info : LinuxFest NorthWest 2016...

The YouTube channel:  https://www.youtube.com/channel/UCqeKeRXssKqbK4XHqLYrctQ

PodCast from jupiterbroadcasting:
http://www.jupiterbroadcasting.com/99036/linuxfest-northwest-2016-las-414/

Big Data Analytics with DB like PostgreSQL

I was asked how to use the data and get it into some kind of repport?

We are testing PowerBI from Microsoft : https://powerbi.microsoft.com

The next one is : 2UDA - Big Data Analytics with PostgreSQL

"2ndQuadrant Unified Data Analytics (2UDA – pronounced tudor) is a data analytics application suite that unifies databases, spreadsheets, data mining and visualisation in one seamless desktop installer. Once installed, 2UDA gives users access to the dramatic data visualisation and data mining capabilities of Orange, utilizing the latest big data features of PostgreSQL, as well as the spreadsheet function of LibreOffice Calc." from : http://2ndquadrant.com/en/resources/2uda/

Data from multi-rotor wind turbine

"Vestas has taken the wraps off a multi-rotor wind turbine as the company looks to raise power outputs from single installations without having to increase hardware tip heights. The manufacturing giant said today it will work with the Technical University of Denmark to install the concept demonstrator at the Risø test site near Roskilde."    Source : http://renews.biz/102367/vestas-turbine-four-play/

How to setup ICAT on the Database (Postgres)

If you got issue with the Postgres DB I can recommend trying : http://www.pgadmin.org/

The code for making ICAT  (the iRODS database) :

-- DROP DATABASE "ICAT";  (  please Note : IF ANY  )

CREATE DATABASE "ICAT"
  WITH OWNER = postgres
       ENCODING = 'UTF8'
       TABLESPACE = pg_default
       LC_COLLATE = 'en_US.UTF-8'
       LC_CTYPE = 'en_US.UTF-8'
       CONNECTION LIMIT = -1;
GRANT CONNECT, TEMPORARY ON DATABASE "ICAT" TO public;
GRANT ALL ON DATABASE "ICAT" TO postgres;
GRANT ALL ON DATABASE "ICAT" TO irods;

Then run (in BASH):
/var/lib/irods/packaging/setup_irods.sh  ; iinit

Using the iRODS Repo

iRODS Packages

This server hosts both APT and YUM repositories for the iRODS Consortium.

Installation

The following two packages add this server to your package manager's list of repositories to check for new software.

APT

The signed APT package installs the public key automatically from this server via https.

wget https://packages.irods.org/renci-irods-repository_20160318_amd64.deb
sudo dpkg -i renci-irods-repository_20160318_amd64.deb

YUM / ZYPPER

The signed YUM / ZYPPER package first requires the installation of the iRODS public key into the RPM DB.

wget https://packages.irods.org/irods-signing-key.asc
sudo rpm --import irods-signing-key.asc

Then, the package can be installed.

sudo rpm -i https://packages.irods.org/renci-irods-repository-20160318-1.x86_64.rpm

Source : https://github.com/irods/renci-irods-repository

Weekend and time to hack stuff

We are testing security - just for fun ;-)

Sorry I can't write anything about our results !

iRODS Upgrade from 4.1.x to 4.1.8

After alot of strange errors we did an upgrade of the iRODs-1 and iRODs-2 server.
More : https://groups.google.com/forum/#!topic/irod-chat/D7I5x-biw7I

My Script looks like this :  (please note do a FULL DB-backup before - better safe than sorry !! )

### BASH
wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/centos7/irods-database-plugin-mysql-1.8-centos7-x86_64.rpm
wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/centos7/irods-database-plugin-postgres-1.8-centos7-x86_64.rpm
wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/centos7/irods-database-plugin-postgres93-1.8-centos7-x86_64.rpm

wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/centos7/irods-dev-4.1.8-centos7-x86_64.rpm

wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/centos7/irods-icat-4.1.8-centos7-x86_64.rpm

wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/centos7/irods-icommands-4.1.8-centos7-x86_64.rpm

wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/centos7/irods-resource-4.1.8-centos7-x86_64.rpm

wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/centos7/irods-runtime-4.1.8-centos7-x86_64.rpm

rpm -U irods-dev-4.1.8-centos7-x86_64.rpm

rpm -U irods-icat-4.1.8-centos7-x86_64.rpm

rpm -U irods-runtime-4.1.8-centos7-x86_64.rpm

rpm -U irods-database-plugin-postgres-1.8-centos7-x86_64.rpm

postgresql-setup initdb

systemctl start postgresql

systemctl enable postgresql

Now Hiring: iRODS Application Engineer

Source : https://groups.google.com/forum/#!topic/irod-chat/4P1d0VChkK4

The iRODS Consortium is seeking an R&D application engineer to join our team. This is a new position. Please see the description below and visit http://unc.peopleadmin.com/postings/94190 for more information and to apply.

R&D Application Engineer

This position will work with internal and external customers, the iRODS development team, and the Consortium member recruitment team to identify, implement, document, and service iRODS-based data management systems. The engineer will generally work at RENCI, though may occasionally be deployed to customer sites for up to several weeks at a time. Additional duties include: providing email and phone support to internal and external customers; submitting bug reports and feature requests to the development team; documenting features and common configuration steps; and training customer administrators and users on deployed systems. 

The R&D Application Engineer will assist the iRODS development team to advance and maintain iRODS client interfaces. Requires advanced knowledge of the iRODS C++ server-client API and communications protocol. Support for Linux, Mac, and Windows.

The R&D Application will be responsible for:
- Providing direct support to iRODS Consortium contract customers by developing iRODS rules, clients, and accompanying software tools.
- Assisting with development of iRODS client APIs, including Jargon, an enhanced C++ API, and graphical interfaces.
- Assisting with development of iRODS storage resource and authentication plugins.
- Identifying novel solutions, leading and managing tasks, and coordinating team members to resolve customer issues on-time and on-budget.
- Diagnosing, troubleshooting, and providing workarounds for product bugs.
- Documenting, proposing, and explaining end-to-end solutions that incorporate unique aspects of the customers’ site: networking, operating system, iRODS configuration, and customer workflow.

At least three years of IT administrative experience and Linux sysadmin/devops experience is required. Experience interacting with data-intensive domain researchers (e.g., earth scientists, bioinformaticians) is required. Excellent customer interaction, negotiation communication, and planning skills. Experience planning and deploying campus-wide, high availability iRODS systems. Experience configuring and deploying iRODS clients with support for multiple authentication mechanisms. Experience configuring iRODS client authentication using PAM and Kerberos. Experience working with and presenting developments to the iRODS user community. Knowledge of Python, C++, and Java programming languages. Proficiency with Unix shell scripting and command line operations, computer network configuration, and machine virtualization. Experience managing projects using version control and collaboration systems (e.g., github). . Experience developing client-side interfaces for iRODS is preferred.

Get python to run with iRODS (FigShare)

To get the python-irodsclient running we need Python-pip.

info : https://pypi.python.org/pypi/python-irodsclient

Repository
Pip is part of Extra Packages for Enterprise Linux (EPEL), which is a community repository of non-standard packages for the RHEL distribution. First, we’ll install the EPEL repository:

rpm -iUvh http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm

Then let’s install python-pip and any required packages:

yum update ; yum -y install python-pip

And then we are able to install irods_figshare 
from : https://github.com/danielduduta/irods_figshare

source file : irods_figshare.conf.json

source file : irods_to_figshare.py

please see : http://bigdatadk.blogspot.dk/2016/03/irods-2-figshare-integration.html