onsdag den 23. marts 2016

irods 2 figshare integration.

Post :  https://groups.google.com/forum/#!topic/irod-chat/bSB89t1Q2pc

We have been doing a iRODS to Figshare PoC py plugin.
Feel free to test it ;-)   

We are missing  : Use the script from the iDROP-Web and maybe have a figshare-button or menu thing.  (any got that working ? let me know! )


irods_figshare playground for irods figshare integration.
irods_to_figshare.py script is a PoC for copying files from iRODS to figshare via the figshare API.

Figshare access token
Create a figshare account, login and then access the Apllications section. There you can generate a Personal Access token that can be used to access the figshare API.
Required configuration
Check out irods_figshare.conf.json for some sample values.
Running the script

Requires python-irodsclient available here. 
Make sure that the file passed as argument to the script exists in iRODS and that it has associated the following metadata fields: title, description, tags. The script will search for and use these in order to create the figshare article.

Execute the script: python irods_to_figshare.py figshare_test.txt

Code :
file : irods_figshare.conf.json
{
"irods": {
"host": "localhost",
"port": 1247,
"user": "irods",
"passwd": "irods",
"zone": "tempZone",
"collection": "/tempZone/home/irods"
},
"figshare": {
"access_token": "6c5e3a8beb74a4e817f8994525d3af60e3ac2e21cbd70a54eabdcf9e925d9672d7a31364cf7233a957",
"api_endpoint": "https://api.figshare.com/v2"
}
}

----  CUT -----
file :  irods_to_figshare.py

import argparse
import requests
import json
import hashlib
import os

from irods.session import iRODSSession



#load config
config = json.loads(open('irods_figshare.conf.json').read())

#argument parser
arg_parser = argparse.ArgumentParser()
arg_parser.add_argument('file', help="iRODS file to work on")
args = arg_parser.parse_args()

#irods stuff
irods_session = iRODSSession(host=config['irods']['host'],
     port=config['irods']['port'],
                             user=config['irods']['user'],
                             password=str(config['irods']['passwd']),
                             zone=config['irods']['zone'])

irods_collection = irods_session.collections.get(config['irods']['collection'])

#figshare stuff
access_token = config['figshare']['access_token']
api_endpoint = config['figshare']['api_endpoint']

#request session
http_session = requests.Session()
api_headers = {
        "Authorization": "token {}".format(access_token),
        "Content-Type": "application/json"
}
http_session.headers.update(api_headers)

#lets get the file and associated metadata
file_path = os.path.join(config['irods']['collection'], args.file)
file_obj = irods_session.data_objects.get(file_path)
file_metadata = file_obj.metadata.items()

#create article
article_metadata = {
"title": "",
"description": "",
"tags": ""
}

for meta in file_metadata:
    name = meta.name.lower()
    if name == "title":
        article_metadata["title"] = meta.value
    elif name == "description":
        article_metadata["description"] = meta.value
    elif name == "tags":
        article_metadata["tags"] = meta.value.split(',')
    else:
        continue

article_create_endpoint = "{}/account/articles".format(api_endpoint)
request = http_session.post(article_create_endpoint, data=json.dumps(article_metadata), headers=api_headers)
article_id = request.headers["Location"].split('/')[-1]

#Add file to article - upload file
##initiate upload

m = hashlib.md5()
with file_obj.open('r') as fd:
for line in fd:
    m.update(line)
fd.seek(0, 0)
md5sum = m.hexdigest()

file_data = {
"name": file_obj.name,
"size": file_obj.size,
"md5": md5sum
}

create_file_endpoint = "{}/account/articles/{}/files".format(api_endpoint, article_id)
request = http_session.post(create_file_endpoint, data=json.dumps(file_data))
file_url = '{location}'.format(**request.json())
file_id = file_url.split('/')[-1]

file_info = http_session.get(file_url).json()
file_parts = http_session.get(file_info["upload_url"]).json()["parts"]

with file_obj.open('r') as fd:
    for part in file_parts:
        size = part['endOffset'] - part['startOffset'] + 1
        address = '{}/{}'.format(file_info['upload_url'], part['partNo'])
        http_session.put(address, data=fd.read(size))

#complete file
file_completed = http_session.post(file_url)

print "Done!"
print "Check out the article at {}/{}?access_token={}".format(article_create_endpoint, article_id, access_token)
print "Check out the file at {}/{}/files/{}?access_token={}".format(article_create_endpoint, article_id, file_id, access_token)

mandag den 21. marts 2016

Test and audit of our network and servers




It is time to do a bit of testing and audit of our network and servers ;-)

The tool(ISO) we are using here is : Cyborg Hawk from : http://cyborg.ztrela.com/



        

Money from Warren Edward Buffett ? Noooo way



Got this mail :


Your Attention Please, 


My name is Warren Buffett and I have an urgent donation information for you 

which will benefit you and your entire family, with the less

privileged in your local community. Respond back to this email 

immediately so as to enable me provide you with the complete 

information regarding this donation.  God Bless you richly. 

Warren Buffett. 

The real Warren Buffett : https://en.wikipedia.org/wiki/Warren_Buffett



Danm... its from Brazil and on a AOL.com... It's fake!  ( ->Never bullshit a bullshitter )

Return-Path: <>
Received: from mail.sbm.org.br (sbm-server6.impa.br [xxx.65.3.26])
by xxxxx.xxx.xx with SMTP id u2JGjnJ1013593-u2JGjnJ6013593; Sat, 19 Mar 2016 17:46:48 +0100
Received: from [xx.35.195.172] (unknown [xx.35.195.172])
(Authenticated sender: jointmeeting_italy@sbm.org.br)
by mail.sbm.org.br (Postfix) with ESMTPA id AD073F77F2;
Sat, 19 Mar 2016 12:35:56 -0300 (BRT)
Content-Type: multipart/alternative; boundary="===============1222434241=="
MIME-Version: 1.0
Subject: DONATION:PLEASE CONTACT ME.
To: "WARREN B." <foundation.bw@aol.com>
From: "WARREN B."<>
Date: Sat, 19 Mar 2016 15:35:46 +0000
Reply-To: foundation.bw@aol.com
X-SBM-MailScanner-ID: AD073F77F2.AC223
X-SBM-MailScanner-From:
X-Spam-Status: No
X-FEAS-HEURISTIC-HASH: W0ZdWUAnPDw3Jic3LjonJTcpLCw6UltGXVlB
X-FEAS-HEURISTIC: 3.51
X-FE-ORIG-ENV-FROM:
X-FEAS-CLIENT-IP: 147.65.3.26
Message-ID: <20160319174704.u2JHl4gH001818@xxx.dk>

tirsdag den 15. marts 2016

While we are waiting for iRODS 4.2

I have been doing a small lighting talk about setting up iRODS 4.1.x at OpenSourceDays 2016.
All my slides will be online at : https://opensourcedays.org/

We are also working on a Figshare Plugin for iRODS iDROP-web ;-)

While we are waiting for the new iRODS 4.2 we have been playing around with the 4.1.8
version from ftp://ftp.renci.org/pub/irods/preview/


Got an error ?  have a look at : https://groups.google.com/forum/#!forum/irod-chat
Also look at : https://scan.coverity.com/projects/2605 for future fixes...