Posts Tagged ‘jiscri’

Data Vitality

Sunday, September 27th, 2009

DIAP Rule of Thumb
Circumstantial observation of my email archive, at 272MBytes, having never deleted an email permanently and the file, ../mail, has been in use for 4 years. During this time my available xDSL line Bandwidth has increased, 2004 500MBits/sec to 1GBit/sec, 2008 1GBit/sec to 6GBits/Sec this is about 150% yearly increase whereas my mailbox has increased yearly by about 50%. It is this difference which DIAP attempts to use classing my email record as ‘mission critical’. Other record types will increase at different rates, as will bandwidth depending on location, but probably less than the average yearly bandwidth increase. This idea needs expanding but forms the foundation for the usefulness of DIAP, describing a DIAP rule of thumb. DIAP can also be viewed as a technique.
A pilot survey was run between June 08 and Aug 08 on Hampshire Lug ML named:- ‘[Hampshire] [OT] Identifying the importance and value of data’. This form the basis of a decision matrix tool to help DIAP ® users select data to use in DIAP ® system.Question 2)

“Following on from a survey I posted back in June, see below, designed to help make decisions about data vitality and importance to individuals and organisations, as well as find out a little bit more about the relationship between the importance of data types and their size, I have some results to publish. The survey was exclusive to Hants Lug and deliberately kept low key and has been a very useful exercise even though the number of participants has been relatively small – but very good for such a small readership, quality not quantity.

Thanks to those that took part and those that provided feedback and constructive criticism also thanks to HL ML readers for living with the thread. I will now design a new survey with some sort of incentive and float this to a much larger audience. The results I have are enough to incorporate a DIAP (R) decision matrix on the project website.

6 Participants:

Question 1)

Rate FIVE of these DATA TYPES if they were lost completely how best describes the effect to your users (and or yourself) organisation or home occupants.

Results:

*total damage – cost crippling – traumatic

2 [participants specified] Documents
1 Presentations
1 Photographs
1 Email boxes on a server
2 Code repository
1 Website code

*massive damage – high cost – devastating

1 Spreadsheets
1 .txt files
1 Email boxes on a server

*major damage – very costly – extremely upsetting

1 pdf documents
1 Photographs

*significant damage – significant cost – very upsetting

1 Spreadsheets
2 Accounting Software data
1 Photographs
1 server configuration files
1 music files
1 Virtual machine images

*damage – expensive – annoying

1 Documents
1 pdf documents
1 MS .pst file
1 Email boxes on a server
1 music files
1 ISO images
1 Virtual machine images
1 Code repository

Size the FIVE choices you made in the previous question

Results:

1 MB         100 MB        8 items
>100 MB  500 MB        1 item
>500 MB   1 GB            8 items
>1 GB       10 GB          8 items
>10 GB     50 GB          0 items
>50 GB     100 GB        2 items
>100 GB                        1 item

So from just this small pilot survey I can deduce qualitatively with reasonable certainly that importance of data, subjective to the individual or organisation, does not depend heavily on the data type.

That importance of data in relation to file size is loosely inversely proportional. So the most important files are generally the smallest in size. This is very encouraging information for the DIAP (R) project.

Thanks again to readers and participants.”

Damian Brasher

DIASER: Long term distributed internet archiving application

Saturday, September 26th, 2009

Requested by: http://code.google.com/p/jiscri/wiki/ProjectDocumentation

According to: JISCRI finalProgressPost requirements.

In this post you are required to address each one of the below elements, this post along with your final prototype will be evaluated for its readiness to launch to the end user. If selected, we’ll be using this post as the final “advertisement” that will go in an ‘Argos-like catalogue’ of JISC Software Prototypes (a vision of the future to come!). We hope to put this publication on the desk of as many senior managers in UK HE/FE as possible as well as other potential investors (e.g. NGOs and VCs) in the New Year so please write this Final post with that audience in mind.

Description of Prototype:

Proactive archiving… an advanced disk based backup volume management system with triple redundancy for HE and SME. A quick and low-cost way to make an environment more robust and data more accessible by archiving in multiple places. This replication also provides fast retrieval of archived data from all node hosting locations. A Perl installer creates the system. The application provides a decentralised, self-contained and managed storage utility.

Screenshot or diagram of prototype:

Diagram:

Vault Building

Vault Building

Flash Demo:

http://www.diaser.org.uk/about.html#configure

Screenshot:

DIASER Usage
DIASER Usage
Bandwidth
Bandwidth

Link to working prototype:

http://sourceforge.net/projects/diaser

http://sourceforge.net/projects/diaser/files/

Link to end user documentation:

http://www.diaser.org.uk/files/DiaserDocsv1.1.pdf

Link to code repository or API:

http://diaser.svn.sourceforge.net/viewvc/diaser

Link to technical documentation:

http://www.diaser.org.uk/manual.html

Date prototype was launched: 2nd June 2009

Project Team Names, Emails and Organisations:

PM/Dev: Damian Lajos Brasher

d {.} brasher @ omii {.} ac {.} uk OMII-UK ECS Interlinux Ltd

Neil P Chue Hong: Project advisor

n {.} ChueHong  @  omii {.} ac {.} uk OMII-UK ECS

Dr Simon J Hettrick: Documentation, graphics and marketing

s {.} hettrick @ omii {.} ac {.} uk OMII-UK ECS

Project Websites:

http://www.diaser.org.uk

http://www.diap.org.uk/background_ip

http://www.omii.ac.uk/wiki/DIASER (OMII-UK)

http://walkingwiththeelephants.co.uk (Sustainability)

http://interlinux.co.uk/wordpress (Final Project Post)

PIMS entry:

https://pims.jisc.ac.uk/projects/view/1267

Table of Content for Project Posts

Microblog posts: hash tag #diaser #jiscri

http://twitter.com/#search?q=%23diaser

DIASER SourceForge RSS news feed

News

18/12/09 v0.2.7 beta-1 released

17/12/09 eBook introduction site live

14/12/09 Interlinux Ltd UK Tax year tasks completed

21/11/09 DIAP IETF Internet Draft v11 released

05/11/09 v0.2.6 beta-1 released

30/09/09 v0.2.5 beta-1 released

21/09/09 v0.2.5 beta-1 released

02/08/09 Screencast – demo / mini tutorial released

21/07/09 DIASER beta-1 released

09/07/09 Attended JISC inf11 projects start-up meeting

04/07/09 Community feedback motivate DIASER architectural changes

25/06/09 Manual for DIASER released

04/06/09 Latest 4 month development roadmap, alpha -> beta-2

02/06/09 Build yourself a private storage cloud…

02/06/09 Deployable alpha of DIASER released

22/05/09 Projected release life cycle

22/05/09 Project memory and thinking

21/05/09 Dec 2006 – Mar 2009

17/05/09 Rollout of first fully deployable alpha planned wk 25th May

08/05/09 DIASER moves from alpha-dev to alpha

27/04/09 DIASER funded by UK Joint Information Systems Committee

Other

Download Statistics SourceForge total 1612 since April 2009

      Rank Pages Downloads

Dec 2009 * N/D 8,063 124
Nov 2009 1,585 8,917 183
Oct 2009 1,901 6,200 164
Sep 2009 1,547 11,562 291
Aug 2009 1,317 7,606 423
Jul 2009 1,026 9,588 300
Jun 2009 1,529 4,463 28
May 2009 1,258 4,468 51
Apr 2009 3,429 2,164 53
Dec 2009 * N/D 8,063
Nov 2009 1,585 8,917
Oct 2009 1,901 6,200
Sep 2009 1,547 11,562
Aug 2009 1,317 7,606
Jul 2009 1,026 9,588
Jun 2009 1,529 4,463
May 2009 1,258 4,468
Apr 2009 3,429 2,164

Google analytics for diaser.org.uk :

Analytics_www.diaser.org.uk_20090407-20091222

Analytics_www.diaser.org.uk_20090407-20091222