DigitalNZ

Your Nomination
Title for nomination: 

DigitalNZ

Nominee's name or organisation: 

National Library of NZ

Overview of your nomination: 

DigitalNZ is an initiative that aims to make New Zealand digital content easy to find, share and use. This includes content from government departments, publicly funded organisations, the private sector, and community groups. We operate a number of services, our most significant being a search service that aggregates metadata from more than 100 organisations that hold NZ images, document, and media... much of which is hidden or buried on the web. Our point of difference is that we operate this service with an open API and actively encourage others to build on top of it. We are run out of the National Library of NZ and, to the best of our knowledge, we were the first central government agency to provide a full API service. We were only able to offer a full service with API keys, metrics, and throttling by building off the work of others in the open source community. We have various supporting websites including a service that allows the public to build their own NZ search engine, a shared research repository for object hosting, and a community driven helpdesk for digitisation support. All these services run on Open Source software. We have worked with a number of vendors including Boost New Media, Codec, 3Months, and YouDo. We would like to particularly acknowledge Boost for their commitment to Open Ource and their guidance in helping us along this path.

Reasons for winning this award: 

We started out with a strong organisational bias and, despite this, have been able to demonstrate what is possible when open source software is embraced. We feel we are part-way through the journey, want to acknowledge that, and now more towards being a contributor. If you you look at the services we are running in the additional info section you will see that these are not simple services. Equivalent proprietary systems are licensed for hundreds of thousands of dollars a year. We have enable more than 100 NZ organisations to provide an API to their content metadata We have enabled Schools and students to find NZ material they could not previously access We have demonstrated to other Government departments what can be achieved with open source software and iterative development

Supporting Information
Use and benefits of open source: 

All aspects of our technology use open source software, or custom software we are working to release. Our search index runs on solr, our metadata repository is mysql, our harvesting tools use various open source parsers and libraries to support the indexing of hundreds of NZ content sources (databases and websites). We use Java for back-end development. Our front-end services and website are built on Ruby on Rails, using the Radiant content management system and numerous Ruby Gems. Our object repository is Fedora with a front-end called Muradora which was released under an open source licence by Macquarie University. We first started development on August 4th 2008, and right from the very beginning we opted for a fully open source approach. Despite the project coming from the National Library, and Government Department, that is heavily reliant on proprietary solutions, it was an obvious and natural choice. The project team was given the ability to make the right choices and it gave us the flexibility to innovate and work at speed. Just thinking about it, I think there was one aspect that wasn't open source. We first launched on Solaris because of an organisational decision, but I'm pleased to say we have just switched to Red hat Linux. We have funded a few small open source releases to date: http://github.com/boost/dnz-client http://drupalmodules.com/module/digitalnz-api http://kete.net.nz/en/blog/topics/show/291-kete-team-releases-external-s...

Recent achievements: 

Being asked to prototype a solution for the Spanish National Library for an aggregation service for the Spanish speaking world. This resulted in us building our first open source package of the search service so the Spanish could continue their prototyping. The package has not yet been widely distributed but will be shortly. We have had similar enquiries from the UK, Saudi Arabia, Taiwan, and California. In 2009 we won the Internet Industry award for the Best Educational Service, for the way our APIs were enabling access to NZ content by Schools (e.g. integrating with learning management systems like Moodle).

Plans for future work: 

We are currently completing work to scale the architecture and provide enterprise-class resilience for the services. We currently have under 2 million records in our production environment, but are successfully supporting 20 million records on our test environment with the new architecture. This new architecture will go live in October. Afterwhich we will be packaging the full stack for release as an open source project for release this calendar year. We currently have more than 20 GIT repositories of various components used in our project that we will be progressively releasing. The newest of which is the code for our shortly to be released iPhone/iPad application. We have received so much value from the open source work we have used and now we are reaching a level of maturity we are committed to repaying some of that value by releasing our own work.

Websites and other information: 

DigitalNZ website http://www.digitalnz.org/ Make It Digital Helpdesk for digitisation support http://makeit.digitalnz.org/ Search service http://search.digitalnz.org/ Custom search builder http://www.digitalnz.org/customise API details http://www.digitalnz.org/developer Matapihi built on top of API and open source radiant platform (hosted by DigitalNZ) http://www.matapihi.org.nz/ Example API integration (see right-hand column) http://www.nzhistory.net.nz/search/mch-search/other/supplejack Example Moodle plugins integrating API http://moodle.org/mod/data/view.php?d=13&rid=2338 http://moodle.org/mod/data/view.php?d=13&rid=2337 Demo of Spanish language version of DigitalNZ search being prototyped by consortia of Spanish speaking countries: http://210.48.120.55/gdl