Learning Journeys >
Open data, open source, APIs and innovation labs
This learning journey was curated by AmbITion Scotland lead consultant Hannah Rudman.
This learning journey introduces you to the concepts of open data and open source, as well as explaining about the opportunities and implications of making your data available to others through an Application Programming Interface (or API).
We’ve worked in a knowledge sharing partnership with the Edinburgh Festivals’ festivalslab project to develop some of the events and resources that make up this learning resource since 2009, so thanks as always to them. If you’re interested in continuing to learn about the benefits of open data and APIs, then follow the work of Sync in Scotland – they’re a part of Creative Scotland’s Digital Development Programme alongside AmbITion Scotland.
1 Getting started: understanding open data and open innovation
Find out what open source, open innovation, and open data are in this webcast masterclass. Essential watching for the basics and for contextualisation of this topic within the arts, cultural and heritage sector.
Open Innovation, Open Data came live from from Inspace, Edinburgh 23.02.11!
2 Beginner's guide to Open Source
This Beginner’s Guide to Open Source explains some of the online platforms and technologies that are out there to help you develop work within the “open” philosophical framework. If you’re thinking about developing an open source website, then this is for you.
For this post we have introduced an additional method for commenting : ‘highlighter’, which allows you to highlight any specific words, sentences, and images and comment mid text. In the true nature of open source please feel free to add your thoughts to what our authors have written.
Who is this guide for?
This guide is for anyone who has heard of the term Open Source, but who is bit confused about what it actually is! You probably use digital technologies in your organisation or individual practice, you know that Open Source is a “good thing”, but you are still unsure exactly is it, how it could benefit you, and what the risks might be.
What is Open Source?
The digital age has created unimaginable opportunities for people across the world, to learn, share and communicate in a global conversation. What does openness mean in our society, and what does it mean for us now in the early 21st century, a digital age?
Tony Hirst from the Open University discusses different ways of being open in a digital environment.
Songs and Recipes
These days, the words “Open Source” are most often applied to software or services on the web. However, the concept of “Open Source” has been around for a very long time. Think of traditional songs that have been passed down over the generations. Even before the words or tunes were written down they were passed on orally, and they are regarded as something that belong to us all. Or think of recipes that have been written down and handed on from mother to daughter, father to son, or passed freely between members of a community.
Both traditional song and family recipes are example of “Open Source” – essentially, something that has been written down and is freely available for anyone for their own use. In the case of the song, the “Source” would be the words or music. For the recipe, the “Source” is the written instructions including the ingredients, measurements, and how to combine and serve them.
A good example of Open Source from beyond the world of computing is the OpenCola project.
Unlike Coca-Cola or Pepsi companies that jealously guard their secret recipes, OpenCola is a brand of cola unique in that the instructions for making it are freely available and modifiable. This is the “Open” part of “Open Source”. Whether it is a recipe, a song or a Cola drink, the instructions, blueprint or code for making or reproducing it should be accessible to all.
Anybody can make the drink, and anyone can modify and improve on the OpenCola recipe as long as they too make the instructions for their modifications freely available.
How does this apply to software and the web?
When talking about the digital world, Open Source refers to the lines of computer code or the coding language or structure that is at the heart of a particular software package, web application or file type.
Open Source means that the intellectual property for this code is not held by anyone, or that all claimants to its intellectual property release their claims of ownership.
The benefits of an Open Source approach to developing digital platforms and applications are many. By sharing the “recipe” freely, there are no barriers to a community of developers and users making use of the new software, web application or file format, or building upon it to create new solutions to meet the needs of a range of different users and sectors.
These benefits are less possible with “closed” or “proprietary” software and solutions. For instance, think of the cola example given earlier. OpenCola is to Coca-Cola as Open Source software is to Microsoft’s software. Just as Coca-Cola jealously guard their popular product’s recipe and only they make changes, Microsoft has a popular range of Office software products (Word, Excel, PowerPoint, etc.) for which they protect the code and only their developers make improvements and modifications.
Even for large scale commercial software companies, their development team may be limited to a few dozen developers. Some would say that this closed approach to development makes their product more robust and secure. However, it also stops the sort wide-spread ownership and bespoke developments seen in the Open Source world, where a project may have thousands of developers around the globe all working to enhance and add their experience to the project.
Open Source as a Philosophy
So, what leads developers to spend a lot of time writing and supporting software that they are then essentially going to give away for free? Well, for many developers it is a way of building their own credibility in a global community of developers. Also, many developers receive lucrative contracts to support or implement the very software that they gave away for free.
Other developers are passionate about Open Source as a philosophy, believing that software should be free and accessible to all. Others see it as the best way of developing software – by laying it open to the scrutiny and input of a global community of developers.
Let’s now explore some examples of other Open Source technologies on the web.
An open format is a published specification for storing digital data which can therefore be used and implemented by anyone. Unlike other Open Source projects, many Open Formats were developed by commercial companies. Some examples include:
- .pdf file format – Portable Document Format, originally developed by Adobe, and now a common standard for sharing documents digitally
- .zip file format – used for compressing a series of files into one
- iCalendar format – used for sharing dates between calendars.
- RSS format – used for syndicating information (most often news), mainly via the Web
- PHP / XML / CSS – Although you may be less familiar with these, they are the building blocks of most modern websites. Without these technologies there would be no Facebook, Twitter, Wikipedia, WordPress, etc.
So, Open Formats that can be shared between different computers and users have driven the development of the modern web – indeed, many provide the infrastructure for our modern day communication. And the speed of development of the web owes much to their being freely available platforms that developers and users alike can access and build upon.
However, that does not mean that all the successful and well-known file formats are Open Formats. Believe it or not, the mp3 format, which has become the de facto standard for compressed digital audio, is not an Open Format, and software that uses mp3 encoders or decoders require a license.
Open Source Software
Open Source Software is computer software that is available in source code form for which the source code and certain other rights normally reserved for copyright holders are provided under a software license that permits users to study, change, improve and at times also to distribute the software.
Again, given the free nature of access to the code behind this software, such software is often available freely to the end user. Currently, Open Source alternatives can be found for most major commercial software packages.
For instance, Open Office is an Open Source and freely available equivalent to Microsoft Office, and likewise contains word processor, spreadsheet and presentation software alternatives to Microsoft’s main suite of products. And the good news is that all your Word, Excel and PowerPoint files will work in Open Office.
OpenOffice started life as a commercial competitor to Microsoft Office distributed by Sun Microsystems. However, when it became apparent that this wasn’t a battle that Sun were going to win, the software went Open Source. It now has a range of developers from around the world giving their time freely to support and develop the project. http://www.openoffice.org
Similarly for users of Adobe Photoshop, a well-used Open Source alternative would be GIMP – or “GNU Image Manipulation Program” (GNU being the Open Source licence under which GIMP is distributed). GIMP can be downloaded at http://www.gimp.org.
If you are looking for an Open Source Internet browser software, Mozilla Firefox is a widely-used alternative to Microsoft’s Internet Explorer and Apple’s Safari browsers. http://www.mozilla.com.
Likewise, there are many Open Source alternatives to the two leading commercial Operating Systems, Microsoft. Windows and Mac OSX. For instance, Linux and Unix are well established Open Source operating systems. http://www.linux.org
An Open Platform is a software system with published and fully documented application programming interfaces (known as APIs) that allow using the software to function in other ways than the original programmer intended, without requiring modification of the source code.
So, an Open Platform does not mean it is open source – however, it is a way of opening proprietary software up to a wider range of developers and uses, without compromising or revealing the core code behind a product.
These APIs allow third-party developers to integrate with the software platform to add functionality or to share information with the software.
Here are a few real-world examples of Open Platforms:
- Apple’s Open Platform and API for its iPad, iPod and iPhone ranges have allowed the development of c. 370,000 apps that have been designed by anyone with the skills to do so. Millions of apps are sold each year, generating an income for the designers as well as for Apple.
- Many websites use Open Platforms to engage a wider community of developers with their services. For example, Facebook have APIs which allow programmers to develop applications (or Apps) on the Facebook platform, or allow users to login to third-party websites using their Facebook username and password.
- An example from the cultural sector would be a Client Relationship Management software package that used an API to share customer information with a separate Box Office system or Accounts package.
Open Source on the Web
Most often, the term “Open Source” is used to refer to the web technologies. As we’ve already seen, some of the building blocks of the modern web are based on Open Source technologies.
However, from these building blocks have come some very powerful tools – tools that you don’t need to be a web developer to take advantage of.
A key example of this is the Content Management Systems or CMS. Most modern websites are driven by a CMS – which is essentially an system them that allows you to add, edit and delete content from your website quickly and easily through a series of secure web pages. In the past, many web developers invested in developing their own “closed” CMS, for which many would then charge clients substantial ongoing license fees. In other cases, developers would build new CMSs from scratch to meet the needs of different clients, which may involve the client paying for many weeks or months of costly development time.
Now, with the advent of a range of powerful and diverse Open Source CMS tools, development of even the most sophisticated websites is no longer solely the province of big multinational companies.
The “big three” Open Source CMS include Joomla!, Drupal and WordPress – although there are many others. You’ll probably have heard of the latter.WordPress started life as a highly-popular blogging platform but, due to the high numbers of developers working on the project, has developed over the years to become a fully-functional CMS supported by a community of many thousand developers across the globe.
What makes these three CMSs so popular is that they have a global network of developers working to develop, support and implement these technologies on behalf of a strong and engaged community of many millions of users.
In addition, many “plugins” (sometimes called “extensions” or “plugins”) and design templates are available for these CMSs. Many are free, and they can be used to develop both the functionality and look of your website. So, if you’re looking for eCommerce functionality, Social Networking features, Client Relationship Management (CRM) or even a Weather Forecast on your website, it’s likely that you’ll be able to find an Open Source solution that goes some of the way to meeting your needs, that can be “plugged in” to your Open Source CMS.
A great resource for comparing the leading Open Source CMSs can be found at http://www.waterandstone.com/open-source-cms-resources.
So Open Source means we can build our website for free, right?
Wrong. This is a common mistake. Think about it – just because someone might give you the tools of a carpenter, it does not qualify you to build an ornate cabinet.
Open Source software is simply that – a tool. And you will still need a skilled professional to implement and tailor Open Source technologies to your own particular needs.
The fact that the code is freely available to you doesn’t make you web developer. And if you start modifying Open Source code yourself, things can easily break – and you won’t have a support contract to fall back on.
Also, whilst Open Source software is software for which the source code is freely and publicly available, the licensing agreements vary as to what one is allowed to do with that code. Open Source software is often made available under the terms of a GNU General Public License. The GNU website explains:
The word “free” has two legitimate general meanings; it can refer either to freedom or to price. When we speak of “free software”, we’re talking about freedom, not price. (Think of “free speech”, not “free beer”.) Specifically, it means that a user is free to run the program, change the program, and redistribute the program with or without changes.
So, if you can find the Open Source technology to suit your project, you could stand to save a lot of development time and money, but it you are serious about having a professional-looking, elegant, unique, secure and functional web presence, you will need professional help to implement your website.
Working with a web developer on Open Source
In many respects, the relationship between a website developer and clients will not be very different whether they are using proprietary technology or Open Source.
Things that will be different are the warranties and support that the developer can give to you. For proprietary solutions and those coded by the developer for the client, they should of course be able to give you full support and warranties for the platform, whereas developers using Open Source tools as the basis of their offer will be somewhat at the mercy of the developers of those platforms, and so may only be able to offer support and warranties on their own customisation work rather than on the Open Source platform itself.
On the other hand, developers using Open Source tools to build your website should be in a position to give you free access to the full code of your website and, whilst you may have support agreements with the original developer, this need not stop you from moving your website to another developer in the future if you become dissatisfied with the service you are getting. On the other hand, proprietary technological platforms can tie you closely to the services of one developer, and contracts may preclude you from having access to the code or moving your site to another developer.
Open Source tools are quick to implement and the best of them are easy to use, and so some of the previously long and laborious data entry and content upload to a new web platform – which previously the web developer would have to manage and charge for – can be done by the client, which may also save money.
Who should use Open Source?
If you want to use the Open Source software “out of the box” – exactly as is without any amendments or additions – and you have some basic technical experience, Open Source may be ideal. For instance, if you want to use the freely available Open Office rather than buy an expensive new copy of Microsoft Office, and you don’t mind spending a bit of time getting used to this new software, go ahead – download it today!
In most cases, however, to take advantage of Open Source software and web services, you will require to have some technical skills. In some cases, you may need to be hugely proficient at various web languages and server technologies in order to get Open Source tools working for you.
As we’ve seen, there are many good reasons for getting involved with Open Source software. For many who have used expensive, outdated or over-complicated Proprietary Content Management Systems to update their website, the thought that they might be able to use a much simpler and ostensibly ‘free’ system like WordPress or Joomla! is too tempting to resist. As a result, many take the plunge without realising that, whilst these systems can be exceptionally easy-to-use and affordable, they are not simple to set up initially, and for the vast majority of organisations, it will be essential to have a developer on board to deliver the project.
Granted, there are many entry-level website builder packages out there that come pre-configured and ready to just choose your design and add content, but it’s unlikely that many cultural or creative organisation will be satisfied with the generic “out-of-the-box” look and functionality of one of these lighter-weight packages.
Think of the OpenCola example given earlier. Now you know the recipe, should your organisation choose to make its own Cola from now on, rather than buy it from the suppliers? Well, of course just because the recipe is freely available to you, it doesn’t mean that there aren’t costs associated in producing your own Cola. Obviously you need the raw ingredients, but you also need to buy the right kit to mix it all together. You would then need to devote a considerable amount of staff time to learning how to mix and bottle the ingredients.
Indeed, by the time you’ve factored in all of these costs in terms of time and money, unless your company is in the business of manufacturing drinks, you’ll probably conclude that it’s easier to go to the experts.
Likewise, with Open Source technology, you’ll probably find that it doesn’t pay to train up your marketing department to take the many months of leave it would require to learn the skills to successfully and securely develop your website using Open Source tools.
So, in most cases, an Open Source web project will require the services of a web developer who can demonstrate Open Source credentials.
Diagram comparing an Open Source set up to an equivalent Proprietary set up.
Open Source vs Proprietary software : Key differences
|Source code open to all||Source code open to partners only|
|Mainly free, but can be charged for||Mainly paid for, but can be free|
|Distribution unlimited||Distribution limited|
|No end user agreement||End user agreement|
Benefits of Open Source
- Whilst you will probably need to invest in the services of a web developer, the technological platform and ongoing licenses are generally free.
- Set-up time can be minimised with an off-the-shelf solution as compared to developing a system from scratch. For Open Source software, it can be as quick and easy as downloading and installing.
- Support and mentoring for the most popular Open Source platforms are widely available from large communities of developers and other users, often globally.
- As a result, security improvements, developments and bug-fixes can occur very quickly.
Drawbacks of Open Source
- Open Source tools are often written by programmers for programmers, so sometimes, their user-interfaces are not as friendly or easy to use as tools that have been tested with end-users.
- Setting up Open Source systems can often be difﬁcult, requiring other ﬁles, Open Source platforms and high levels of technical know-how – seek professional help!
- If the community of users and developers around an Open Source tool is small, documentation and support may be limited.
- Likewise, if this community is small, security updates and bug-ﬁxes may be slow to materialise.
- You don’t get the software assurances and support that you might through a commercial supplier. However, in some cases you can buy into ongoing support services from those that developed the Open Source platform.
Things to look out for
Keep in mind that Open Source might not always be the best or most cost-effective option.
There is not necessarily an elegant Open Source solution for your organisation’s particular needs. You may find that an Open Source product has half of the functionality you need, but requires significant new functionality adding. In this case it may be better to develop a new solution from scratch, rather than to appoint a web developer to go through potentially thousands of lines of someone else’s web coding in order to make the changes you require. That is, in many cases, it would be less time-consuming and therefore less costly to have developed a bespoke solution from scratch.
On the same note, customised changes to the underlying code of Open Source solutions can severely limit the ongoing support for the application. This is because changes made to the core software program can impede the ability to apply future updates, fixes or additional modules that are developed for the improvement of the solution without adversely impacting on the customisations you have made. In this event, an organisation may find itself in the precarious position of having to remain with the current version together with any limitations that version may have. If your web developer is heavily customising an Open Source solution, find out what this means for future updates to the core software.
Even without customisations, simply adding lots of different add-ins, extensions and plugins to an Open Source website can cause problems. These bits of software are of course written by a diverse range of developers. All it takes is for two developers to have named a variable in their software by the same name for the two bits of software to conflict, and this could cause your website to malfunction.
Any good Open Source tool will have an active community of users and developers – often worldwide – demanding new features and driving the product forwards. It pays to check how active and accessible this community is before you commit to any platform. Search the web to find out if the product has active discussion boards online, and whether a good range of both users and developers are contributing to the discussion. Check if developers are responding to user’s requests for new functionality and bug-fixes.
Likewise, check how long the product has been around for, and whether the developers are keeping it updated. For instance, if version 1.0 of the product was released three years ago and there has been no major update since, it’s probably not a very future-proof product upon which to base aspects of your business.
Watch out for so-called “Open Source” software and services that are not themselves based on Open Source platforms or technologies. For instance, you might pay nothing for the software, but need to invest in costly proprietary database software products in order to run it.
It is also worth checking with your existing software and systems suppliers to see if the Open Source platform that you are looking at can be integrated with these (for instance, box office, online merchant software, emailing services, etc.).
On the other hand, free is not always best – and stubbornly opting for the free software, web applications and plugins may be a false economy in the long run. This is because support for these applications from the developer can often be patchy or non-existent. However, if by paying a small annual license fee to a developer, you can find that the support that is provided is invaluable. For instance, WPMU Dev provides very fine premium themes and plugins for the WordPress platform. However, their membership fee is worth it just for the great levels of speedy support and helpful advice they provide to members in their forum.
Finally, you will hear some web developers – particularly those with their own proprietary Content Management Systems to sell – that Open Source tools are inherently insecure. They argue that, because anyone has access to the coding behind these technologies, they can use that knowledge to exploit insecurities and vulnerabilities in the software.
However, security vulnerabilities are equally problematic in proprietary software – as we know from the regular upgrades and security fixes users need to download from Microsoft, Apple or other suppliers. So, just as you would with other software, it is important to ensure that you are keeping up-to-date with the latest version of your Open Source software, and ensuring you have the latest bug fixes installed. Check with your developer to ensure they are keeping you updated and protected.
A website designer on Open Source
Jaco Justice is an Edinburgh based designer who, through the Open Source WordPress site at http://www.go-reborn.co.uk, promotes a collective of artists and designers, alongside related “pop-up” events across Scotland.
As a non-techie, Jaco discusses his relationship with WordPress and Open Source.
“How much control do you have over the look of an Open Source website?”
Control over the look of an Open Source website will often depend on the template you have chosen for the particular project. With good knowledge of web developing skills there isn’t technically any restriction.
It is fair to acknowledge that in the early days of blogging, the layout formats, albeit freely available, were a pretty standardised affair. While the branded uniformity of these designs held a functional benefit to the blog-host by prompting aspiring bloggers to also choose their service, it also resulted in a style repetition that easily tired on the eyes of users.
Currently the vast range of WordPress templates available is staggering. Many are free or of relative little cost and would only involve a small credit to the theme’s author in the design (much as a web design company would usually add at the bottom of a website anyway).
Ongoing updates to these themes are often included. These could range from mail-out plugins to new photo gallery models.
However problems will certainly arise where the very nature of what attracted you to a template is over-looked – perhaps because too much information is being crammed into an unsuitable design. Keeping designs simple and clean is always my priority.
“What skills will I need to manage an Open Source website?”
One of the main hurdles, in my experience, may be with a site’s imagery. However, the advantage of WordPress is that even knowledge of FTP and Adobe Photoshop isn’t necessary for uploading images. Pre-sets in the CMS can handle re-sizing and the naming of ﬁles.
WordPress’ content management “dashboard” is self-explanatory, and can easily be explored by experimenting drafts that can be kept private.
“What aspects of developing an Open Source website might I need help with?”
Plug-ins. A new Open Source site may be the expansion of a current blog hosted elsewhere. Therefore utilising a plug-in to import previously written posts, contacts and attachments saves considerable time. This also extends to spam ﬁlters, adapting the site for mobiles and an ever-evolving list of ways to make your OS site (and your business) run more efﬁciently.
Shopping carts are also notoriously buggy, and its likely you will need help in keeping the online shop’s look in-line with the site’s main design.
“You work in partnership with web developers to create your Open Source websites. How does that relationship work?”
As a designer from a predominantly print background, the ever-evolving world of web development can be a daunting area to contemplate, never mind fully understand. It’s hard enough just keeping up with the buzz words via the ‘Twitterpshere’!
Web design is a very different discipline to design for print, so tools that allow greater and easier communication between designer and web-developers like WordPress are key to successful collaborations.
In my view the opposing, but utterly intrinsic, methods of designer and developer perform best when both are allowed the freedom to create using their preferred tools. In my case I want to conceptualise the look and feel of a website.
Navigation and some basic functionality is included in many of the Open Source web design templates that are available, but I don’t want to be bogged down by the practicalities of how the site will work on a technical level. This isn’t so much “passing the buck”, but a much more conscience decision to allow specialists to perform a division of labour to achieve the best possible results. Specifically, my designs are completed in Adobe Photoshop/Illustrator formats with the confidence that the developer can then take the baton.
“I thought that WordPress was just for blogs. Can I use it for my organisation’s website?”
WordPress was originally an easy target for, in my opinion, skills-protective developers to brand as “just a blogging tool”. The very nature of Open Source has meant it has far outstripped any constraints. Primarily, I don’t feel I have any design restrictions when using WordPress as a CMS – and I’ve had experience in both off-the-shelf templates and totally bespoke sites. Most importantly, I can say I’ve encountered no drawbacks to WordPress’ functionality and ease of use for the end client.
What you have to ask yourself is: do you want an easily manageable website that you and all of your staff can be trained to update at a fraction of the cost of a bespoke site and CMS?
“What is the future of Open Source?”
I see Open Source as a reflection of the human appetite for learning. While some protect their knowledge and hesitate to share (mostly stimulated by intellectual property) others see the dynamic future benefits of enabling those ʻstragglingʼ behind to gets the bits and pieces of information essential to global progress.
Freelance graphic designer and illustrator.
http://www.go-reborn.co.uk – WordPress template purchased for £40. Site updates were achieved with web developerʼs assistance to introduce shopping cart and subtle design changes.
As more organisations lead collaborative efforts, the implications of sharing data come to the forefront. Data sharing – whether for marketing, ticketing, fundraising purposes – raises a host of issues. Does pooling information about patrons lead to greater revenues for all parties? Or do organizations risk a negative response from patrons? Integrating the range of software that arts organisations to ensure that data sharing is more open is also a problem, but open data also offers a host of new opportunities.
Tim Berners-Lee, inventor of the World Wide Web, talks here on the 01.02.2011 OU/BBC Digital Planet programme’s on the value of open data.
This positive value of open data is why AmbITion Scotland supports and works together with Edinburgh Festivals Innovation Lab http://www.festivalslab.com @festivalslab – to ensure that the cultural sector maximises the value of open data by innovating new ideas/products. Ben Werdmuller from Edinburgh Festivals’ Innovation Lab explains open data:
“Open data sounds like a much more techie concept than it really is. It’s really a way to let third parties plug into and spread your organization’s information, in a way that you control, and allows them to create publications, products and services that you don’t have the time, resources or inclination to develop or maintain. You become the centre of a creative ecosystem – something arts organizations, and especially festivals – are already brilliant at. It’s a perfect fit.
More verbosely, open data takes the valuable information that an organization already owns and turns it into a product in itself, free of any specific form or function. Usually this is as a kind of database that third parties can query and automatically incorporate into their products.
You can think of open data as a public, read-only interface directly into the parts of your data that you want other organizations to get hold of – and, in fact, as the easiest, cheapest way to share that information. You might want to do this in order to ensure the accuracy of the information as it spreads across the web, for promotional reasons, to stimulate an ecosystem of third party developers, publishers, creatives and private companies around your products or services, because you believe making the data available is the right thing to do – or a combination of any of these things.
One other good reason is that if you don’t release your data in this way, someone else might do it for you – and you may well want to retain control, in order to maximize usefulness for the public, enforce your ownership, and keep track of who’s using it.
One of an arts festival’s most valuable sets of data (although by no means the only one in its possession) is its listings information. By making it available as open data, the festival can allow third parties to incorporate accurate listings – and have corrections, changes and cancellations filter down into the ecosystem almost instantly.
Better yet, they can ensure that these third-party listings link to the official ticketing page in order to drive awareness and revenue, and they can also retrieve detailed information about how it is used. This is because while the data is open, it isn’t public domain; that is to say, the festival retains complete ownership and licenses the information to the third parties who use it. That license can force the party to authenticate whenever they ask for information, resulting in a detailed set of user information from across the festival ecosystem that could be aggregated to infer demographic information,
trends in audience attention, and environmental details like geographic location. The license can also impose restrictions on how the data is used. It could require that applications include a copyright statement, for example, or a link back to the festival’s homepage as part of each listing.
Many arts organizations already use forms of open data: they might publish an RSS feed, for example, or push information through a Twitter account.”
Rohan Gunatillake of Edinburgh Festivals Innovation Lab goes on to consider in more depth why open data and the arts are natural partners in this blog: http://www.festivalslab.com/why-open-data-the-arts-are-natural-partners-o
Edinburgh Festivals Innovation Lab is formally part of the AmbITion Scotland programme, and in February 2011, AmbITion Scotland & Edinburgh Festivals Innovation Lab co-hosted Open Innovation, Open Data: What it is & Why it Matters with Roland Harwood and Ben Werdmuller.
Open Source can sometimes be used to describe a methodology for engaging or or set of principles for participation with an event. For example, BarCamps, and hack days modus operandi is:
- open exchange: free exchange of ideas is critical to creating an environment where people can learn and use existing ideas towards creating new one.
- power of participation: collaborations (we) can solve problems that no one person may be able to solve on their own.
- rapid prototyping: rapidly produced prototypes can lead to rapid failures, but that leads to better solutions found faster as you learn by doing.
- meritocracy: the best ideas rise and gather effort from the community
- communities: formed around a common purpose, a global community’s capabilities and capacity far exceed what any individual could do
Open Office http://www.openoffice.org
Culture Hackday http://culturehackday.org.uk
Culture Hack Scotland http://culturehackscotland.com
Open University/BBC Digital Planet series of programmes exploring what what open means
About the authors
Hannah Rudman runs Rudman Consulting, is Lead Consultant of AmbITion Scotland, having been director of the AmbITion pilot in England. She advises on national cultural policy around digital development in the UK; is an Associate of Mission Models Money; Specialist Advisor to Cultural Enterprise Office; and is a Lecturer at Edinburgh Napier University’s School of Computing. Hannah is also Arts Professional’s “Harnessing IT” columnist; and on the board of New Media Scotland.
Hannah has overseen the strategic digital development of over 100 cultural sector organisations, across all art forms and operational models. She has also developed an online hands-on toolkit for organisations looking to self-lead their own digital developments – the AmbITion Approach.
Hannah also runs Envirodigital. Envirodigital is a lighthouse brand, guiding the creative and digital industries in a sustainable direction through the use of digital tools. Envirodigital believes that getting digital can facilitate being sustainable, and helps organisations create truly economically and ecologically sustainable digital developments. Envirodigital helped to establish the new National Theatre Wales’ digital and environmental policies, and is helping to ensure many organisations’ and events’ environmental sustainability through providing webcasting production and consultancy.
Pooka.Pro is a new business providing web development services and consultancy to the cultural, creative and charitable sectors. Pooka.Pro specialises in Open Source technologies, and offers environmentally-friendly hosting solutions.
Established by experienced arts marketer and web developer Marcus J Wilson, and including a range of partnerships with other designers and developers, Pooka.Pro believes in showcasing compelling content through well-designed, accessible and interactive websites.
Marcus is a former Broadband Britain Champion for his work on the Northings website, which is the arts magazine and social network for arts and culture in northern Scotland.
Between 2003 and 2005, Marcus project managed a £250,000 project to develop an online ticketing service for the Highlands and Islands of Scotland. The Booth quickly achieved all projected client adoption and income targets set, and Booth Scotland Ltd became an independent company in 2007, and has since rolled-out to service cultural organisations across Scotland. As a result, Marcus sat on the Scottish Government’s National Box Office Scoping Group in 2007.
Marcus is currently a Reference Group member for Creative Scotland’s national Amb:IT:ion programme, which is engaged with over 200 cultural organisations across Scotland, funding a range of these to take forward major web development projects for their organisations.
Marcus runs the weekly Joomla! Breakfast in Edinburgh, a discussion group for Open Source developers using Joomla! CMS.
 Source: http://www.festivalslab.com/open-data-in-the-arts-an-introduction
3 Case study: the Edinburgh Festivals' API
The Edinburgh Festivals working together made a communal decision to open up their data, and provide access to it through an API. This case study video highlights some of the implications, the journey and some of the outcomes of the API. Interesting stuff!
4 Case study: projects powered by open data, open innovation and APIs #1
The Edinburgh Festivals’ API, and other Scottish cultural organisations opening up their data also enabled Culture Hack activity to take place. This post records the live proceedings from Culture Hack Scotland 2011 so that you can see the process of the 24hrs.
5 Case study: projects powered by open data, open innovation and APIs #2
This post presents the Culture Hack process though the eyes of a cultural professional turned hacker, Taras Young. He explains from his perspective how his hack of the festivals data using the API enabled a new interactive, playful app to emerge in the cultural landscape!
Over the weekend, I was incredibly fortunate to take part in what was – to my knowledge – the first proper Scottish hack day, Culture Hack Scotland.
For those not familiar with the term, a hack day is essentially 24 hours in which a large group of geeks, furnished with everything they need – raw data, coffeef, sugared foods, collaboration, enthusiasm, and somewhere to park their laptops – create many wonderful and ingenious things, seemingly out of thin air.
This is a long post, so grab a mug of tea, and join me as we enter the weird and wonderful world of speed-coding with Scotland’s rawest cultural datasets.
It started on Friday evening at the InSpace building – formerly, in my student days, Crichton St Car Park. A fairly informal networking-and-beer session was followed by introductions from the organisers and supporters of the project. Even at that point, there was excitement in the air, and it was obvious it would be a great event.
By the end of the evening, developers were being encouraged to go and have dinner, while cultural people were being gently and politely directed towards the exit. Although I had a developer ticket, I felt more cultural, so made my excuses. I was left, however, with the nagging feeling that I should have stayed.
[Skip the next bit if you don’t want to read about me and my hack.]
Ah, great, I knew you’d make the right decision! Now, although my day job involves working with ‘digital media’ – I’m currently looking after the development of our brand new website, building up our social networking community, designing stuff for plasma screens, training staff to use video cameras, and so on – not ‘digital media’ as in blank CDs – I wouldn’t call myself a developer. I’ve been dabbling in code since I was a nipper, but anyone who’s actually seen my code would probably baulk at how baroque and twisted it is. This, I guess, is what I was thinking: when it comes to software development, getting away with writing bad code should be my forté, so why shouldn’t I give it a go?
So first, I downloaded the datasets provided to Culture Hack Scotland by a variety of big-name Scottish cultural organisations, like the National Museums of Scotland, National Theatre of Scotland, and Edinburgh Council (who had provided 365 days of brilliant data about footfall around Edinburgh). However, I realised that if I was going to build something, I’d have to pick a dataset that had already been formatted; I didn’t want to spend precious time getting the data into a format that I could work with.
That’s where the Edinburgh Festivals listings API, provided by FestivalsLab, came in. An API (in this case, anyway) is a system you can query in order to find out information, and this one gave you raw information about last year’s shows at the Festivals – like show title, description, date, time, venue, map location, price, and so on. After staring blankly at the types of information available, it eventually struck me that it was all quite similar to the datasets I’d once tapped into a Commodore 64, some 20 (jesus, 20?!) years ago – when I was trying to make my own text adventures. Eureka!
First off, I had to find a way of interfacing with the API, which returned information in the JSON format. Ah. I’d never done that before. Or heard of JSON. Right. Since I’d done a little coding in PHP before (basically, customising my website), I chose to go with that, and headed for the excellent PHP online manual. After a lot of tinkering, searching, copying, pasting, learning and adjusting, I finally managed to get a script working that could send off a request to the FestivalsLab server, and return a PHP array/object thing (still not sure what it’s called) with all the data I needed. (Actually, it took a while because I originally wrote an iCal parser out of a fear of tackling JSON, before realsing the JSON results not only had more useful data, but were much easier to work with.)
To take a break, I headed for more familiar territory – design and layout using HTML/CSS. I also gave it a name – EDVENT, a sort of amalgamation of ADVENT, the classic text adventure from ‘the day’, and Ed-inburgh. By the time I’d done that, it was past 3am, so I surreptitiously added my proposed entry to the Culture Hack Scotland wiki – more as an incentive to actually see it through the next day than an admission that I knew what I was doing – and went to bed.
Eight hours pass… and I’m being woken up by the postman, who, unrelated to the hack day, is delivering me a Penguin Classics copy of Casanova’s abridged memoirs from Amazon. Well, you never know. But – yikes – it’s 11am. The hack day’s been going for, well, 11 hours over in Edinburgh. And I’m still in Dunfermline, at my front door, holding a copy of Casanova’s abridged memoirs.
So, after a lightning-fast shower, breakfast and check of the excellent Train Times website, I was off back to Edinburgh. I arrived just after 1pm, with a hack so far consisting of a web page that looked a bit like a retro monitor, and a function for requesting listings from the FestivalsLab API. I thought I hadn’t a nun’s hope in Borstal of finishing the project, to coin a phrase. (No, I don’t get it either.)
I’d arrived at lunchtime, so the room was devoid of hackers – it was a free lunch. Organiser Ben was the only one at a desk, and also had pretty much the only space in the room next to him, so I sat down and started coding. Handy, since Ben was the brains – and hard work – behind the API I was using! (Sorry, this is beginning to sound like a story from ‘Take a Break’ or ‘Bella’.) In the event, I didn’t need to ask him much, but I suspect his API-ness washed over me as I furiously coded in order to get the thing done by the deadline of 4.15pm. Which, to my amazement, I did. A real-time text adventure based on the live data being fed from the Festivals server. I even had time to add a few comedy bits in – additional random phrases that, I hoped, would turn it from a rehash of boring listings into something more resembling life on the streets during Festival time.
What followed was a series of presentations of some amazing hacks from some extremely talented people. Inspired perhaps by the presentation the night before, there were useful hacks, beautiful hacks, playful hacks and ‘placeful’ hacks. While they were all impressive, my favourites were Alex and Jen‘s Steal It!, where you could steal artefacts from the National Museums of Scotland; and Culturephone, a listings interface via VoIP with voice recognition, which blew my mind. Some were also very obviously marketable as products – such as the lovely Festafriend, which we’ll no doubt be seeing again come August.
I gave my presentation, and – as you can see – I looked less silly than I felt (what do you mean I.. oi!) In the event, the jury was kind enough to give me a totally unexpected prize for my efforts, and EDVENT was also named Most Playful Hack, which was fantastic. You can see all the hacks, including mine, on the website.
Other highlights: meeting the brilliant Tom Scott – who loathes being called a celebrity, and is pictured below with an adoring fan – and getting my very own Culture Hack Scotland mug.
Taras’s Final Thought
Hack days aren’t just a case of having fun and making cool stuff, though. They showcase the sometimes overlooked breadth of talent out there (and, believe me, there’s plenty of it in Scotland). They also highlight how important open data is, and that cultural organisations should do as much as they possibly can to free their data; if it’s useful enough, and if it’s in an accessible format, there’s no telling what incredible hack some friendly geek could turn it into. It was a great opportunity for the cultural and developer communities to network, too, and – for a lucky few – there were already partnerships emerging at the event, which look like they may evolve into commercial opportunities – clearly a great result.
The organisers, including Rohan, Ben, and Suzy (and others!) did a fantastic job, and I overheard many participants saying it had been the most productive and fun hack day they’d ever attended. It was my first time, and it’s certainly set the bar very high. If, as is being suggested, there’s another one already being planned, I’m sure it’ll be at least as good – if not better. I can’t wait.
Originally posted on Taras Young’s website.
6 Case study: Edinburgh International Book Festival using the festivals' API
This in depth case study of Edinburgh International Book Festival‘s mobile optimised website site made great use of their API. Watch to find out more about the benefits and implications.
7 Get active - create your own innovation lab
This How To… Guide explains how any organisation can set up their own innovation lab, to maximise the opportunities of open innovation. It’s practical and full of ideas, so don’t expect to be still sitting down after reading this brilliantly illustrated guide!
Hot off the press from Festivalslab especially for AmbITion Scotland is a manifesto and How To… Guide explaining how to create your own innovation lab/build in innovation practices to your organisation. Festivalslab shares the last 2 years’ learning from the Festivals Edinburgh group on how to create a ripe environment for innovation in your organisation. A handy manifesto and practical How To… Guide packed with simple actions. Read it here!
How to Create Your Own Innovation Lab
8 Case study: innovation labs at work - festivalslab
And here’s proof that innovation labs work. Festivals Edinburgh commissioned a long-running innovation lab (festivalslab) to work with all of Edinburgh’s Festivals over the past couple of years. This is a showcase of the work achieved.
The methodologies and opportunities that festivalslab piloted have been made available to Scotland’s arts, cultural and heritage sector more widely for 2012-14 through the Sync programme.
This summer marks two years since the inception of the festivalslab project. To celebrate, The Festivals Lab team have compiled a showreel document of the projects, which highlights what worked and what didn’t. The content and knowledge sharing partnership that AmbITion Scotland has held with Festivalslab has enabled the learnings of the festivals to be made available to the wider Scottish creative and cultural sector over these last few years. Thanks guys!
Also: watch the videos AmbITion Scotland made on the API project, and on the Culture Hack Scotland 2012 Edinburgh International Festival’s mobile site: