Datamashup Workshop
Thanks to the Mix and Mash datamashup competition and the lovely folks at Digital NZ MESA was able to hold a workshop for datamashups and API at Victoria on Friday 2nd September 2011. The speaker was Tim McNamara, a technology analyst and data ninja from Wellington. And a big thanks to CityLink (www.citylink.co.nz) for the great wifi and Charities Commission for sponsoring the Beginners’ Guide to Mashups.
PLEASE NOTE: All opinions and views expressed in the material below are those of the author(s) alone and do not represent those of MESA, the MacDiarmid Institute or the Mix and Mash datamashup competition or its sponsors.
Overview
What you need for a data mashup:
- Data! Find your datasets that you want to mashup (see the data sources links below for ideas)
- Extract the datasets from their source.
- Mash the sets up together, either via writing your own code, or using a spreadsheet editor like Google Refine (and geocode the points if you need to!)
- Interpret/analyze the results (try Google Fusion Tables or ggplot2)- remember, the best mashups solve a problem.
- Stick on some form of UI (User interface) – mapbox is free and fun!
- Post the mashup online – some free hosting services are listed below
- Complete the Mix and Mash NZ entry form and make sure you have followed the Mix and Mash rules
-
Win $10 000! – that’s it
Workshop Slides
Workshop Notes
Workshop Recordings
MixandMash Workshop – Part 1 from Elf Eldridge on Vimeo.
MixandMash Workshop – Part 2 from MESA on Vimeo.
MixandMash Workshop – Part 3 from ENZSN on Vimeo.
Videos within presentation
Visualizing Data Using R and ggplot2
Mashing Government Data
Google Refine
Mapbox’s Tilemill map generation software
TileMill: Open Source Map Design from Development Seed on Vimeo.
Links
Useful Tools
Yahoo Query Language (YQL)
hurl.it
datasciencetoolkit.org
mapbox.com
Google Fusion tables
Yahoo Pipes
scraperwiki.com – searches XML tabs and pulls out what you want
Locations to ask for data
Data manipulation resources
Google Refine
http://mbostock.github.com/d3/
r studio
ggplot2
More ggplot2 goodness
Data hosting resources
Code-learning resources
codeacademy.com
Google Code University
Software Carpentry
Code-hosting resources
https://bitbucket.org/
cloud9ide.com
Mashup resources
http://www.mixandmash.org.nz/the-competition/mashup-guide/
General Datasources
data.govt.nz
thedatahub.org
Google Public Data Explorer
pachube.com
NZ Science Datasets
Historical Weather Database
Biodiversity Vegetation
Births per Day 2009/2010
Department of Conservation
DigitalNZ
NIWA
IRD
Economic Datasets
OECD Patent Database
Monthly Economic Indicators