Blog

Hacks and Hackers Hack Day Manchester: Tweeting police, local gigs and Preston’s summer spend

We’re sure that Greater Manchester Police had us in mind when they set about tweeting 24 hours of calls on the eve of our Hacks and Hackers Hack Day Manchester. (Photos courtesy of Michael Brunton-Spall).

It proved a fantastic data set to work with at our hack day on Friday 15th October at Vision+Media in Salford, and sparked four different ‘splinter’ groups. Michael Brunton-Spall, a developer from Guardian Platform (one of the event sponsors), set about making the tweets usable and created a Json GMP24 dataset [link].

Meanwhile, for the ‘Genetically Modified Policing’ project, Louise Bolotin, from Inside the M60, Lee Swettenham from MEN Media, programmer David Kendal and Megan Knight from the University of Central Lancashire scraped tweets and analysed peak times of tweets, the categories of calls and the number of followers of the feeds throughout the day.

Obviously, they would love to work with a dump of the police calls database, but in the meantime, this would do, said Megan, who presented the team’s work.

David Kendal also produced his own project mapping 999 calls in the area. He took the tweet data and put it through the Yahoo placemaker tool, plotting information on a Google map, to see which areas got calls over certain periods of time.

Yuwei Lin and Enrico Zini took the stage and First Prize for the final police project, a GMP tweet database, and showed a very neat search tool that allowed analysis of certain aspects of the police data (3257 items).

For example, we could look at the number of incidents that involved ‘sex’, or ‘youths and drinking’, whether the incidents involved males or females (“men are troublesome than women!” ), and at a tag cloud for certain locations. We could see a list of keywords and place names. It involved using the Json dataset created by Michael Brunton-Spall [dataset link] and adding keyword sets. The source code has been released here, along with a handy explanation.

Second prize went to ‘Preston’s Summer of Spend’, built by Uclan student Daniel Bentley and Scraperwiki’s Julian Todd. They took spending data from Preston City Council, converting PDFs to machine readable formats.

Once in a CSV file, they were able to create interactives, and identify interesting aspects of the data. It might be worth, for example, looking into why quite so much went to one individual that Google told us was a “legal representative of a controversial city development”.  A further step might be to request the same information from other local councils and compare the spending levels.

Third prize and the Scraperwiki mug for best scraper went to the ‘Quarternote’ project built by developers Kane, Robin, Zen, Becky, Andrew and Andrew. This web application, which got many of the audience very interested,  provided local music and band information for venue owners, promoters and event organisers.

By scraping MySpace, you could easily find band gigging in your area for your event. Simply put, you could put together a gig list in three clicks. While something like LastFM would have been an easier hack, the team targeted MySpace as a source to which more local bands were contributing. (Photo from video by @josephstash)

Tom Mortimer-Jones of Scraperwiki, freelance writer Ruth Rosselson, InsidetheM60′s Nigel Barlow, Journal Local developer Philip John and freelance Mark Bentley decided to hack data showing ‘Manchester Rich and Poor’. They made a comparison by ward in Manchester, showing different factors, eg. population density, unemployment rate, incapacity benefit and severe disablement allowance, and education.

Lastly, the Judgmental group, Francis, Chris, James (with some help from Kane and Philip) decided to do some work with legal data [disclaimer: I was also part of this one!]. Thanks to a friendly unknown donor, one of our team had been given a CD full of United Kingdom case judgment data. At the moment this is only available via Bailli and the team wanted to make something more usable and searchable (Bailli’s data cannot be scraped or indexed by Google). So judgmental.org.uk was created.

It is still a work in progress, but could eventually provide a very useful tool for journalists. Although the data is not updated past a certain point, journalists would be able to analyse the information for different factors: which judges made which judgments? What is the level of activity in different courts? Which times of year are busier? It could be scrutinised to determine different aspects of the cases.  [21/10/10: Updated screen grab of project, still in development]

Judge Andy Dickinson from the University of Central Lancashire has since blogged his thoughts about the day overall:

Give the increasing amount of raw data that organisations are pumping out journalists will find themselves vital in making sure that they stay accountable. But I said in an earlier post that good journalists don’t need to know how to do everything, they just need to know who to ask.

With thanks to our judges (Andy, along with developer Tim Dobson and Julian Tait from Open Data Cities), our host Vision+Media and our lovely sponsors Inside the M60, Guardian Open Platform, the Digital Editors NetworkVision+Media (supported by the European Regional Development Fund and the Northwest Regional Development Agency), Journal Local and MEN Media.

A special thanks to Louise  & Nigel at InsidetheM60 and Jacqui at Vision+Media for the organisational help.

Links to posts about Hacks and Hackers Hack Day Manchester:

Any more you have spotted? Any names I’ve missed off? Videos will be added to this post soon. If you have technical detail, or screen shots, or presentations to add please email judith [at] scraperwiki.com.

Our youngest hacker yet, with Aidan:

Tags: , ,

13 Responses to “Hacks and Hackers Hack Day Manchester: Tweeting police, local gigs and Preston’s summer spend”

  1. David Kendal October 17, 2010 at 12:28 pm #

    Errata: My name is David, not Dave, and Kendal has one L not two. My Mapping 999 Calls project is here.

    Also: Where’s the video? /me wants to watch with his eyes.

  2. jtownend October 17, 2010 at 5:00 pm #

    I’m so sorry David. It’s corrected now. I have also added the link. Video coming shortly…

  3. David Kendal October 17, 2010 at 7:09 pm #

    Don’t worry about it—lots of people make both mistakes. Looking forward to the video.

  4. jtownend October 17, 2010 at 10:52 pm #

    That’s no excuse for a journalist – especially because I frequently moan when people add an ‘s’ to my surname ;-)

    anyway, video now posted for your enjoyment – thanks to Joseph Stashko’s great work.
    http://blog.scraperwiki.com/2010/10/17/758213944/

  5. stevieflow October 18, 2010 at 1:08 pm #

    Hi – I also posted on the Plings blog in relation to some of the GMP tweets mangling we did in the blue room ;)

    http://blogs.plings.net/index.php/2010/10/15/are-7-of-police-tweets-about-youths/

Trackbacks/Pingbacks

  1. links for 2010-10-17 « Sarah Hartley - October 17, 2010

    [...] Hacks and Hackers Hack Day Manchester: Tweeting police, local gigs and Preston’s summer spend | Sc… Yuwei Lin and Enrico Zini took the stage and First Prize for the final police project, a GMP tweet database, and showed a very neat search tool that allowed analysis of certain aspects of the police data (3257 items). (tags: hhhmcr hacksandhackers scraperwiki) Possibly related posts: (automatically generated)Tip of the day – get back quicklyCitizen Journalism: Power to the People? Or a Traditional Journalist’s Wo…Graduation, fall 2009 [...]

  2. Video: Hacks and Hackers Hack Day Manchester | Scraperwiki Data Blog - October 17, 2010

    [...] Go to ScraperWiki.com → ← Hacks and Hackers Hack Day Manchester: Tweeting police, local gigs and Preston’s summer… [...]

  3. Scraperwiki/RBI launch first in-house Hacks & Hackers event – for B2Bs | Scraperwiki Data Blog - October 19, 2010

    [...] Past hacks and hackers days have run in London, Liverpool, Birmingham and Manchester. For a flavour of the projects please see this blog post. [...]

  4. #hhhmcr Hacks & Hackers Hack Day, Manchester - October 22, 2010

    [...] Digital Editors Network. ScraperWiki have done an extensive run through of the day’s events on their blog, and Andy Dickinson has also written about Friday’s goings [...]

  5. Tweet success for Greater Manchester Police? « Public-i Blog - October 24, 2010

    [...] GMP’s 24 hours. (UPDATE: Well, someone has already made it easily digestible. Working at the Scraperwiki Manchester Hacks and Hackers Day, Enrico Zini and Yuwei Lin turned a dataset of tweets assembled by the Guardian‘s Michael [...]

  6. Manchester Police tweets and the MEN – local data journalism part 2 | Online Journalism Blog - November 10, 2010

    [...] at a Hacks & Hackers Day last week, which the Manchester Evening News took part in. The results of that can be read on the Scraperwiki blog and on Andy Dickinson’s blog. These included: “David Kendal produced his own [...]

  7. Plings - Blog - Things to do when the #uksnow arrives - November 25, 2010

    [...] Closures Information We met James and Anna at the recent Hacks and Hackers event last month. They both work for Bauer Media, who run several large commercial radio stations [...]

  8. Bold steps in entrepreneurial journalism « Ink Think - March 10, 2011

    [...] the Hacks and Hackers events run by ScraperWiki have shown me is that great things can happen when journalism geeks and [...]