Archive by Author

New ScraperWiki tool lets you extract data from reports with complete accuracy

It’s not always possible to automate data gathering, even with scrapers. Often we find customers want to regularly update data in ScraperWiki via spreadsheets. Either they’ve made the spreadsheets via a report from another system (typically one that isn’t on the web), or they gather the data by hand (for example, by phoning someone up […]

Sharing in 6 dimensions

Hands up everyone who’s ever used Google Docs. Okay, hands down. Have you ever noticed how many different ways there are to ‘share’ a document with someone else? We have. We use Google Docs at lot internally to store and edit company documents. And we’ve always been baffled by how many steps there are to […]

Open your data with ScraperWiki

Open data activists, start your engines. Following on from last week’s announcement about publishing open data from ScraperWiki, we’re now excited to unveil the first iteration of the “Open your data” tool, for publishing ScraperWiki datasets to any open data catalogue powered by the OKFN’s CKAN technology. Try it out on your own datasets. You’ll […]

Your questions about the new ScraperWiki answered

You may have noticed we launched a completely new version of ScraperWiki last week. Here’s a suitably meta screengrab of last week’s #scraperwiki twitter activity, collected by the new “Search for tweets” tool and visualised by the “Summarise this data” tool, both running on our new platform. These changes have been a long time coming, […]

Data analysis using the Query with SQL tool

Ferdinand Magellan, the Renaissance’s most prodigious explorer. He almost certainly knew lingua franca – but did he know SQL?It’s Summer 1513. Rome is the centre of the Renaissance world, and Spanish, Italian, and Portuguese merchant ships criss-cross the oceans, ferrying textiles from the North, spices from the East, and precious metals from the newly-discovered Americas. […]

It’s all about tools

The new ScraperWiki is all about tools. People talk a lot about data, big data, data mining, data science. But the action happens in tools. Tools like Excel, R, SPSS, Python. Or, on the new ScraperWiki, tools like View in a table, Summarise this data and Query with SQL. We’ve just pushed an improvement to […]

Programmers past, present and future

As a UX designer and part-time anthropologist, working at ScraperWiki is an awesome opportunity to meet the whole gamut of hackers, programmers and data geeks. Inside of ScraperWiki itself, I’m surrounded by guys who started programming almost before they could walk. But right at the other end, there are sales and support staff who only […]