One of the most widely used methods of journalism today entails the use of huge repositories of data to produce visual stories. These repositories are created by “data mining” or “scraping”.
With the use of complex statistical algorithms, “data mining” is the automated search and analysis of patterns in large stores of data. The 3 basic areas of data mining are content mining (web scraping), usage mining (identifying and analysing patterns) and structure mining (relationship between website links and data).
“Scraping” is a more direct process where information is gathered from various sources of content on the internet such as blogs, news and government agency websites using one of many “scraping” techniques, such as the use of “bots” or “crawlers”.
Regardless of how the data is amassed, many software tools are available, some being around for decades, like Microsoft Excel, Google Sheets and Google Charts. Of the modern tools available, Tableau has become a leader in creating visual interpretations of big data.