History & Background

Origins

Heurist was born in 2005, in the Arts eResearch team at the University of Sydney. It arose to meet a pressing need in Humanities research. Humanities researchers typically work with sparse, highly interrelated and complex data. But this kind of data is unsuitable for most databasing systems, which are designed for scientific or commerical applications. Heurist redressed this problem with its innovative design philosophy. It concealed the underlying technology from the researcher, empowering the researcher to create, test and modify their data structures easily with no programming. Through this simple yet profound innovation, Heurist eliminated the major barriers towards Humanities databasing.

This innovation was possible due to the insight of Dr Ian Johnson, Heurist’s designer-in-chief, and the engineering skills of Artem Osmakov, who remains Heurist’s chief developer. Before Heurist, Ian had designed two earlier Humanities databasing systems, Minark (1980-1987) and TimeMap (1997-2005). He had founded the University of Sydney’s Archaeological Computing Laboratory, which later morphed into the Arts eResearch team, and would go on to be a founding member of the Australasian Association of Digital Humanties. Through these decades of experience, Ian developed the ten core design principles that make Heurist such a successful research tool in the Humanities.

Today, Heurist is a global institution. In its early years, it quickly spread across Australia, powering major projects such as HuNI, FAIMS, The Dictionary of Sydney, Digital Harlem and Beyond 1914. It subsequently expanded across the world. There are Heurist servers running in Greece and Germany, and the system has been installed as a ‘third party service’ on Huma-Num, the centralised computing infrastructure for Humanities research in France. Heurist hosts more than 1500 databases in Australia, Europe and the Americas, and the Heurist network continues to grow daily.

Philosophy

Humanities data is different. In business and the sciences, data tends to be voluminous, homogeneous and indepdendent. An astronomer might take millions of separate observations of the night sky. A climatologist might run millions of separate iterations of a climate model. A bank might process millions of separate transations moving money from one account to another.

In the Humanities, by contrast, data tends to be sparse, heterogeneous and highly interdependent. A historian might wish to reconstruct the web of relations between several hundred people, the letters they wrote and the places they visited. A literary scholar might wish to reconstruct the web of characters, events, motifs and references that form a literary text. An archaeologist might wish to record each item from a dig, along with precise information about its unique form, location and meaning. These kinds of data are virtually impossible to model in conventional databasing systems without a high level of coding ability and an exceptional level of forward planning.

These unique features of Humanities data lead directly to Heurist’s ten core design prinicples:

1. Data structures stored in the database itself
2. Data structures configurable by the user through the interface
3. Change of record structure on the fly without loss of data
4. Flexible record structure allowing missing and repeated fields
5. Dynamic construction of data entry forms from structure definitions
6. Variable length free text
7. Enumerated and lookup fields
8. Enumerated lists extended on the fly
9. Import and export of csv and self-documentating database dumps
10. Built-in database publication tools, including a website builder, timemapping tool, network visualisation and custom report builder

All these design prinicples serve the same purpose: to get the technology out of the way. In a conventional databasing system such as Access or FileMaker, the researcher needs to solve the problem of how to represent their data in the form of a database. Heurist solves this problem for you, and largely eliminates the need for ‘data modelling’ in the traditional sense. Instead of grappling with the databasing software, the researcher is free to structure, explore and visualise their data in an intuitive way, without need to worry about how to convert the complex world of culture and society into a system of ‘tables’ (Access, MySQL, PostGres, FileMaker), ‘documents’ (MongoDB) or ‘nodes’ (Neo4J).

These principles had preoccupied Ian for 25 years before the creation of Heurist in 2005. His first system, Minark, was designed to make it easier for archaeologists to record information about their worksites. His second major system, TimeMap, was designed to make it easier for scholars to visualise historical data in space and time. By the time he came to design Heurist, he had carefully refined his design philosophy through decades of practice. He continues to refine this philosophy today, and works with chief developer Artem Osmakov, a veteran of TimeMap, to improve Heurist’s systems.

Technology

Although Heurist aims to insulate the user from the underlying technology, users can be assured that Heurist relies on the most robust and time-tested technology in the business. Heurist runs on the LAMP Stack, which at the time of writing remains by far the most popular ‘technology stack’ on the web. LAMP relies entirely on free, open-source and exceptionally robust technologies:

Heurist’s online interface also relies largely on JQuery, another popular open-source web technology that powers large portions of the web.

Because of its reliance on popular technology, Heurist’s systems are guaranteed to last in the long term. Thousands of developers across the world continually improve and update each component of the LAMP stack, and the technology will continue to function long into the future.

Because the technology is open-source, Heurist is also able to adopt a highly open and sustainable business model. Every component of the LAMP stack is free to use, and the source code can be freely accessed online. Accordingly, we have been able to release all of Heurist’s own source code on Github, and it is possible for anyone anywhere to download Heurist’s software and install, rebuild or modify it from scratch.

In 2021, Heurist founded a Technical Steering Group, comprising a core group of technically experienced scholars, developers and administrators who guide the future development of the software. Heurist is increasingly decentralised, with developers and server adminstrators from across the world contributing to its codebase and design.