Heurist’s research-driven data management system allows any confident researcher or data manager to design, create, manage, analyse and publish their own richly-structured database(s) within hours, through a simple web interface, without the need of programmers or consultants.
Use Sydney University’s free service (click the Try It button) to build a new database in minutes and a complete application in hours. Design and build incrementally; existing data are not affected. Nor are you locked in; migrate later to a personal or institutional server, or access data directly via SQL.
Whatever your field of study – History, Archaeology, Anthropology, Cultural Research, Linguistics, Media Studies etc. – Heurist handles everything from text, tags and categories, through networks of rich relationships and offset annotations, to images, video, spatial data and dating (including non-western systems and dating uncertainty).
Get results: Sophisticated searches (including faceted and rule-based filters) can be saved, organised and rendered as layered interactive maps, timelines, network visualisations, cross-tabulations, lists, custom reports, XML and XSLT transforms, all of which can easily be printed or embedded (live) in a website.
Designed by researchers, for researchers, Heurist reduces complex underlying decisions to simple, logical choices.
Heurist’s intuitive web-based interface is uniquely tailored to the data needs of Humanities scholars, rather than to business or scientific data. The ability to import, and then adapt, a library of useful structures (templates) developed by the Heurist Team and other users is both a great time-saver and a powerful force for standardisation without compulsion. Annotated forms make it easy to build your data structure, add and edit data,
Heurist resolves the problem that
As an alternative to either building a dedicated database (time-consuming and expensive) or knocking together an ad-hoc solution using whatever tools are to hand (creating a ‘silo’ environment), Heurist offers researchers an integrated, free and Open Source single web-based tool to handle all their data.
Without investing in technical expertise, researchers can use Heurist’s simple forms (with tips) to rapidly build their own, complex databases with a host of advanced features not normally available without special programming, including the ability to: easily create new entity types and relationships (record linking); add and modify fields; manage term lists; import, edit (including bulk edit) and verify data; access information from anywhere and share data (selectively) with colleagues; visualise data through automatic mapping, timelines and network diagrams; modify structures record on-the-fly; and export/publish data.
And users can quickly get started by leveraging off existing work, through the reuse of preconfigured data schemas (templates) created by other users and stored
To get started, you can use the free service hosted at the University of Sydney Data Centre (heurist.sydney.edu.au) or one of the other listed services, or set up Heurist on your own server or cloud service in a couple of hours. Databases are easily migrated between servers.
Heurist’s set of powerful, robust and adaptable features extends its application to a wide range of scenarios and provides scope for multiple applicabilities. From a single system, Heurist users have the tools and flexibility to create a wide-range of solutions, only restricted by their imagination.
The following shows a sub-set of current and planned uses:
A text and media-rich public encyclopaedia of historical content for a city, with maps and timelines. | A visual representation of a city’s historic crimes and criminals. | A record of ballads, pamphlets, tunes and people. | Information on buildings, people and events on campus for use in an undergraduate course. | A bug tracking system. | A rental database. | A database of photographs from an archaeological survey of a country. | An interlinked set of descriptions for artists, paintings, collections, and villages for a country. | A history of people, sports, events across a single culture & city. | A visually-rich collection of medieval menus, recipes and ingredients.| Excavation phases, contexts, structures, finds, and type specimens. | Field survey and excavation data collected via Android tablets. | Records of educational service providers and their services. | A central repository of all kinds of information about cities with the ability to freely link and build as new data becomes available. | Interlinked life histories of WW1 soldiers, with rich photographic and documentary material, and timelines. | An archaeological record of Alpine sites. | A collection of photographs and videos of historic agricultural practices still predominately being performed in the Middle East, including a record of the types of activity, the landscape and the social context. | A collection of photographs synchronised with Filmmaker and Lightroom databases, making them available for download on-line. | A collection of relevant research information about the boundaries of Ancient Mesopotamia. | Index of historic maps of a major city. | Annotation and linking of historical texts. | Linked annotation and publication of literary work. | Artwork collections. | Medieval menus. | Field data collection via Android tablets. | Site survey data. | Complex stratified excavation data. | Content-rich life histories and connections. | Event records | Bibliographic, administrative and job-tracking databases.
Heurist’s complete database-design environment, incorporating sophisticated database facilities, lets you manage complex Humanities data and its structure in an evolving manner. Get started immediately with something small and simple, and then modify and extend your database structure as you become more comfortable with it or your data model and needs evolve.
New databases take only a few minutes to set up, although an investment in downloading well-designed data structures (templates) and in fine-tuning these (e.g. eliminating unneeded fields, setting field labelling, field order, field widths, field requirements and repetition of values), will lead to a better user experience.
And you can continue to evolve your database design at any time without affecting your existing data.
Heurist’s import feature takes you step-by-step through the import of structured tabular data, while accommodates a range of sources and different data sources: CSV, SQL, KML, Bibliographic, Media, with entity matching on one or more key fields and merging of data.
Large volumes of images and other multimedia files can be quickly loaded in bulk and indexed. In addition, some forms of data can be synchronised; for example, Zotero bibliographies can be synchronised to connect objects of study with full bibliographic references.
Heurist supports the import of KML (Keyhole Markup Language) data, a file format used to display geographic data in an Earth browser such as Google Earth.
From simple text searches to faceted searches and rulesets for retrieving a network of connections, Heurist provides numerous options for searching and filtering databases. Search filters can be saved and re-used to analyse or publish data subsets.
A quick search tool assists you in building editable filter expressions, to target particular record types, fields and values from controlled lists, without needing to know the required syntax. Refine your search criteria
Additionally, Heurist lets you expand/refine the search result-set through:
Rulesets: Expand the search result-set through one or more rules, each of which describes the set of pointers and relationships (including reverse pointers) to follow from each of the records in the initial result-set in order to add related records to it. A rule can comprise several steps out from the initial result-set.
Faceted Searches. Selectively refine the result-set based on selected data properties.
Filters (including rules and access settings), Rulesets and Faceted Searches can be saved and reused, for grouping and sharing records for particular purposes.
Analysis & Visualisation
Rule-based network expansion, cross-tabulation analysis, interactive maps, timelines and network graphs provide
Heurist has a rich-set of tools for analysing your data structure and data.
You can visualise the structure (schema) of your database, in both list form and as a network diagram. The network diagram provides an interactive (spring-loaded) visualisation of your database’s record types. Record types are shown as nodes, and the connections (pointer fields and relationships) as the links between nodes (edges). You can update links (relationships) directly via the diagram by simply drawing new connections. Data result-sets can also be visualised through a data network diagram.
A map & timeline view shows geospatial and timeline data stored for ‘map documents’ attached to one or more records.
Heurist provides basic cross-tabulation (of up to three fields, with counts and percentages) which can be applied to the result-set, allowing segmentation of the data.
Data can easily be exported from Heurist in a variety of text formats, including comma and tab separated files and GEPHI GEFX. Custom analysis functions can also be added to the interface with PHP programming.
Whilst securily maintaining your personal and private information, Heurist provides an integrated workgroup environment for sharing and collaborating online with other users, through workgroups, sharing, tagging, saved filters, messaging, record-level ownership and publishing control.
In addition to being used by individual researchers who wish to create and use a database for their own projects, Heurist’s shared collaborative knowledge-space allows project members to share an overlapping set of records in a single database (securely available online) and communicate with one another around those records through a range of communication channels:
By setting up workgroups and adding members to workgroups, Heurist can be used for group projects, with tasks divided between those designing the database and those populating and searching within it. Members can collaborate on research projects by sharing research data/functions with their colleagues (i.e. fellow members of the workgroup) through a range of collaboration tools. Members can display a subset of record types for specific groups of users, protected from modification or viewing by non-members.
Within shared databases, researchers can ‘bookmark’ sets of records (their own records and other user records of relevance) to develop a personal subset of the shared database. Bookmarks are a way of separating private information about a record and shared information about a record; for any bookmarked record, you can attach information, either for your own personal use or for sharing. And you can search specifically within just bookmarked records, to quickly find commonly used records.
Bookmarks incorporate many useful features:
- Add and (optionally) share personal notes.
- Engage in a discussion with other workgroup members.
- Rate records (sortable in search results).
- Join group discussions per record.
- Leave comments for others to reply to.
- Add web links, web pages, images and geographic location to shared records.
- Attach personal password reminders to bookmarks for password protected resources.
- Notify other users of records that might be of interest to them. Or attach email reminders to shared records to notify a user or workgroup about multiple records of interest at specified intervals (specific date, daily, weekly, monthly, yearly).
Set workgroup tags (tag records with meaningful, multiple search keys) and share these among workgroup members. Tag creation
Record-level ownership and publishing control
Control workgroup members’ access and permissions, as well as access to published reports.
Use Heurist’s Smarty-based Report Builder to define custom reports to be printed or published in a variety of formats, including as live-feeds to a web, CMS or blogging site; Heurist even writes the HTML/JS code for you to paste into your web page editor, with no programming required. XML output can be transformed inline through stylesheets.
Reports can be run on demand or based on a user-defined schedule (which can be used to set up a call for writing reports to file: HTML and JS.
Sustainability & Access
Sustainability of complex digital resources is a critical issue to anyone committing their research data to digital format. To this end, data is stored in a fixed format in the most widely used Open Source SQL database, allowing independent access into the future through any programming tools. Heurist also supports fully-documented generic SQL queries, XML feeds, XSL
Heurist databases have high sustainability—of the order of decades—for the following reasons:
1. Heurist is built on MySQL, the most widely used Open Source relational database server today (used by many major applications including WordPress, Drupal, Wikipedia, Facebook, Twitter, Tumblr, Flickr, YouTube among others).
2. All Heurist databases have an identical MySQL structure, which is internally documented with consistent field-naming conventions and a concise and informative comment section on every field.
3. The structure, data and logic of a database built in Heurist is built into the database itself, not the software; the structure, data and logic in a Heurist database are fully self-documenting. This contrasts with typical database applications where the database structure maps to the domain of the application and the application software is essential to using, if not interpreting, the content. Such applications are inherently unsustainable and probably have an unmaintained life of less than 5 years. The data in Heurist databases will be accessible and comprehensible/usable via standard SQL as long as MySQL databases are readable, which is likely to be a matter of decades.
One-click creation of sustainable archive package: Database owners/managers can export the complete contents of a database to an archive package, consisting of: a text file containing a MySQL database dump (the database rendered in SQL statements) which can be reloaded into any recent MySQL server (or other SQL server with a bit of work); a text file containing an XML rendering of the database content; any uploaded files (text, images, videos, spreadsheets etc.) in their original format; a textual description of the Heurist database structure.
Ancient History, Annotation, Annotation of Records, Archaeological Data, Archaeology, Art, Art History, Artifact, Artworks, Audio, Bibliographic Listing, Bibliographic Record, Bibliography, Biographical Data, Biography, Blog Writing, Bookmarklet, Central Index of Databases, Classical Studies, Collection, Connections, Cultural and Visual Studies, Data, Data Collection, Data File, Data Import, Data Modeling, Database, Digital and Physical Archival Collection, Digital Documents, Digital Humanities, Digitisation, Directed Relationships, End-user Structure Modification, Entity Extraction, Events, Excavation, Excavation Data, External Message Trigger, Faceted Search, Feeds, Field Data, File, Film Studies, Flexible Database Structure, Fuzzy Date, Fuzzy Date Value, Geographic Coordinates, Geographic Objects, Geospatial Data, Historical Data, Historical Map, History, History, Hyperlinks, Image Annotation, Image Management, Image Resizing, Images, Interactive Network Visualisation, Internal Messaging, Laboratory Data, Link, Literature, Manuscript, Map, Map Layers, Map with Timeline, Mapping, Media Studies, Metadata, Monument, Multimedia, Network, Network Visualization, Objects, Offset Text Annotation, Organisations, PDF, Persons, Photography, Place, Project, Prosopography, Relationship, Research, Rich Field Type, Rule-based Network Search, Simple and Advanced Search, Sites, Spatial Data, Spreadsheet, Structure Import, Structures, Survey Data, Tablet, Tablet Data Collection, Tagging, Task, Text, Time-stamped Relationship, Timeline, Tools, User-defined Report, Video , Web Publishing, Workgroup, Workgroup Ownership, Workgroup Permission, Workgroup-based Saved Search, XML, XML Output, XSLT, XSLT Inline Transform