Heurist’s research-driven data management system allows any confident researcher or data manager to design, create, manage, analyse and publish their own richly-structured database(s) within hours, through a simple web interface, without need of programmers or consultants.
Use Sydney University’s free service (click the Try It button) to build a new database in minutes and a complete application in hours. Design and build incrementally; existing data are not affected. Nor are you locked in; migrate later to a personal or institutional server, or access data directly via SQL.
Whatever your field of study – History, Archaeology, Anthropology, Cultural Research, Linguistics, Media Studies etc. – Heurist handles everything from text, tags and categories, through networks of rich relationships and offset annotations, to images, video, spatial data and dating (including non-western systems and dating uncertainty).
Get results: Sophisticated searches (including faceted and rule-based) can be saved, organised and rendered as layered interactive maps, timelines, network visualisations, cross-tabulations, lists, custom reports, XML and XSLT transforms, all of which can easily be printed or embedded (live) in a website.
Designed by researchers, for researchers, Heurist reduces complex underlying decisions to simple, logical choices.
Heurist’s ability to import, and then adapt, a library of useful structures developed by other users is both a great time-saver and a powerful force for standardisation without compulsion. Simple forms with helpful tips provide for the easy creation of new entity types and relationships, addition and modification of fields, addition of new terms, data verification, bulk edits, data import and data export.
PROBLEM: You need: a proper database, several entity types, relationships between entity types, full control of your information, access from multiple locations, selective sharing with colleagues, compatibility with standards, sustainable and archivable data, free and open source. You need to be able to: change database structure at a moment’s notice, import existing data, reference external data, have assistants safely enter data, publish selectively to a website, map and timeline the contents, network graph the content, do simple (cross)tabulation, and export data for analysis and visualisation. And: you don’t have a technical team, you don’t have much/any funding, and you need it today!
SOLUTION: Heurist resolves the problem that eResearchers have typically faced in managing their complex research data coming from different sources and in different forms (e.g. bibliographies, research notes, images, map data etc.).
As an alternative to either building a dedicated database (time-consuming and expensive) or knocking together an ad-hoc solution using whatever tools are to hand (creating a ‘silo’ environment), Heurist offers researchers an integrated, free and Open Source single web-based tool to handle all of your data, allowing you to build your own, complex databases without investing in technical expertise.
Any researcher can now rapidly create a simple database in minutes and a complex one in hours; a database with a host of advanced features that would not have been available without special programming, such as the ability to: reuse data models created by other users; publish information to the web; access information from anywhere; share data (selectively) with colleagues; and visualise data through maps, timelines and network charts.
Simple forms provide helpful tips that allow for the easy creation of new entity types and relationships, addition and modification of fields, addition of new terms, data verification, bulk edits, data import and data export.
And Heurist incorporates features such as repeatable fields, simple creation and use of relationships (record linking), automatic mapping and timelines, on-the-fly record structure change, and sharing of pre-configured database schemas through a central clearing-house.
You can use the free service hosted at the University of Sydney Data Centre (heurist.sydney.edu.au) or one of the other listed services, or you set up Heurist on your own server or cloud service in a couple of hours. Databases are easily migrated between servers.
Example uses (current and planned) include:
A text and media-rich public encyclopaedia of historical content for a city, with maps and timelines. | A visual representation of a city’s historic crimes and criminals. | A record of ballads, pamphlets, tunes and people. | Information on buildings, people and events on campus for use in an undergraduate course. | A bug tracking system. | A rental database. | A database of photographs from an archaeological survey of a country. | An interlinked set of descriptions for artists, paintings, collections, and villages for a country. | A history of people, sports, events across a single culture & city. | A visually-rich collection of medieval menus, recipes and ingredients.| Excavation phases, contexts, structures, finds, and type specimens. | Field survey and excavation data collected via tablets. | Records of educational service providers and their services. | A central repository of all kinds of information about cities with the ability to freely link and build as new data becomes available. | Interlinked life histories of WW1 soldiers, with rich photographic and documentary material, and timelines.| An archaeological record of Alpine sites. | A collection of photographs and videos of historic agricultural practices still predominately being performed in the Middle East, including a record of the types of activity, the landscape and the social context. | A collection of photographs synchronised with Filmmaker and Lightroom databases, making them available for download on-line. | A collection of relevant research information about the boundaries of Ancient Mesopotamia. | Index of historic maps of a major city. | Annotation and linking of historical texts. | Linked annotation and publication of literary work. | Artwork collections. | Medieval menus. | Field data collection via Android tablets. | Site survey data. | Complex stratified excavation data. | Content-rich life histories and connections. | Event records | Bibliographic, administrative and job-tracking databases.
Incremental Heurist’s complete database-design environment, incorporating leading-edge database facilities, lets you manage complex Humanities data and its structure, in an evolving manner. Get started immediately with something small and simple, and then modify and extend your database structure as you become more comfortable with it or your data model and needs evolve.
New databases take only a few minutes to set up, although an investment in downloading well-designed data structures and in fine-tuning these by eliminating unneeded fields, setting field labelling, field order, field widths, field requirements and repetition of values will lead to a better user experience.
And you can continue to evolve your database design at any time without affecting your existing data.
Existing Data You can easily import, export and manage different types of data. Heurist’s import/export tools support many formats, including CSV, KML, Bibliographic, Media.
Heurist supports the import of KML (Keyhole Markup Language) data, a file format used to display geographic data in an Earth browser such as Google Earth.
Sophisticated Search From simple text searches to faceted searches and rules for retrieving a network of connections, Heurist provides numerous options for searching and filtering databases. Searches can be saved and re-used to analyse or publish data subsets.
A quick search tool assists you in building complex, editable filter expressions, to target particular record types, fields and values from controlled lists, without needing to know the required syntax. Refine your search criteria by:
- Entering partial strings to match against record titles, tags, keywords, special fields, or all fields.
- Using comparison operators (e.g. Greater Than) on numeric fields and dates, including relative to ‘today’.
- Searching for records of specified types or bookmarked by specific users across databases.
- Sorting records by text, date, popularity, rating, date of addition and modification, specific fields etc.
To build more complex search strings use the advanced ‘Assisted Search’ dialog. A complete search syntax is provided. Expand your search by building RulesSets, or refine the search through faceted searches.
The result of any search can be saved and reused, letting you create personal searches for grouping records for particular purposes (saved searches can be shared with any workgroups of which you are a member). Saved searches can also store your filter rules and ownership settings.
Heurist has a rich-set of tools for analysing your data.
Data result-sets can be visualised through a data network diagram that provides an interactive (spring-loaded) visualisation of the (displayed) result-set. Records are shown as nodes, and the connections (pointer fields and relationships) as the links between nodes (edges). You can update links direclty via the diagram.
Heurist also lets you visualise the structure (schema) of your database, in both list form and as a network diagram.
A map & timeline view shows geospatial and timeline data stored for ‘map documents’ attached to one or more records).
Heurist provides basic cross-tabulation (of up to three fields, with counts and percentages) which can be applied to the result-set, allowing segmentation of the data.
Data can easily be exported from Heurist in a variety of text formats, including comma and tab separated files and GEPHI GEFX. Custom analysis functions can also be added to the interface with PHP programming.
Web Publishing Use Heurist’s Report Builder to define custom reports to be printed or published in a variety of formats: as live-feeds to a web, CMS or blogging site; Heurist even writes the HTML/JS code for you to paste into your web page editor, with no programming required. XML output can be transformed inline through stylesheets.
The Scheduled Reports function can be used to set up a call for writing reports to file (HTML and JS). Reports can be run on demand or based on a user-defined schedule.
Collaboration Heurist provides an integrated workgroup environment for sharing and annotating data. Collaborate online with other users through sharing, tagging, saved searches, messaging, record-level ownership and publishing control (while maintaining your personal tags, bookmarks, ratings & notes).
Heurist’s shared collaborative knowledge-space allows project members to share an overlapping set of records in a single database (securely available online) and communicate with one another around those records through a range of communication channels (shared text, tags, bookmarks, discussions, blogs, notations etc.), while individual users can privately manage their own personal tags, bookmarks, ratings, notes etc.
Heurist can be used for group projects, with tasks divided between those designing the database and those populating and searching it. Or it can be used by individual researchers who wish to create and use a database for their own project and who take on both tasks. Within shared database, researchers can ‘bookmark’ sets of records to maintain their own personal, private set of information.
You can share information with other users through:
- Setting up workgroups and adding members to workgroups. These members can collaborate on research projects by sharing research data/functions with their colleagues (i.e. fellow members of the workgroup) through a range of collaboration tools.
- Bookmarking records*. Bookmarks are a way of separating private information about a record and shared information about a record. For any bookmarked record (your own records or other user records of relevance) you can attach personal (private) information about the record (e.g. attach personal notes, ratings, discussions, web links, web pages, images and geographic location to records, either for your own personal use or for sharing).
- Displaying a subset of record types for a specific group of users, protected from modification or viewing by non-members.
- Overriding record types.
- Setting workgroup tags and sharing these between workgroup members. Tag creation includes multi-word tags, tag disambiguation, clickable lists of suggestions, instantly search by tags, including partial matching, follow tags to related records.
- Adding shared text. You can add extended notes to any record or engage in a discussion with other workgroup members.
- Sending email reminders. Notify other users of records that might be of interest to them.
*Bookmarks: Personal bookmarks in Heurist provide the following features:
- Quickly find commonly used records from the Saved Filters Pane.
- Develop a personal subset of the shared database.
- Tag records with meaningful, multiple search keys.
- Search records of similar interest based on common tags and follow tags to related records.
- Manage tags through popularity, deletion, combining and renaming.
- Add personal rich-text notes.
- Leave comments for others to reply to.
- Rate records, and sort search results by ratings.
- Store personal password reminders attached to bookmarks for password protected resources.
- Create personal saved searches, grouping bookmarked records for particular purposes.
- Send reminders to notify a user or workgroup about multiple records of interest at specified intervals (specific date, daily, weekly, monthly, yearly).
Sustainability & Access Sustainability of complex digital resources is a critical issue to anyone committing their research data to digital format. To this end, data is stored in a fixed format in the world’s most widely used Open Source SQL database, allowing independent access through any programming tools into the future. Heurist also supports fully-documented generic SQL queries, XML feeds, XSL transforms, and creates JSON, XML & SQL archive packages.
Heurist databases have high sustainability—of the order of decades—for the following reasons:
- Heurist is built on MySQL, the most widely used Open Source relational database server today (used by many major applications including WordPress, Drupal, Wikipedia, Facebook, Twitter, Tumblr, Flickr, YouTube among others).
- All Heurist databases have an identical MySQL structure, which is internally documented with consistent field-naming conventions and a concise and informative comment section on every field.
- The structure, data and logic of a database built in Heurist is built into the database itself, not the software; the structure, data and logic in a Heurist database are fully self-documenting. This contrasts with typical database applications where the database structure maps to the domain of the application and the application software is essential to using, if not interpreting, the content. Such applications are inherently unsustainable and probably have an unmaintained life of less than 5 years. The data in Heurist databases will be accessible and comprehensible/usable via standard SQL as long as MySQL databases are readable, which is likely to be a matter of decades.
One-click creation of sustainable archive package: Database owners/managers can export the complete contents of a database to an archive package, consisting of: a text file containing a MySQL database dump (the database rendered in SQL statements) which can be reloaded into any recent MySQL server (or other SQL server with a bit of work); a text file containing an XML rendering of the database content; any uploaded files (text, images, videos, spreadsheets etc.) in their original format; a textual description of the Heurist database structure.
Ancient History, Annotation, Annotation of Records, Archaeological Data, Archaeology, Art, Art History, Artifact, Artworks, Audio, Bibliographic Listing, Bibliographic Record, Bibliography, Biographical Data, Biography, Blog Writing, Bookmarklet, Central Index of Databases, Classical Studies, Collection, Connections, Cultural and Visual Studies, Data, Data Collection, Data File, Data Import, Data Modeling, Database, Digital and Physical Archival Collection, Digital Documents, Digital Humanities, Digitisation, Directed Relationships, End-user Structure Modification, Entity Extraction, Events, Excavation, Excavation Data, External Message Trigger, Faceted Search, Feeds, Field Data, File, Film Studies, Flexible Database Structure, Fuzzy Date, Fuzzy Date Value, Geographic Coordinates, Geographic Objects, Geospatial Data, Historical Data, Historical Map, History, History, Hyperlinks, Image Annotation, Image Management, Image Resizing, Images, Interactive Network Visualisation, Internal Messaging, Laboratory Data, Link, Literature, Manuscript, Map, Map Layers, Map with Timeline, Mapping, Media Studies, Metadata, Monument, Multimedia, Network, Network Visualization, Objects, Offset Text Annotation, Organisations, PDF, Persons, Photography, Place, Project, Prosopography, Relationship, Research, Rich Field Type, Rule-based Network Search, Simple and Advanced Search, Sites, Spatial Data, Spreadsheet, Structure Import, Structures, Survey Data, Tablet, Tablet Data Collection, Tagging, Task, Text, Time-stamped Relationship, Timeline, Tools, User-defined Report, Video , Web Publishing, Workgroup, Workgroup Ownership, Workgroup Permission, Workgroup-based Saved Search, XML, XML Output, XSLT, XSLT Inline Transform