What is it?
Lytic is a freely-available tool for exporatory data analyis. It is a spinoff of the Test Matrix Tool and thus was originally developed for analyzing large simulation datasets. We call it a scientific intelligence tool: similar to business intelligence tool, it provides insight into scientific data through visualization and filtering.
We have used it as both a production tool for engineering analysis efforts in EOSL and as a research platform with collaborators in the Georgia Tech School of Computational Science and Engineering (see our paper at the SIGKDD IDEA 2013 workshop).
What good is it?
Common data exploration activities include visualization, filtering and comparison. Lytic's core design is centered around making these things as easy as possible, which in turn makes iterative discovery and hypothesis testing more pleasurable and productive. We have found Lytic to be useful in a variety of sponsored and unsponsored contexts:
- Health data
- Manufacturing optimization
- Sports statistics
- Public records (e.g., data.gov)
How does it work?
Lytic ingests CSV data and caches it in an SQL database in a manner conducive to OLAP-style querying. It dynamically creates UI elements from the characteristics of the data themselves, and optimizes user data requests (visualization, filtering, etc.) to maximize UI update speed.
The UI has been designed with a particular focus on ease of end-user operation, so that obvious semantic operations on data (filtering, comparisons, mapping to simple visualizations) are straightforward.
The database cache can be either an embedded sqlite or more recently a column-store MonetDB database; the latter improves UI responsiveness dramatically for larger dataset sizes.