ELKI: Environment for Developing KDD-Applications Supported by Index-Structures

Quick Summary

ELKI is an open source (AGPLv3) data mining software written in Java. The focus of ELKI is research in algorithms, with an emphasis on unsupervised methods in cluster analysis and outlier detection. In order to achieve high performance and scalability, ELKI offers data index structures such as the R*-tree that can provide major performance gains. ELKI is designed to be easy to extend for researchers and students in this domain, and welcomes contributions of additional methods. ELKI aims at providing a large collection of highly parameterizable algorithms, in order to allow easy and fair evaluation and benchmarking of algorithms.

Background

Data mining research leads to many algorithms for similar tasks.
A fair and useful comparison of these algorithms is difficult due to several reasons:

On the other hand, efficient data management tools like index-structures can show considerable impact on data mining tasks and are therefore useful for a broad variety of algorithms.

In ELKI, data mining algorithms and data management tasks are separated and allow for an independent evaluation. This separation makes ELKI unique among data mining frameworks like Weka or Rapidminer and frameworks for index structures like GiST. At the same time, ELKI is open to arbitrary data types, distance or similarity measures, or file formats. The fundamental approach is the independence of file parsers or database connections, data types, distances, distance functions, and data mining algorithms. Helper classes, e.g. for algebraic or analytic computations are available for all algorithms on equal terms.

With the development and publication of ELKI, we humbly hope to serve the data mining and database research community beneficially. The framework is free for scientific usage (“free” as in “open source”, see ELKI license for details). In case of application of ELKI in scientific publications, we would appreciate credit in form of a citation of the appropriate publication (see ELKI publications), that is, the publication related to the release of ELKI you were using.

The people behind ELKI are documented on the team page.

The ELKI wiki: Tutorials, HowTos, Documentation

Beginners may want to start at the HowTo documents, Examples, Tutorials, and FAQ to help with difficult configuration scenarios and beginning with ELKI development.

The most important documentation pages are: Tutorial, searchable JavaDoc, FAQ, InputFormat, DataTypes, DistanceFunctions, DataSets, Development, Parameterization, Visualization, Benchmarking, and the list of Algorithms and publications implemented in ELKI.

Getting ELKI: Download and Citation Policy

You can download ELKI including source code on the releases page. ELKI uses the AGPLv3 license, a well-known open source license.

ELKI is available on GitHub and Maven. You can clone an example project to get started.

<!-- https://mvnrepository.com/artifact/de.lmu.ifi.dbs.elki/elki -->
<dependency>
    <groupId>de.lmu.ifi.dbs.elki</groupId>
    <artifactId>elki</artifactId>
    <version>0.7.1</version>
</dependency>

There is a list of publications that accompany the ELKI releases. When using ELKI in your scientific work, you should cite the publication corresponding to the ELKI release you are using, to give credit. This also helps to improve the repeatability of your experiments. We would also appreciate if you contributed your algorithm to ELKI to allow others to reproduce your results and compare with your algorithm (which in turn will likely get you citations). We try to document every publication used for implementing ELKI: the page related publications lists over 220 publications that we used for implementing ELKI, and is generated from annotations in the source code. Also we list publications that used or cited ELKI, see references.

Compiling ELKI

ELKI is compiled using Maven and Python. The compilation process is explained here.

Information on ELKI APIs and coding styles is collected at the development starting page. Please contribute!

Efficiency Benchmarking with ELKI

ELKI is fast (see some of our benchmark results) but the focus lies on a broad coverage of algorithms and variations. We discourage cross-platform benchmarking, because it is easy to produce misleading results by comparing apples and oranges. For fair comparability, you should implement all algorithms within ELKI, and use the same APIs. We have also observed Java JDK versions have a large impact on the runtime performance. To make your results reproducible, please cite the publication of the version, which you have been using.

Bug Reports and Contact

You can browse the open bug reports and create new issues.

We also appreciate any comments, suggestions and code contributions.
You can contact the core development team by e-mail: elki () dbs ifi lmu de

You can also subscribe to the user mailing list of ELKI, to exchange questions and ideas among other users or to get announcements (e.g., new releases, major changes) by the ELKI team. If the web interface is still inaccessible, you can also subscribe via elki-user-request () lists ifi lmu de (use subject “subscribe” or “unsubscribe”).

Our primary “support” medium is this community mailing list. We appreciate if you share experiences and also success stories there that might help other users. This project makes a lot of progress, and information can get outdated rather quickly. If you prefer a web forum, you can try asking at StackOverflow, but you should understand that this is a general (and third-party operated) programming community; questions just on how to use it may be considered off-topic there.

Design Goals

Fork me on GitHub