快速重复数据删除引擎

Duke has moved to Github Duke is a fast and flexible deduplication (or entity resolution, or record linkage) engine written in Java on top of Lucene. The latest version is 1.1 (see ReleaseNotes). Features High performance. Highly configurable. Support for CSV, JDBC, SPARQL, and NTriples DataSources. Many built-in comparators. Plug in your own data sources, comparators, and cleaners. GeneticAlgorithm for automatically tuning configurations. Command-line client for getting started. API for embedding into any kind of application. Support for batch processing and continuous processing. Can maintain database of links found via JNDI/JDBC. Can run in multiple threads. Duke has moved to

duke-1.1
lucene-core-4.0.0.jar
lucene-analyzers-common-4.0.0.jar
lucene-spatial-4.0.0.jar
spatial4j-0.3.jar
lucene-queries-4.0.0.jar
doc
README.txt
duke-1.1-javadoc.jar
duke-1.1-sources.jar
duke-1.1.jar