org.wandora.piccolo.utils.crawler

Class LuceneCrawler

java.lang.Object

org.wandora.piccolo.utils.crawler.LuceneCrawler

All Implemented Interfaces:

CrawlerAccess

public class LuceneCrawler
extends java.lang.Object
implements CrawlerAccess

Author:

olli

Field Summary

Fields

Modifier and Type	Field and Description
(package private) CrawlerAccess	crawler
private java.util.HashMap	properties
(package private) org.apache.lucene.index.IndexWriter	writer

Constructor Summary

Constructors

Constructor and Description
LuceneCrawler(CrawlerAccess crawler, org.apache.lucene.index.IndexWriter writer) Creates new LuceneCrawler

Method Summary

Modifier and Type	Method and Description
void	add(java.lang.Object crawlerObject, int depth) Adds an url to the queue of the crawler.
void	addObject(java.lang.Object data) Gives any object constructed from the crawled page to the call back object.
static org.apache.lucene.document.Field	content(java.lang.String data)
void	forceExit()
java.lang.Object	getProperty(java.lang.String key)
static org.apache.lucene.document.Field	keywords(java.lang.String data)
static org.apache.lucene.document.Field	location(java.lang.String data)
void	setProperty(java.lang.String key, java.lang.Object value)
static org.apache.lucene.document.Field	subject(java.lang.String data)
static org.apache.lucene.document.Field	title(java.lang.String data)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

properties

private java.util.HashMap properties

crawler

CrawlerAccess crawler

writer

org.apache.lucene.index.IndexWriter writer

Constructor Detail

LuceneCrawler

public LuceneCrawler(CrawlerAccess crawler,
                     org.apache.lucene.index.IndexWriter writer)

Creates new LuceneCrawler

Method Detail

add

public void add(java.lang.Object crawlerObject,
                int depth)

Description copied from interface: CrawlerAccess

Adds an url to the queue of the crawler.

Specified by:

add in interface CrawlerAccess

addObject

public void addObject(java.lang.Object data)

Description copied from interface: CrawlerAccess

Gives any object constructed from the crawled page to the call back object. The type of the given data can be anything and it is up to the CrawlerAccess implementation to decide what to do with it.

Specified by:

addObject in interface CrawlerAccess

keywords

public static org.apache.lucene.document.Field keywords(java.lang.String data)

title

public static org.apache.lucene.document.Field title(java.lang.String data)

subject

public static org.apache.lucene.document.Field subject(java.lang.String data)

content

public static org.apache.lucene.document.Field content(java.lang.String data)

location

public static org.apache.lucene.document.Field location(java.lang.String data)

forceExit

public void forceExit()

Specified by:

forceExit in interface CrawlerAccess

setProperty

public void setProperty(java.lang.String key,
                        java.lang.Object value)

Specified by:

setProperty in interface CrawlerAccess

getProperty

public java.lang.Object getProperty(java.lang.String key)