Goals and history of IR. The impact of the web on IR. The role of artificial intelligence (AI) in IR.
Boolean and vector-space retrieval models; ranked retrieval; text-similarity metrics; TF-IDF (term frequency/inverse document frequency) weighting; cosine similarity.
Simple tokenizing, stop-word removal, and stemming; inverted indices; efficient processing with sparse vectors; Java implementation.
Performance metrics: recall, precision, and F-measure; Evaluations on benchmark text collections.
Relevance feedback; Query expansion; Query languages.
Word statistics; Zipf's law; Porter stemmer; morphology; index term selection; using thesauri. Metadata and markup languages (SGML, HTML, XML).
Search engines; spidering; metacrawlers; directed spidering; link analysis (e.g. hubs and authorities, Google PageRank); shopping agents.
Categorization algorithms: Rocchio; naive Bayes; decision trees; and nearest neighbor. Clustering algorithms: agglomerative clustering; k-means; expectation maximization (EM). Applications to information filtering; organization; and relevance feedback.
Collaborative filtering and content-based recommendation of documents and products.
Extracting data from text; XML; semantic web; collecting and integrating specialized information on the web.