Clustering art

Jacobus J Barnard, Pinar Duygulu, David Forsyth

Research output: Chapter in Book/Report/Conference proceedingConference contribution

103 Citations (Scopus)

Abstract

We extend a recently developed method for learning the semantics of image databases using text and pictures. We incorporate statistical natural language processing in order to deal with free text. We demonstrate the current system on a difficult dataset, namely 10,000 images of work from the Fine Arts Museum of San Francisco. The images include line drawings, paintings, and pictures of sculpture and ceramics. Many of the images have associated free text whose varies greatly, from physical description to interpretation and mood. We use WordNet to provide semantic grouping information and to help disambiguate word senses, as well as emphasize the hierarchical nature of semantic relationships. This allows us to impose a natural structure on the image collection, that reflects semantics to a considerable degree. Our method produces a joint probability distribution for words and picture elements. We demonstrate that this distribution can be used (a) to provide illustrations for given captions and (b) to generate words for images outside the training set. Results from this annotation process yield a quantitative study of our method. Finally, our annotation process can be seen as a form of object recognizer that has been learned through a partially supervised process.

Original languageEnglish (US)
Title of host publicationProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Volume2
StatePublished - 2001
Externally publishedYes
Event2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Kauai, HI, United States
Duration: Dec 8 2001Dec 14 2001

Other

Other2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
CountryUnited States
CityKauai, HI
Period12/8/0112/14/01

Fingerprint

Semantics
Museums
Painting
Probability distributions
Processing

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Software
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Cite this

Barnard, J. J., Duygulu, P., & Forsyth, D. (2001). Clustering art. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. 2)

Clustering art. / Barnard, Jacobus J; Duygulu, Pinar; Forsyth, David.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2 2001.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Barnard, JJ, Duygulu, P & Forsyth, D 2001, Clustering art. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. vol. 2, 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, United States, 12/8/01.
Barnard JJ, Duygulu P, Forsyth D. Clustering art. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2. 2001
Barnard, Jacobus J ; Duygulu, Pinar ; Forsyth, David. / Clustering art. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2 2001.
@inproceedings{e4435f003e67444d99d556ed6866ed81,
title = "Clustering art",
abstract = "We extend a recently developed method for learning the semantics of image databases using text and pictures. We incorporate statistical natural language processing in order to deal with free text. We demonstrate the current system on a difficult dataset, namely 10,000 images of work from the Fine Arts Museum of San Francisco. The images include line drawings, paintings, and pictures of sculpture and ceramics. Many of the images have associated free text whose varies greatly, from physical description to interpretation and mood. We use WordNet to provide semantic grouping information and to help disambiguate word senses, as well as emphasize the hierarchical nature of semantic relationships. This allows us to impose a natural structure on the image collection, that reflects semantics to a considerable degree. Our method produces a joint probability distribution for words and picture elements. We demonstrate that this distribution can be used (a) to provide illustrations for given captions and (b) to generate words for images outside the training set. Results from this annotation process yield a quantitative study of our method. Finally, our annotation process can be seen as a form of object recognizer that has been learned through a partially supervised process.",
author = "Barnard, {Jacobus J} and Pinar Duygulu and David Forsyth",
year = "2001",
language = "English (US)",
volume = "2",
booktitle = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

}

TY - GEN

T1 - Clustering art

AU - Barnard, Jacobus J

AU - Duygulu, Pinar

AU - Forsyth, David

PY - 2001

Y1 - 2001

N2 - We extend a recently developed method for learning the semantics of image databases using text and pictures. We incorporate statistical natural language processing in order to deal with free text. We demonstrate the current system on a difficult dataset, namely 10,000 images of work from the Fine Arts Museum of San Francisco. The images include line drawings, paintings, and pictures of sculpture and ceramics. Many of the images have associated free text whose varies greatly, from physical description to interpretation and mood. We use WordNet to provide semantic grouping information and to help disambiguate word senses, as well as emphasize the hierarchical nature of semantic relationships. This allows us to impose a natural structure on the image collection, that reflects semantics to a considerable degree. Our method produces a joint probability distribution for words and picture elements. We demonstrate that this distribution can be used (a) to provide illustrations for given captions and (b) to generate words for images outside the training set. Results from this annotation process yield a quantitative study of our method. Finally, our annotation process can be seen as a form of object recognizer that has been learned through a partially supervised process.

AB - We extend a recently developed method for learning the semantics of image databases using text and pictures. We incorporate statistical natural language processing in order to deal with free text. We demonstrate the current system on a difficult dataset, namely 10,000 images of work from the Fine Arts Museum of San Francisco. The images include line drawings, paintings, and pictures of sculpture and ceramics. Many of the images have associated free text whose varies greatly, from physical description to interpretation and mood. We use WordNet to provide semantic grouping information and to help disambiguate word senses, as well as emphasize the hierarchical nature of semantic relationships. This allows us to impose a natural structure on the image collection, that reflects semantics to a considerable degree. Our method produces a joint probability distribution for words and picture elements. We demonstrate that this distribution can be used (a) to provide illustrations for given captions and (b) to generate words for images outside the training set. Results from this annotation process yield a quantitative study of our method. Finally, our annotation process can be seen as a form of object recognizer that has been learned through a partially supervised process.

UR - http://www.scopus.com/inward/record.url?scp=0035693999&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0035693999&partnerID=8YFLogxK

M3 - Conference contribution

VL - 2

BT - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

ER -