Search system and database: QBIC color search, QBIC layout search.
-
1. Theme content analysis
In recent years, with the comprehensive popularization of computer networks, multimedia information retrieval has developed rapidly. Content-based image retrieval is to search according to the semantic and contextual relations of images and their contents, and to detect other images with similar characteristics from the image database with the semantic features of images as clues. Because the scale of images is generally larger than pure text information, content-based image retrieval requires higher retrieval speed and efficiency. At present, there are many content-based image retrieval systems applied in practical environments, such as the earliest commercial QBIC system developed by IBM, WebSeek system developed by Columbia University and Photobook system developed by MIT. To retrieve Web images through content-based technology, it is necessary to separate the images from the Web to form an image set, and to conduct content-based feature analysis and similarity matching for each object in the image set. Through the analysis and evaluation of the content-based multimedia information retrieval system developed by IBM, represented by QBIC system, this topic learns and grasps the most advanced images and image retrieval tools on the Internet at present, and then deeply understands the contents of multimedia information processing technology and retrieval technology.
2. Systematic investigation and analysis
The digital library scheme of IBM converts physical information into digital multimedia and sends it to users all over the world safely through the network. Almaden Research Center of IBM launched QBIC system. This system has created a new field of image information query. Images can be queried according to color, gray level, texture and location. The query requirements will be expressed graphically, such as selecting colors from a color table or selecting textures of images from illustrations. The query results can guide subsequent queries according to the relevant order. This method can make users screen and determine visual information more quickly and simply. Content-based image retrieval system generally includes image processing module, query module, object library, feature library and knowledge base.
2. 1 image processing module: the image processing module includes the extraction process of input images and image features.
The image input process inputs images into the system, which is similar to the text content input process in the text retrieval system. CBIR system generally allows users to segment the image in a fully automatic or semi-automatic way (requiring user intervention) and identify the key points of the required object or content, so as to extract features from the target in a targeted manner. For example, the user interface usually provides a set of examples for the user to choose from, or the user draws a sketch himself and enters it into the system.
Feature extraction performs feature extraction on image objects marked by users or systems. Feature extraction can be done by people, such as manually giving some keywords describing features, or automatically extracting some image features that retrieval users may be concerned about through corresponding image processing programs. The extracted features can be global, such as the color distribution of the whole image, or for internal local objects, such as sub-regions in the image. There are many feature representation methods, such as color histogram, color moment, color set and so on. In the color representation method, the texture representation method adopts Tramura texture feature and the texture feature representation method based on wavelet transform. However, when it comes to the advanced abstract features of images, it will be limited by knowledge fields and retrieval tasks, and often needs external knowledge to provide assistance.
2.2 Query module: The query module mainly realizes the retrieval and matching process, realizes the matching and screening of questions and records according to the correlation calculation method, and finally gets the results that meet the requirements and feeds them back to users. CBIR provides users with a retrieval interface in the form of sample query, which transforms users' retrieval requests into problems that can operate the database. Global objects (such as the whole image) are allowed to be retrieved, but sub-objects and any combination thereof are also allowed to be retrieved. The results returned by retrieval are output according to similarity, and further query can be made based on the obtained retrieval results if necessary. Like content-based retrieval, CBIR realizes similarity retrieval and imitates human cognitive process. Therefore, it is often necessary to refine the retrieval results in constant interaction with retrieval users.
2.3 Object Library and Feature Library: The object library of CBIR stores the input image resources, and the feature library contains the image features input by users and the features automatically extracted during preprocessing. Object database and feature database can realize fast search by organizing indexes matching with images, so they can be applied to the retrieval process of large-scale image databases.
2.4 Knowledge base: In CBIR system, the purpose of knowledge base is to limit the retrieval to a certain range in any domain, so as to avoid different retrieval requirements and different domain backgrounds that may lead to different requirements for the semantics of media content. Therefore, retrieval needs certain domain knowledge to improve the accuracy of retrieval.
3. Search and use
QBIC(Query By Image Content) is an image and dynamic image retrieval system developed by IBM in 1990s, which means "query according to image content" in English.
QBIC users do not need to provide text search words (of course, it also provides keyword search) in the retrieval process, but they can retrieve a series of similar images by inputting the retrieval requirements expressed in the form of images.
QBIC system provides a variety of query methods, including: using standard paradigm for retrieval, drawing sketches or scanning images, selecting colors or structures for retrieval, inputting dynamic image fragments and foreground objects for retrieval, etc.
3. 1 QBIC color search (image color retrieval)
QBIC color search finds two-dimensional artworks that match the colors you specify in the digital collection. You choose a color from the spectrum, define the scale, and then perform a search. It's really that simple. Visit QBIC Color Search Demo to see a step-by-step demonstration of the search.
3. 1. 1 Usage steps
① Select a color from the palette with the mouse.
② Click the arrow button to add color.
(3) Adjust the percentage of triangle processing on the sliding barrel, and this color.
You can repeat this process until the bucket is full. When you are ready, click Search.
3. 1.2 search interface (omitted)
3. 1.3 retrieval example
Search content: RGB = {128,255,252}, and the barrel is filled with this color.
Search results: (image omitted)
1) Plan of three burial caves in Bingema Mountain, Malta
Jean-Pierre-Laurent Huel1Late 1970s
2) The plan of the salt field on Gozo Island (the watchmaker's salt field)
Jean-Pierre-Laurent Huel1Late 1970s
3) Portrait of Yakov Knyazhnin
Stepan Filipovic Garaktinov, about 1825.
Design of Coucert Hall in Catherine Park of Tsarskoye Selo. vertical section
Quarenghi, giacomo, zip code 1780.
5) Portrait of Vasily Pra Virsh Chikov
Konstantin Jakovljevic Afanasev1the first third of the 9th century.
6) marble building fragments found in Citta Vecchia and Rabbato, Malta
Jean-Pierre-Laurent Huel1Late 1970s
Vivid pictures: memories
Julius Schopp, zip code: 1829
8) Cup design
Unknown 1900
9) The title page of K.F. Schellenberg's poem Blancheflour.
Greb, Karl george anton and Shu Ci, Ernst Friedrich Godthold, II 1854.
10) Portrait of Count Ludwig Kobenz, Austrian Ambassador to Russia
Unknown1late 8th century
1 1) shows the bucket pattern of ancient Russian warriors.
Unknown 19 10s
12) design of crystal ink bottle with silver base
Unknown 19 10s
3.2 QBIC layout search (image layout retrieval)
With QBIC page search, you become an artist. Using geometric shapes, you can arrange color areas on the virtual canvas to approximate the visual organization of the art you are searching for. Visit Q BIC Layout Search Demo to see a step-by-step demonstration of this search.
3.2. 1 Use steps
① Select a color from the palette with the mouse.
② Choose a round tool or a square tool.
③ Hold down the mouse button and drag the cross canvas to create a color shape.
④ Repeat this process until your custom layout is completed. When you are ready, click Search.
3.2.2 Search Interface (Sketch)
3.2.3 retrieval example
Search content: RGB = {128,255,252}, draw a circle.
Search results: (image omitted)
1) Internal view of the crater
Jean-Pierre-Laurent Huel1Late 1970s
2) Wedding demonstration
Heinrich urmson 193 1
3) Viewing the British Embankment and Galerny Dvor from Vasilyevski Island (Part III).
Benjamin Patterson 1799
4) Floor plan of the building foundation in Gurkinti, Malta
Jean-Pierre-Laurent Huel1Late 1970s
5) Page 8 of K.F. Scherenberg's poem "Blancheflour"
Greb, Karl george anton and Shu Ci, Ernst Friedrich Godthold, II 1854.
6) Scenery of taormina Theatre
Huel, Jean-Pierre-Laurent, between 1776 and 1779.
7) Top of Etna in the East. From San Leonardo's view
Huel, Jean-Pierre-Laurent, between 1776 and 1779.
8) British Embankment Seen from Vasilyevski Island (Part II)
Benjamin Patterson 1799
9) terrace with fountain on the boulevard. Elevation and plan
Giacomo Quarenxi 1800 in the early days.
10) Neva River embankment beside the summer garden
Unknown 1827
1 1) pine landscape on a moonlit night
Jean-Francis O 'Brzin 1890
12) Queen Elizabeth Petrovsky's Summer Palace
Grecov, Alexei Angelievich
3.3 format description
Pine landscape on a moonlit night-image name.
Auburtin, Jean-Francis 1890s- Author, creation time.
4. Analysis and evaluation
QBIC(Query By Image Content) image retrieval system is an image and dynamic scene retrieval system developed by IBM in 1990s, and it is the first content-based commercial image retrieval system. QBIC system provides a variety of query methods, including: retrieval using standard paradigm (the system comes with it), retrieval of user's sketching or scanning input images, selection of color or structure query methods, retrieval of user's input dynamic image fragments and foreground moving objects. When a user inputs an image, a sketch or a video clip, QBIC analyzes and extracts the color, texture, shape and other features of the input query image, and then performs different processing according to the query mode selected by the user. The color features used by QBIC include color percentage, color position distribution and so on. The texture feature used is an improvement of Tamura's texture representation, which integrates roughness, contrast and directionality. The shape features used are area, roundness, eccentricity, principal axis deviation and a set of algebraic moment invariants. QBIC is also one of the few systems that consider high-dimensional feature indexing.
QBIC not only carries out retrieval based on the above content characteristics, but also assists with text query means. For example, every work in the San Francisco Museum of Modern Art has standard descriptive information: author, title, date, and many works also have natural descriptions of the content.