Link Search Menu Expand Document

Poster

alt text

Motivation

the Wikibase frontend the Wikibase API MariaDB
manual page creation WikidataIntegrator wikibase-insert
  WikibaseIntegrator RaiseWikibase
  wikibase-cli  
  WikidataToolkit  
  Pywikibot  
  QuickStatements  
less than 1 page per second 1-6 pages per second 100-300 pages per second

RaiseWikibase

Main functions

The page function executes inserts into the database but does not commit them:

from RaiseWikibase.dbconnection import DBConnection
from RaiseWikibase.raiser import page
connection = DBConnection()
page(connection=connection, content_model=content_model,
     namespace=namespace, text=text, page_title=page_title, new=True)
connection.conn.commit()
connection.conn.close()

Multiple page functions are wrapped into a transaction inside the batch function:

from RaiseWikibase.raiser import batch
batch(content_model='wikibase-item', texts=[item for i in range(1000)])

where item is the JSON representation of an item created using the entity function.

Performance

  • The insert rate decreases approximately linearly with increasing number of characters per wikitext and with increasing number of claims per item.
  • Small pages are uploaded at rates of 250-350 wikitexts per second and 220-280 items per second.
Wikitexts Items
alt text alt text

Configuration

alt text

Federated properties

alt text

Creating a knowledge graph

alt text

Conclusions

1. Clone RaiseWikibase and adapt the files for your use case.

2. Build fast your own Wikibase knowledge graph.

3. Join the Wikibase Ecosystem.
alt text

See also

Wikibase Wikidata BERD@BW bbw QuickStart RaiseWikibase