4. Datasources

4.1. Introduction

Datasources are an abstraction layer to handle different types of data storages in a type-independent way. Query by generalized rules, versioning, futurechanges, fallback, multiple servers are some feature examples. See Section 13, “Datasource” to learn about implementation of datasources.

4.2. Configuration

Datasources are configured in two steps:

  • Datasource handles define the connections to specific data repositories (e.g. ldap, sql databases, ...)

  • Datasources use one or multiple handles and define the abstract storage mechanism that portlets can use.

For details on the configuration see Section 8.6, “datasource-section”

4.3. ContentRepository

4.3.1. Introduction

A simple database scheme, used for storing content published by Gentics Content.Node and all standard modules provided by Gentics. The ContentRepository may be versioning. You don’t need to access the repository by hand. Use Datasource for reading, the AdministrationPortlet for managing the data structure and Views for visualisation and data management.

Simplified ER diagram of a contentrepository:

A detailed SQL table layout can be found below.

4.3.2. Tables

4.3.2.1. contentobject

Stores definitions of object types

Table 3.3. Columns of table contentobject

ColumnDescription
typeUnique type of this object. Gentics range is from 0 to 59999. Customertypes should have ranges of 1000 beginning from 100000. Integer, unique.
nameName of the object, preferable an i18n key, otherwise readable and displayed as-is. String, unique.
id_countercounter for object-ids (for generating unique object ids), managed by the Datasource of type contentrepository
exclude_versioningFlag to mark objecttypes that are excluded from versioning.
4.3.2.2. contentattributetype

Stores definitions of attributes for object types

Table 3.4. Columns of table contentattributetype

ColumnDescription
nameInternal name of an attribute. String, unique per object.
attributetypeType of this attribute. All attributes with same name must have same type, independent of their objecttype. See “Attribute Types” (Table 4.169) for a description of the possible attribute types.
optimizedIf this column is optimized for faster read access in the contentmap table. 1 or 0
quicknameThe name of the column in the contentmap table, where this column is stored. Prefixed with "quick_". Column must be added and indexed manually if managed manually. Gentics Content.Node; Content.Repositories do this automaticaly. 1 or 0
multivalueIf this attribute is multivalue. Multivalue attributes must not be optimized. Multivalue attributes are represented by multi contentattribute rows with an optional sortorder. 1 or 0
objecttypeWhich object this attribute belongs to. Integer, Relation to contentobject.type
linkedobjecttypeWhich object this attribute links to (attributetype 2 & 7). Integer
foreignlinkattributeIn which attribute of the foreign object, the link to the current object is stored in (attributetype 7). Name of attribute. Text
foreignlinkattributeruleOptional rule to restrict the objects related with this attribute (attributetype 7). You may use the object.attribute syntax to refer to the foreign object, but currently there is no way to access your source object. Text
exclude_versioningFlag to mark attributetypes that are excluded from versioning.
4.3.2.3. contentmap

Stores objects (one object per row)

Table 3.5. Columns of table contentmap

ColumnDescription
idUnique row id (required for versioning)
contentidContentid of the object (unique)
obj_idId of the object (unique per obj_type)
obj_typeType of the object (refers to types in table contentobject)
motheridcontentid of the mother object
mother_obj_idobject id of the mother object
mother_obj_typeType of the mother object
updatetimestampTimestamp of the last change in this row
quick_...optional quick columns (for every optimized attribute, see Section 4.3.2.2, “contentattributetype”
4.3.2.4. contentattribute

Stores attributes of objects (each row for a value of an attribute).

Table 3.6. Columns of table contentattribute

ColumnDescription
idunique row id (required for versioning)
contentidcontentid of the object this attribute belongs to
nameName of the attribute, refers to the name in table contentattributetype
value_textColumn for storing attribute values (types 1, 2 see Section 4.3.2.2, “contentattributetype” )
value_clobColumn for storing attribute values (type 5 see Section 4.3.2.2, “contentattributetype” )
value_blobColumn for storing attribute values (type 6 see Section 4.3.2.2, “contentattributetype” )
value_intColumn for storing attribute values (type 3 see Section 4.3.2.2, “contentattributetype” )
value_longColumn for storing attribute values (type 8 see Section 4.3.2.2, “contentattributetype” )
value_doubleColumn for storing attribute values (type 9 see Section 4.3.2.2, “contentattributetype” )
value_dateColumn for storing attribute values (type 10 see Section 4.3.2.2, “contentattributetype” )
SortorderSortorder for multivalue attributes
4.3.2.5. contentmap_nodeversion

Versioning table for contentmap (required when datasource is versioning).

Table 3.7. Columns of table contentmap_nodeversion

ColumnDescription
... columns of contentmapAll columns of contentmap
nodeversiontimestampTimestamp of the versioned record
nodeversion_userUser reference (who created the version)
nodeversionlatestFlag for latest (current) versions of records
nodeversionremovedTimestamp of the removal of the record
nodeversion_autoupdateFlag for future record versions that still have to ba updated automatically (when nodeversiontimestamp is reached)
4.3.2.6. contentattribute_nodeversion

Versioning table for contentattribute (required when datasource is versioning).

Table 3.8. Columns of table contentattribute_nodeversion

ColumnDescription
... columns of contentmapAll columns of contentattribute
nodeversiontimestampTimestamp of the versioned record
nodeversion_userUser reference (who created the version)
nodeversionlatestFlag for latest (current) versions of records
nodeversionremovedTimestamp of the removal of the record
nodeversion_autoupdateFlag for future record versions that still have to be updated automatically (when nodeversiontimestamp is reached)
4.3.2.7. contentstatus

This table may contain meta information about the contentrepository. Currently it is used to store the timestamp (lastupdate) when Gentics Content.Node; last synced content into the contentrepository. This is needed for datasource caching.

Table 3.9. Columns of table contentstatus

ColumnDescription
nameMetaproproperty name
intvalueValue of the metaproperty

4.3.3. Attribute Prefetching

To reduce the number of SQL Statements needed to filter objects from a content repository and access attributes of the objects, the Gentics .Node PortalConnector provides a feature called "Attribute Prefetching".

To successfully use attribute prefetching, the attributes which will be used lateron an shall be prefetched have to be provided.

  • When using the Gentics .Node PortalConnector in Java Code:

    // this is the datasource we are using
    Datasource ds = ...;
    
    // generate an expression to filter all pages (objects of type "10007")
    // from the content repository
    Expression expr =
    	ExpressionParser.getInstance().parse("object.obj_type == 10007");
    
    // now create a (reusable) datasource filter for our datasource
    DatasourceFilter filter = ds.createDatasourceFilter(expr);
    
    // now comes the magic, get the objects and have the names prefetched
    Collection objects = ds.getResult(filter, new String[] {"name"});
    								

  • When using the DatasourceQueryImp in a template:

    // set the filter rule to get all pages (objects of type "10007")
    #set($portal.imps.query.rule = "object.obj_type == 10007")
    
    // now comes the magic, have the names prefetched
    #set($portal.imps.query.prefillAttributes = ["name"])
    
    // now get the objects
    #set($pages = $portal.imps.query.result)
    								

  • When using the datasourcetreecomponent in a view:
    <datasourcetreecomponent>
       ...
       // set the root rule to get all pages (objects of type "10007")
       <rootrule>object.obj_type == 10007</rootrule>
    
       // now comes the magic, have the names prefilled
       <prefillattributes>name</prefillattributes>
    </datasourcetreecomponent>
    							

How does the attribute prefetching actually work? What about the caches? The steps to filter objects and prefetch attributes are

  1. Get the objects without any attributes. When the filtering result is found in the cache, this will not do execute any SQL statements.
  2. Calculate the maximum number of attributes that eventually need to be prefetched: [number of objects] x [number of attributes]
  3. Check whether the maximum number of attributes to prefetch exceeds the configured prefetchAttribute.threshold (which defaults to 1000 ). If that is the case, prefetch all attributes with a single (or at least few) SQL statements (algorithm finishes here).
  4. When the total number of attributes is less than the prefetchAttribute.threshold , start fetching the attributes from the cache.
  5. Calculate a threshold for cache misses (attributes not found in the cache) by taking the lesser of prefetchAttribute.cacheMissThreshold (defaults to 100 ) and the percentage given in prefetchAttribute.cacheMissThresholdPerc (defaults to 20 ) of the total number of attributes to prefetch.

    Example: 20 objects are filtered, and 2 attributes prefetched per object.

    Table 3.10. 

    Total number of attributes:20 x 2 = 40
    absolute cachemiss thresholdconfigured to 100
    relative cachemiss threshold (20%)40 x 20 / 100 = 8
    effective cachemiss thresholdmin(100, 8) = 8

  6. When more than the calculated effective cachemiss threshold attributes were not found in the cache, prefetch ALL attributes (like above). Otherwise, the attributes not found in the cache will be fetched in single SQL statements, when accessed.