Skip to main content

Migration of Service Registry Metadata after Custom Schema Update

Generally, the metadata schema is supposed to be created before any metadata are uploaded to the Service Registry. But you can still modify the original metadata schema after service metadata are uploaded to the Service Registry.

For a compatible schema change, proceed as follows:

  1. Add optional elements to the schema definition and the ElasticSearch template like the following:

    • In the <TalendRuntimePath>/etc/org.talend.esb.registry.server.metadata.xsd file, add

      ...
      <element name="CustomNewField" type="string" minOccurs="0"></element>
      ...
    • In the <TalendRuntimePath>/etc/org.talend.esb.registry.el.sync.template.json file, add

      ...
          "CustomNewField" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
      ...
  2. Copy the changed schema and ElasticSearch template to the <TalendRuntimePath>/etc folder to replace the old files.

  3. Restart the Runtime container, or at least run the following commands to restart the Service Registry:

    tesb:stop-registry
    tesb:start-registry
  4. Re-synchronize ElasticSearch with the new template:

    tregistry:sync metadata

For incompatible schema changes, a full-fledged migration is needed as described in the following steps:

  1. Export the existing metadata:

    tregistry:export metadata <PATH>/metadata_export.xml
  2. List all the metadata:

    tregistry:list metadata
  3. Delete each metadata, for example:

    tregistry:delete metadata '{http://www.talend.org/service/}DemoService'
  4. Update the schema configuration <TalendRuntimePath>/etc/org.talend.esb.registry.server.metadata.xsd and index template <TalendRuntimePath>/etc/org.talend.esb.registry.el.sync.template.json to the new files.

  5. Restart the Runtime container, or at least run the following commands to restart the Service Registry:

    tesb:stop-registry
    tesb:start-registry
  6. Re-synchronize the Service Registry to enable the new index template. This will result in dropping and recreating the index in Elasticsearch.

    tregistry:sync metadata
  7. Update the old metadata to fit the new structure in <PATH>/metadata_export.xml.

  8. Import the updated metadata:

    tregistry:import metadata <PATH>/ metadata_export.xml
  9. Check Elasticsearch for the updated data, for example:

    tregistry:search metadata '{ "query": { "term": { "metadataExtension.ServiceName.raw" : "{http://www.talend.org/service/}DemoService" }}}'

Generally, the metadata schema is supposed to be created before any metadata are uploaded to the Service Registry. But you can still modify the original metadata schema after service metadata are uploaded to the Service Registry.

For a compatible schema change, proceed as follows:

  1. Add optional elements to the schema definition and the ElasticSearch template like the following:

    • In the <TalendRuntimePath>/etc/org.talend.esb.registry.server.metadata.xsd file, add

      ...
      <element name="CustomNewField" type="string" minOccurs="0"></element>
      ...
    • In the <TalendRuntimePath>/etc/org.talend.esb.registry.el.sync.template.json file, add

      ...
          "CustomNewField" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
      ...
  2. Copy the changed schema and ElasticSearch template to the <TalendRuntimePath>/etc folder to replace the old files.

  3. Restart the Runtime container, or at least run the following commands to restart the Service Registry:

    tesb:stop-registry
    tesb:start-registry
  4. Re-synchronize ElasticSearch with the new template:

    tregistry:sync metadata

For incompatible schema changes, a full-fledged migration is needed as described in the following steps:

  1. Export the existing metadata:

    tregistry:export metadata <PATH>/metadata_export.xml
  2. List all the metadata:

    tregistry:list metadata
  3. Delete each metadata, for example:

    tregistry:delete metadata '{http://www.talend.org/service/}DemoService'
  4. Update the schema configuration <TalendRuntimePath>/etc/org.talend.esb.registry.server.metadata.xsd and index template <TalendRuntimePath>/etc/org.talend.esb.registry.el.sync.template.json to the new files.

  5. Restart the Runtime container, or at least run the following commands to restart the Service Registry:

    tesb:stop-registry
    tesb:start-registry
  6. Re-synchronize the Service Registry to enable the new index template. This will result in dropping and recreating the index in Elasticsearch.

    tregistry:sync metadata
  7. Update the old metadata to fit the new structure in <PATH>/metadata_export.xml.

  8. Import the updated metadata:

    tregistry:import metadata <PATH>/ metadata_export.xml
  9. Check Elasticsearch for the updated data, for example:

    tregistry:search metadata '{ "query": { "term": { "metadataExtension.ServiceName.raw" : "{http://www.talend.org/service/}DemoService" }}}'

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!