Skip to main content Skip to complementary content

Upgrading from version 8.0.x R2024-05 to 8.0.xR2025-02 or later

Migrating Talend Dictionary Service from 8.0.x R2024-05 to 8.0.xR2025-02 or later

No migration is required for Talend Dictionary Service, as semantic types are already stored in Cloud from 8.0.1 R2022-07.

Migrating Talend Data Preparation from 8.0.x R2024-05 to 8.0.xR2025-02 or later

Migrating Talend Data Preparation involves backing up the relevant data and database, completing the upgrade, and then updating the application properties file.

Procedure

  1. Back up the data folder and the MongoDB database.
  2. Back up the dataprep/config/application.properties file.
  3. Update the application.properties file as follows:
    1. Remove the following properties:
      # Service documentation
               service.documentation=true
               service.documentation.name=Talend Data Preparation - API
               service.documentation.description=This service exposes high level services that may involve services orchestration.
               service.paths=api
               springfox.documentation.swagger.v2.host=${public.ip}:${server.port}${gateway-api.service.path}
               springfox.resources.prefix.url=${gateway-api.service.path}
               # see https://github.com/springfox/springfox/issues/2265
               logging.level.io.swagger.models.parameters.AbstractSerializableParameter=error
      
               security.basic.enabled=false
               security.oidc.client.expectedIssuer=${iam.uri}/oidc
               iam.license.url=${iam.uri}/oidc/api
               security.oidc.client.keyUri=${iam.uri}/oidc/jwk/keys
               security.oauth2.client.clientId=<security client id>
               security.oauth2.client.clientSecret=<security client secret>
               security.oidc.client.claimIssueAtTolerance=120
               # security.oauth2.resource.serviceId=${PREFIX:}resource
               security.oauth2.resource.tokenInfoUri=${iam.uri}/oidc/oauth2/introspect
               security.oauth2.resource.uri=/v2/api-docs,/api/**,/folders/**,/datasets/**,/dataset/**,/preparations/**,/transform/**,/version/**,/acl/**,/apply/**,/export,/export/**,/aggregate,/sampling/**,/receivers/**,/error,/datastores/**,/preparation/**,/actions/**,/suggest/**,/dictionary/**,/transformation/preparations/**,/transformation/v2/**,/sharing/**
               security.oauth2.resource.filter-order=3
               security.scim.enabled=true
      
               security.oauth2.client.access-token-uri=${iam.uri}/oidc/oauth2/token
               security.oauth2.client.scope=openid refreshToken
               security.oauth2.client.user-authorization-uri=${iam.uri}/oidc/idp/authorize
               security.oauth2.sso.login-use-forward=false
      
               security.oauth2.resource.tokenInfoUriCache.enabled=true
      
               # FIXME: temporary fix related to dataquality-semantic-consumer version = 11.2.0
               # FIXME: that will need to be removed as soon as we bump to spring boot 2.7 and bump dq library at the same time
               spring.cloud.stream.binders.kafka-binder-consumer.environment.spring.cloud.stream.kafka.binder.brokers=${spring.cloud.stream.kafka.binder.brokers}
      
               # Zuul properties
               gateway-api.service.url=http://${public.ip}:${server.port}
               zuul.servletPath=/gateway/upload
      
               zuul.routes.dq.path=/gateway/dq/semanticservice/**
               zuul.routes.dq.sensitiveHeaders=${zuul.sensitiveHeaders}
               zuul.routes.dq.url=${dataquality.server.url}/
               proxy.auth.routes.dq=oauth2
      
               zuul.routes.me.path=/gateway/api/v1/scim/me/**
               zuul.routes.me.url=${iam.scim.url}/Me
               proxy.auth.routes.me=oauth2
      
               zuul.routes.pendo.path=/gateway/api/iam-server/pendo/**
               zuul.routes.pendo.url=${iam.scim.url}/pendo
               proxy.auth.routes.pendo=oauth2
      
               zuul.routes.sharingset.path=/gateway/api/v1/sharingset/**
               zuul.routes.sharingset.sensitiveHeaders=${zuul.sensitiveHeaders}
               zuul.routes.sharingset.url=http://${public.ip}:${server.port}/sharing/v1/sharingset
               proxy.auth.routes.sharingset=oauth2
      
               zuul.routes.sharing.path=/gateway/api/v1/sharing/**
               zuul.routes.sharing.sensitiveHeaders=${zuul.sensitiveHeaders}
               zuul.routes.sharing.url=http://${public.ip}:${server.port}/sharing/v1/sharing
               proxy.auth.routes.sharing=oauth2
      
               zuul.routes.sharings.path=/gateway/api/v1/sharings/**
               zuul.routes.sharings.sensitiveHeaders=${zuul.sensitiveHeaders}
               zuul.routes.sharings.url=http://${public.ip}:${server.port}/sharing/v1/sharings
               proxy.auth.routes.sharings=oauth2
      
               zuul.routes.api.path=/gateway/api/**
               zuul.routes.api.sensitiveHeaders=${zuul.sensitiveHeaders}
               zuul.routes.api.url=http://${public.ip}:${server.port}/api
               proxy.auth.routes.api=oauth2
      
               spring.cloud.gateway.mvc.routes[0].id=api
               spring.cloud.gateway.mvc.routes[0].uri=http://${public.ip}:${server.port}
               spring.cloud.gateway.mvc.routes[0].predicates[0]=Path=/gateway/api/**
               spring.cloud.gateway.mvc.routes[0].filters[0]=RewritePath=/gateway/api/(?<segment>.*), /api/$\{segment}
      
               zuul.routes.upload.path=/api/upload/**
               zuul.routes.upload.sensitiveHeaders=${zuul.sensitiveHeaders}
               zuul.routes.upload.url=http://${public.ip}:${server.port}/api
               proxy.auth.routes.upload=oauth2
      
               zuul.routes.swagger1.path=/gateway/v3/api-docs/**
               zuul.routes.swagger1.sensitiveHeaders=${zuul.sensitiveHeaders}
               zuul.routes.swagger1.url=http://${public.ip}:${server.port}/v3/api-docs
               proxy.auth.routes.swagger1=oauth2
      
               zuul.ignoredPatterns=/login,/logout,/signOut,/signIn
               zuul.sensitiveHeaders=Cookie,Set-Cookie
               zuul.host.socket-timeout-millis=300000
               zuul.host.connect-timeout-millis=5000
    2. Add the following properties:
      # Service documentation
                springdoc.server.url=${gateway-api.service.path}
                springdoc.swagger-ui.path=/swagger-ui/
                springdoc.swagger-ui.disable-swagger-default-url=true
                springdoc.swagger-ui.docExpansion=none
                springdoc.swagger-ui.operationsSorter=alpha
                springdoc.swagger-ui.tagsSorter=alpha
      
                springdoc.swagger-ui.urls[0].name=semanticservice
                springdoc.swagger-ui.urls[0].display-name=Talend Data Preparation - Semantic Service
                springdoc.swagger-ui.urls[0].url=${gateway-api.service.path}/dq/semanticservice/api-docs
      
                # Auth0 issuer uri  - Used only for daikon configuration
                spring.security.oauth2.resourceserver.auth0.jwt.issuer-uri=https://talend.auth0.com/
                # IAM authentication validation
                spring.security.oauth2.resourceserver.iam.jwt.jwk-set-uri=${iam.uri}/oidc/jwk/keys
                spring.security.oauth2.resourceserver.iam.opaque-token.introspection-uri=${iam.uri}/oidc/oauth2/introspect
                spring.security.oauth2.resourceserver.iam.opaque-token.query-entitlements=true
      
                # IAM authorization code flow
                spring.security.oauth2.client.registration.oidc.clientId=<security client id>
                spring.security.oauth2.client.registration.oidc.clientSecret=<security client secret>
                spring.security.oauth2.client.registration.oidc.authorization-grant-type=authorization_code
                spring.security.oauth2.client.registration.oidc.scope=openid,refreshToken
                spring.security.oauth2.client.registration.oidc.redirectUri=http://${public.ip}:${server.port}/signIn
                spring.security.oauth2.client.registration.oidc.provider=oidc
      
                spring.security.oauth2.client.provider.oidc.authorization-uri=${iam.uri}/oidc/idp/authorize
                spring.security.oauth2.client.provider.oidc.token-uri=${iam.uri}/oidc/oauth2/token
                spring.security.oauth2.client.provider.oidc.jwk-set-uri=${iam.uri}/oidc/jwk/keys
      
                # Kafka configuration
                spring.cloud.function.definition=internalMessage;internalMessageDatasetModule;tsdEventConsumer;tsdSemanticCacheUpdate
      
                spring.cloud.gateway.mvc.http-client.connect-timeout=5000
    3. Replace the client ID and client Secret in the new properties. You can get them from the backup file.
    4. Remove the following properties:
      ## Enable JWT needed in hybrid mode:
              security.oauth2.resource.jwt.key-uri=${iam.uri}/oidc/jwk/keys
              ## Redefine security scope by adding entitlements for hybrid
              security.oauth2.client.scope=openid refreshToken entitlements
    5. Add the scope "entitlements" to the following property: spring.security.oauth2.client.registration.oidc.scope.
      The property looks like that: spring.security.oauth2.client.registration.oidc.scope=openid,refreshToken,entitlements.

Results

You have migrated your data to Talend Data Preparation 8.0.xR2025-02 or later. To configure Talend Data Preparation, follow the instructions in Configuring Talend Data Preparation.

Migrating Talend Data Stewardship from 8.0.x R2024-05 to 8.0.xR2025-02 or later

Procedure

  1. Stop your instance of the previous version of Talend Data Stewardship.
  2. Copy install_folder/tds/config/data-stewardship.properties and paste it to a secure location.
  3. Optional: If needed, back up the Talend Data Stewardship database by running the following command:
    mongodump -h source_mongodb_host:source_mongo_port -u mongo_tds_username -p mongo_tds_password -d mongo_tds_database -o backup_location
  4. Uninstall the previous Talend Data Stewardship version (R2024-05).
  5. Install the new version of Talend Data Stewardship (R2025-02).
    Make sure to configure it with the same settings as the previous instance. If you have customized the install_folder/tds/config/data-stewardship.properties file, you can copy the values from the old configuration file to the new file.
  6. Start the new version of Talend Data Stewardship (R2025-02).
  7. Check that data is accessible from Talend Data Stewardship. If needed, you can restore the Talend Data Stewardship database by running the following command:
    mongorestore -h mongo_host:mongo_port -u mongo_tds_username -p mongo_tds_password -d mongo_tds_database backup_location

Results

You have migrated your data to Talend Data Stewardship 8.0.xR2025-02 or later. To configure Talend Data Stewardship, follow the instructions in Configuring Talend Data Stewardship.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!