スキーマレジストリーから情報を取得
このシナリオでは、Spark StreamingジョブでtKafkaInputAvroを使ってスキーマレジストリーからフライト情報を取得する方法を説明します。
このシナリオでは次のSpark Streamingジョブを作成します。
このシナリオを複製する前に、Kafkaシステムが稼動していること、また、使用するKafkaトピックにアクセスするための適切な権限とアクセス権があることを確認してください。
このシナリオでは、次ののAvroスキーマからデータを取得します。
{
"type": "record",
"name": "flightRecord",
"namespace": "flightInformation",
"fields": [
{
"name": "flightNumber",
"type": "string",
},
{
"name": "departure",
"type": "string",
},
{
"name": "destination",
"type": "string",
},
{
"name": "nbPassengers",
"type": "int",
},
{
"name": "aircraftSize",
"type": "string",
}
]
}このシナリオでは、次ののAvroメッセージからデータを取得します。
{"flightNumber":"OMP45","departure":"Paris","destination":"Athens","nbPassengers":120,"aircraftSize":"Medium"}
{"flightNumber":"FGH34","departure":"Paris","destination":"Oslo","nbPassengers":122,"aircraftSize":"Medium"}
{"flightNumber":"XHK20","departure":"Madrid","destination":"Buenos Aires","nbPassengers":247,"aircraftSize":"Large"}
{"flightNumber":"TUI09","departure":"Zurich","destination":"Johannesburg","nbPassengers":322,"aircraftSize":"Large"}
{"flightNumber":"CDI03","departure":"Frankfurt","destination":"New-York","nbPassengers":366,"aircraftSize":"Large"}
{"flightNumber":"JKF77","departure":"Paris","destination":"Los-Angeles","nbPassengers":380,"aircraftSize":"Large"}
{"flightNumber":"LBZ23","departure":"London","destination":"Shanghai","nbPassengers":416,"aircraftSize":"Large"}
{"flightNumber":"NSV50","departure":"London","destination":"Vienna","nbPassengers":95,"aircraftSize":"Small"}
{"flightNumber":"LRS12","departure":"Roma","destination":"Rio de Janeiro","nbPassengers":395,"aircraftSize":"Large"}
{"flightNumber":"ALJ67","departure":"Roma","destination":"Warsaw","nbPassengers":102,"aircraftSize":"Small"}
サンプルデータはあくまでも例示用です。