DEV Community

loading...

Migrating from Apache Storm to Apache Flink

tspannhw profile image Timothy Spann Originally published at datainmotion.dev on ・3 min read

Migrating from Apache Storm to Apache Flink

The first thing you need to do is to not just pick up and dump to a new system, but to see what can be reconfigured, refactored or reimagined. For some routing, transformation or simple ingest type applications or solution parts you may want to use Apache NiFi.

For others Spark or Spark Streaming can quickly meet your needs. For simple Thing to Kafka or Kafka to Thing flows, a flow with Kafka Connect is appropriate. For things that need to run in individual devices, containers, pods you may want to move a small application to NiFi Stateless. There are also sometimes a simple Kafka Stream application will meet your needs.

For many use cases you can replace a compiled application with some solid Flink SQL code. For some discussions, check this out.

For some really good information on how to migrate Storm solutions to Flink, Cloudera has a well documented solution for you:

https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-migration-process.html

Conceptual

https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-concept.html

Architecture

https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-architecture.html

Redistribution

https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-redistribution.html

References

Discussion (0)

pic
Editor guide