DEV Community

Bisvarup Mukherjee
Bisvarup Mukherjee

Posted on

Running apache flink locally mac m1

I have an intel mac m1 air and I want to install and run apache flink locally. Instead of installing correcrt jdk version and flink version, etc. I used docker.

Step 1. Create docker network if it doesn't exist
todo: check if it exists
docker network create flink-network

Step 2. Set env variable in terminal 1
FLINK_PROPERTIES="jobmanager.rpc.address: jobmanager"

Step 3. Launch JobManager in terminal 1

 docker run \
    --rm \
    --name=jobmanager \
    --network flink-network \
    --publish 8081:8081 \
    --env FLINK_PROPERTIES="${FLINK_PROPERTIES}" \
    flink:latest jobmanager
Enter fullscreen mode Exit fullscreen mode

Step 4. Set env variable in terminal 2
FLINK_PROPERTIES="jobmanager.rpc.address: jobmanager"

Step 5. Launch TaskManager in terminal 2

docker run \
    --rm \
    --name=taskmanager \
    --network flink-network \
    --env FLINK_PROPERTIES="${FLINK_PROPERTIES}" \
    flink:latest taskmanager
Enter fullscreen mode Exit fullscreen mode

Step 6. Open localhost UI
http://localhost:8081/#/overview

Step 7. Submit a job
./bin/flink run examples/streaming/WordCount.jar
This step assumes you have downloaded flink, we are not using their runtime execution, but just their examples and submitting jobs.

Step 8. Install maven (mvn), this is needed for scaffolding some stuff later (not sure about this, todo)
brew install maven

Step 9. Scaffold sample project

 mvn archetype:generate \
    -DarchetypeGroupId=org.apache.flink \
    -DarchetypeArtifactId=flink-walkthrough-datastream-java \
    -DarchetypeVersion=1.16-SNAPSHOT \
    -DgroupId=frauddetection \
    -DartifactId=frauddetection \
    -Dversion=0.1 \
    -Dpackage=spendreport \
    -DinteractiveMode=false
Enter fullscreen mode Exit fullscreen mode

Top comments (0)