Relative Content

Tag Archive for javaapache-flinkflink-streaming

Dependency requirement for local execution of Flink Applications

Finding the steps to create a local deployment for starting Flink jobs in https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/datastream/overview/#local-execution-environment, shows a code snippet of how one would create a local test env for quick debugging, I recreated the instructions and attempted to execute by issuing mvn clean compile exec:java -Dexec.mainClass="com.test.RunFlink" -Dexec.classpathScope="provided" since normal Flink dependencies are marked provided, however, I am always getting the error

Flink task manager does not unload classes

I have Standalone Flink Cluster. When I stop the process on Task Manager, classes that were loaded as ChildFirst are not removed. After several start/stop repetitions, the metaspace exceeds the maximum value and throws OutOfMemory Metaspace.
I run servals application in one task manager. Application are different so a cannot add the JAR file to /lib folder. Restarting TaskManager after severals operation is bothersome.

Intermittent High Latency in Flink

I am working with an Apache Flink application that reads data streams from Kafka. The application processes the streams, creating objects from the data and calculating the processing time by marking the start time at object creation and the end time after performing a series of transformations including keyBy and some calculations.