{ "source_file": "0930", "source_platform": "Ghost", "author": "Achal Shah / In the tubes", "title_listed": "First Impressions on Apache Flink", "keywords": "", "domain": "数据处理", "content_type": "工程实践", "is_paid_flag": "否", "url": "https://blog.achals.com/first-impressions-on-apache-flink/", "publish_date": "2025-08", "corpus_stage": "", "notes": "", "id": "a45fddbd2b3b", "collected_at": "2026-10-02T12:28:12", "title": "First Impressions on Apache Flink", "body_markdown": "First Impressions on Apache Flink\nAchal Shah\n22 Aug 2025\n—\n2 min read\nShare\nAt $DAYJOB, we use a lot\nApache Flink\nfor all our stream processing needs. I've used\nApache Spark\nat previous employers and found a lot things very interesting about Flink and figured that writing down my thoughts may help me how I feel about it 😄.\nWhat is Flink?\nAs their website says:\nApache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams.\nTypically, stream processing engines are used to build near-real-time systems that can consume data that is published to streams (like Kafka) and \"do\" stuff with said data - aggregate data, enrich it, persist it, etc.\nThings I like about Flink\nFlink is built from the ground up to\nprocess streams\n. The API that it exposes, from sources to sinks, is quite\nintuitive\n.\nAt the core of the API is the concept of a\nDataStream\n. A DataStream represents an immutable collections of data that can contain duplicates. They can be bounded (e.g. if reading from a Table) or unbounded (e.g. if reading from a Stream).\nDataStreams can be created via Sources (and there's a\nrich collection\nof predefined sources already).\nKafkaSource source = KafkaSource.builder()\n .setBootstrapServers(\"localhost:9092\")\n .setTopics(\"source\")\n .setGroupId(\"my-group\")\n .setStartingOffsets(OffsetsInitializer.latest())\n .setValueOnlyDeserializer(new SimpleStringSchema())\n .build();\n\nDataStream sourceStream = env.fromSource(\n source, WatermarkStrategy.forMonotonousTimestamps(), \"Kafka Source\")\n .uid(\"kafkasourceuid\");\nWhich brings me to the first thing I really like about Flink - DataStreams are\nstrongly typed\n. Which means your code is generally much more legible on the contents of your stream, when, e.g. you're keying your streams to create \"Keyed\" Streams:\nKeyedStream keyedStream = inputStream.keyBy(event -> event.logKey())\nJoining\nstreams is also super easy. In fact, it's kind of absurd that there's such an easy way to express such a complicated operation.\n// this will join the two streams so that\n// key1 == key2 && leftTs - 2 < rightTs < leftTs + 2\nkeyedStream.intervalJoin(otherKeyedStream)\n .between(Duration.ofMillis(-2), Duration.ofMillis(2)) // lower and upper bound\n .upperBoundExclusive(true) // optional\n .lowerBoundExclusive(true) // optional\n .process(new IntervalJoinFunction() {...});\nWhat could be better:\nGenerally this is more a list of nits, but a quick list:\nFine-Grained Resource Management\nremains hard to configure and grok and scale, even though it's pretty important at scale. Tuning flink apps which use it with increasing scale is rough.\nThe API is most definitely designed for Streams and isn't great if for batch use cases. This is kind of in contrast with Spark which is a Batch API that is not well suited for Streaming use cases.\nWe're currently deploying Flink using the\nKubernetes Operator\n- which makes deployment of apps easy, but doesn't have blue-green deployments implemented yet. This is on\ntheir roadmap\n, but results in extended downtime when you're dealing with apps with thousands of Task Managers.", "metadata": { "title": "First Impressions on Apache Flink", "description": null, "keywords": null, "author": null, "og:site_name": "In the tubes", "og:type": "article", "og:title": "First Impressions on Apache Flink", "og:description": "At $DAYJOB, we use a lot Apache Flink for all our stream processing needs. I've used Apache Spark at previous employers and found a lot things very interesting about Flink and figured that writing down my thoughts may help me how I feel about it 😄.\n\n\nWhat is Flink?\n\nAs", "og:url": "https://blog.achals.com/first-impressions-on-apache-flink/", "og:image": "https://static.ghost.org/v5.0.0/images/publication-cover.jpg", "og:image:width": "1200", "og:image:height": "840", "twitter:card": "summary_large_image", "twitter:title": "First Impressions on Apache Flink", "twitter:description": "At $DAYJOB, we use a lot Apache Flink for all our stream processing needs. I've used Apache Spark at previous employers and found a lot things very interesting about Flink and figured that writing down my thoughts may help me how I feel about it 😄.\n\n\nWhat is Flink?\n\nAs", "twitter:url": "https://blog.achals.com/first-impressions-on-apache-flink/", "twitter:image": "https://static.ghost.org/v5.0.0/images/publication-cover.jpg", "twitter:label1": "Written by", "twitter:data1": "Achal Shah", "article:published_time": "2025-08-22T07:01:57.000Z", "article:modified_time": "2025-08-22T07:01:57.000Z" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/a45fddbd2b3b/code/body-000.txt", "chars": 433, "language": "text" }, { "file": "attachments/a45fddbd2b3b/code/body-001.txt", "chars": 85, "language": "text" }, { "file": "attachments/a45fddbd2b3b/code/body-002.txt", "chars": 354, "language": "text" } ], "images": [], "audio": [], "videos": [], "links": [ { "href": "https://blog.achals.com/author/achal/", "label": "", "type": "a" }, { "href": "https://flink.apache.org/?ref=blog.achals.com", "label": "Apache Flink", "type": "a" }, { "href": "https://spark.apache.org/?ref=blog.achals.com", "label": "Apache Spark", "type": "a" }, { "href": "https://nightlies.apache.org/flink/flink-docs-release-2.1/docs/dev/datastream/overview/?ref=blog.achals.com", "label": "DataStream", "type": "a" }, { "href": "https://nightlies.apache.org/flink/flink-docs-release-2.1/docs/connectors/datastream/overview/?ref=blog.achals.com", "label": "rich collection", "type": "a" }, { "href": "https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/finegrained_resource/?ref=blog.achals.com", "label": "Fine-Grained Resource Management", "type": "a" }, { "href": "https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/overview/?ref=blog.achals.com", "label": "Kubernetes Operator", "type": "a" }, { "href": "https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=337677648&ref=blog.achals.com", "label": "their roadmap", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:47:49Z", "source_url": "https://blog.achals.com/first-impressions-on-apache-flink/", "papers": [], "actual_video_files": [], "body_chars": 3198, "expected_body_images": 0, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/a45fddbd2b3b/pages/source.html", "local_html": "attachments/a45fddbd2b3b/pages/article.html", "body_base_path": "attachments/a45fddbd2b3b/pages/", "body_markdown_path": "attachments/a45fddbd2b3b/pages/article.md" }