A Few Streams to Learn From: Apacha Kafka!
Here I am in the middle of Qcon SF, about to enjoy the “Demystifying Stream Processing with Apache Kafka“ talk with Neha Narkhede @nehanarkhede. The background on this talk is rooted in Neha being a co-founder of Confluent.io, with co-founder Jay Kreps of Karka co-creation fame. Neha is providing a fundamental talk on the insight and usage of streams across distributed systems.
#Qcon "Demystifying Stream Processing w/ Apache Kafka" w/ @nehanarkhede pic.twitter.com/zgR2H87xVn
— Λdrøn (@Adron) November 18, 2015
Holy smokes the distributed systems track is CRAZY packed!! #qconsf pic.twitter.com/tVmUxExk0j
— Λdrøn (@Adron) November 18, 2015
That second tweet was of the room before we had to move to the keynote space to make room for everybody that was interested in the topic! Holy snikies!
If you’d like to read some more information on Kafka and streaming, check out some of these posts.
- Distributed Consensus Reloaded: Apache Zookeeper and Replication in Kafka by Neha Narkhede & Flavio Jungueira
- Spark + Kafka Streaming Integration (from Docs)
- The Kafka Project
- Getting the Code
- A crazy awesome list of papers & related material for those that love papers and want to read papers on Kafka. ;)
- Follow the Kafka project on Twitter, you should be on twitter, for professional and time killing reasons, because… just follow the account if you really want to keep up with the latest.
- Putting Apache Kafka to Use: A Practical Guide to Building a Streaming Data Platform Part 1 and Part 2.
- More on Spark and Kafka integration.
Visual Studio Code goes OSS & more Wicked F#!
As I’m sitting listening to Neha’s talk I see a stream (because I multi-task like a crazy person) of things getting mentioned about something something OSS and something something F# and something something Visual Studio Code. So even though we’re heavy into the middle of compaction, stream processings, discussions of queue points and how to manage so many things Kafka using a library with kstream DSL, processor API, and interfaces in a library Neha is discussing. It’s very interesting soI’m going back and forth between what Neha is talking about, taking notes on the specific topic points I’ll need to research after her talk, and reading up on these something something OSS something something F# somethign something Visual Studio Code tweets. Then I stumbled into the rabbit hole of goodies that I was seeing…
Visual Studio Code is OSS now w/ F# Goodies!At least, that’s my first response because this fixes my #1 complaint about Visual Studio Code. I hated that it wasn’t open source, when there was very little reason for it to be closed source. So much of it was open already, it just seemed confusingly absurd that it wasn’t open source. But here it is, wide open and ready for PRs yo! The Marketplace for Visual Studio now has a few new goodies for F# too which is excellent!HELL YEAH!
Most of this is mentioned on the Visual Studio Code blog of course, but I’m outlining a few of the bits here, since I know not too many follow the VSC blog that read my blog - for various good reasons! ;)
With that, I leave you with the two key tidbits that worked their way into my brain while I enjoyed learning about Kafka in Neha’s talk. Cheers!
