Michio Hasai Michio Hasai is a social strategist and car guy. Find him on Facebook, Twitter, and Pinterest.

Google and Cloudera are bringing Dataflow to Spark

36 sec read

Google Cloud Dataflow allows developer to create and monitor their data-processing pipelines without the hassle of underlying clusters. Now the company is bringing this technology to Apache’s Spark data-processing engine with help from the people at Cloudera.

Google today announced that it has teamed up with the Hadoop specialists at Cloudera to bring its Cloud Dataflow programming model to Apache’s Spark data processing engine. With Google Cloud Dataflow, developers can create and monitor data processing pipelines without having to worry about the underlying data processing cluster. As Google likes to stress, the service evolved out of the company’s internal tools for processing large datasets at Internet scale. Not all data processing tasks are the same, though, and sometimes you may want to run a task in the cloud or on premise or on different processing engines. With Cloud Dataflow — in its ideal state — data analysts will be able use the same system for creating their pipelines, no matter the underlying architecture they want to run them on.

Source
Avatar of Michio Hasai
Michio Hasai Michio Hasai is a social strategist and car guy. Find him on Facebook, Twitter, and Pinterest.

Google and Huawei might be working on a new…

Google and Huawei might be joining forces once again. According to a tweet from Evan Blass, once of the most respected leakers in the...
Avatar of Brian Molidor Brian Molidor
1 min read

Google is killing off yet another thing that nobody…

Google isn’t afraid to experiment, and as a result, the company ends up having to kill off a lot of products and services that didn’t end...
Avatar of Michio Hasai Michio Hasai
1 min read

The next version of Android will be called Android…

A little more than a month after asking Android users to suggest names for the next version of Android, codenamed Android N, Google announced...
Avatar of Lorie Wimble Lorie Wimble
38 sec read

Leave a Reply

Your email address will not be published. Required fields are marked *