Artificial intelligenceCloudInterviewsOpen SourceShows

Evolution of Hadoop and Hortonworks: An Interview with Alan Gates

0

Alan Gates is one of the co-founders and architect of Hortonworks. I sat down with Gates at the DataWorks Summit (2018) to talk about a wide range of topics.

Gates touched upon the origins and evolution of Hortonworks, going all the way back to Apache Pig and Apache Hive days.

He reflected on the evolution of Hadoop and said that in the beginning, Hortonworks was trying to help out users consuming technologies like Hadoop, Pig, Hive and Spark with greater ease. Hortonworks built a platform that bundled and tested all of these technologies. But today it’s not just about managing data in a Hadoop cluster; it’s about bringing data in, streaming it, and also processing it on edge.

Now Hortonworks users are using the cloud, and they need the same capabilities. “Even people who aren’t necessarily moving their data off-prem onto the cloud, are seeing that containers are the way to effectively manage their resources, whether they are on-prem or in the cloud,” said Gates.

Hortonworks is dynamically evolving to solve these new problems for its customers.

“What we would once call Hadoop is now really a complete ecosystem. That’s why I think it’s better we call it Big Data. A few years ago we changed the name of this conference from the Hadoop Summit to the DataWorks Summit, and that really distressed that it’s not all about Hadoop anymore. That’s true about Hortonworks as well,” said Gates.

We talked about emerging use-case like IoT and Edge computing. We also talked about how Kubernetes is disrupting Big Data and how Hortonworks is embracing it.

We also discussed how Open Source has democratized software development. “Open Source is the only place where I have seen people solving their own problems instead of running to the vendor every time,” he said.

We concluded the interview by talking about Gate’s hobbies. He does have some interesting hobbies.

Watch the whole interview on YouTube & SoundCloud. Here is the timestamp.

Follow TFiR on Twitter, Facebook, YouTube and Reddit. Subscribe to our Weekly Newsletter
Swapnil Bhartiya
I have more than 12 years of experience covering Enterprise Open Source, Cloud, Containers, IoT, Machine Learning and general tech. My stories cover a very broad spectrum - traditional Linux, data center and Free Software to contemporary emerging technologies like 'serverless'. Widely Read: My stories have appeared in a multitude of leading publications including CIO, InfoWorld, Network World, The New Stack, Linux Pro Magazine, ADMIN Magazine, HPE Insights, Raspberry Pi Geek Magazine, SweetCode, Linux For You, Electronics For You and more.

Salesforce open sources TransmogrifAI

Previous article

It’s time to test RHEL 7.6 beta

Next article