Post 52 | HDPCD | The conclusion

Hi everyone. Finally, we have reached the end of this tutorial series. It's been so long. We started this journey together on January 15th, 2017, and, 276 days later this beautiful journey is coming to an end. But, we do not need to worry, because, I am working on something new and would love to … Continue reading Post 52 | HDPCD | The conclusion

Post 51 | HDPCD | Set Hadoop or Hive Configuration property

Hello, everyone. Welcome to the last technical tutorial in the HDPCD certification series. It's funny! This beautiful journey is coming to an end. In the last tutorial, we saw how to sort the output of a Hive query across multiple reducers. In this tutorial, we are going to see how to set a Hadoop or Hive configuration … Continue reading Post 51 | HDPCD | Set Hadoop or Hive Configuration property

Post 50 | HDPCD | Order Hive query output across multiple reducers

Hello, everyone. Welcome to one more tutorial in the HDPCD certification series. In the last tutorial, we saw how to enable vectorization in Hive. In this tutorial, we are going to see how to run a subquery within a Hive query. Let us begin, then. The following infographics show the step-by-step process of performing this operation. From … Continue reading Post 50 | HDPCD | Order Hive query output across multiple reducers

Post 48 | HDPCD | Printing the execution plan of a Hive query

Hello, everyone. Welcome to one more tutorial in the HDPCD certification series. In the last tutorial, we saw how to enable vectorization in Hive. In this tutorial, we are going to see how to print the execution plan of a Hive query. Let us begin, then. This is one of the simplest tutorials in this certification series. In … Continue reading Post 48 | HDPCD | Printing the execution plan of a Hive query

Post 47 | HDPCD | Run a Hive query using Vectorization

Hello, everyone. Welcome to one more tutorial in the HDPCD certification series. In the last tutorial, we saw how to run a Hive Query using TeZ execution engine. In this tutorial, we are going to see how to run a Hive Query using Vectorization. Let us begin, then. Before starting off with the objective of this tutorial, let … Continue reading Post 47 | HDPCD | Run a Hive query using Vectorization

WordCount in Spark

Hello friends, Today we are going to implement the very famous WordCount code in Spark in spark-shell. For folks who are not familiar with WordCount, in this implementation, we count the occurrences of each word and as a result present a pair of word and their respective count. For example, if my input is as … Continue reading WordCount in Spark

Spark 1.6.1 Installation on Ubuntu 14.04 and Hadoop 2.6.0

Hi friends, I have started learning Apache Spark and it is time now to share few things with you. In this series related to Spark and Scala, we are going to see essential things that we should know regarding both Spark and Scala. So as I always begin, we are going to start with Spark … Continue reading Spark 1.6.1 Installation on Ubuntu 14.04 and Hadoop 2.6.0