Post 52 | HDPCD | The conclusion

Hi everyone. Finally, we have reached the end of this tutorial series. It’s been so long. We started this journey together on January 15th, 2017, and, 276 days later this beautiful journey is coming to an end. But, we do not need to worry, because, I am working on something new and would love toContinue reading “Post 52 | HDPCD | The conclusion”

Post 51 | HDPCD | Set Hadoop or Hive Configuration property

Hello, everyone. Welcome to the last technical tutorial in the HDPCD certification series. It’s funny! This beautiful journey is coming to an end. In the last tutorial, we saw how to sort the output of a Hive query across multiple reducers. In this tutorial, we are going to see how to set a Hadoop or Hive configurationContinue reading “Post 51 | HDPCD | Set Hadoop or Hive Configuration property”

Post 50 | HDPCD | Order Hive query output across multiple reducers

Hello, everyone. Welcome to one more tutorial in the HDPCD certification series. In the last tutorial, we saw how to enable vectorization in Hive. In this tutorial, we are going to see how to run a subquery within a Hive query. Let us begin, then. The following infographics show the step-by-step process of performing this operation. FromContinue reading “Post 50 | HDPCD | Order Hive query output across multiple reducers”

Post 48 | HDPCD | Printing the execution plan of a Hive query

Hello, everyone. Welcome to one more tutorial in the HDPCD certification series. In the last tutorial, we saw how to enable vectorization in Hive. In this tutorial, we are going to see how to print the execution plan of a Hive query. Let us begin, then. This is one of the simplest tutorials in this certification series. InContinue reading “Post 48 | HDPCD | Printing the execution plan of a Hive query”

Post 47 | HDPCD | Run a Hive query using Vectorization

Hello, everyone. Welcome to one more tutorial in the HDPCD certification series. In the last tutorial, we saw how to run a Hive Query using TeZ execution engine. In this tutorial, we are going to see how to run a Hive Query using Vectorization. Let us begin, then. Before starting off with the objective of this tutorial, letContinue reading “Post 47 | HDPCD | Run a Hive query using Vectorization”

WordCount in Spark

Hello friends, Today we are going to implement the very famous WordCount code in Spark in spark-shell. For folks who are not familiar with WordCount, in this implementation, we count the occurrences of each word and as a result present a pair of word and their respective count. For example, if my input is asContinue reading “WordCount in Spark”

Spark 1.6.1 Installation on Ubuntu 14.04 and Hadoop 2.6.0

Hi friends, I have started learning Apache Spark and it is time now to share few things with you. In this series related to Spark and Scala, we are going to see essential things that we should know regarding both Spark and Scala. So as I always begin, we are going to start with SparkContinue reading “Spark 1.6.1 Installation on Ubuntu 14.04 and Hadoop 2.6.0”