tag : spark

Develop Spark WordCount

It is quite often to setup Apache Spark development environment through IDE. Since I do not cover much setup IDE details in my Spark course, I am here to give detail steps for developing the well know

Use UDF in Spark DataFrame

It is very convenient to create, register, and use user define functions with data. In addition, the recent release of Apache Spark also supports writing user-defined aggregation functions UDAF. Belo

Happy New Year 2016

It is the end of 2015 and HAPPY NEW YEAR - 2016. It is time to wrap up my writing calendar with some summary on Sparkera, myself, and Big Data ecosystem. In past 2015, I have published 21 articles in

Light Big Data With Apache Spark

Ten tools for ten big data areas 03_Apache Spark from Will Du Above presentation is the third topic I have covered for series of talks about the Ten Tools for Ten Big Data Areas. Apache Spark is

Apache Spark Memo

Here, I am collecting the memo while I am learning the spark so that people like me can benefit fot this collection. 1. Spark Core The stage creation rule is based on the idea to pipeline as many narr

Setup Spark in MAC

It is great to see that Brew supports install Spark. It makes installation of Spark quite easier in Mac. I just follow few steps to get my spark instance installed locally. 1. Install brew utility.123

Apache Spark Books Reviews

Apache Spark Cookbook Level Ent. Level Mid. Level Adv. I have completed the reading of this book on Dec. 2015. This is really a great a book of learning Apache Spark. It does not spend too

Moving to the Spark

It has been a while that the blog is now updated since 2014 is a ready busy year. After I almost completed my first book recently, I think it is the right time to start new journey in big data for rea